Water meter dial multi-mode identification method and system based on AI vision

Abstract

The invention relates to the technical field of water meter dial recognition, in particular to a water meter dial multi-mode recognition method and system based on AI vision. An Ai visual analysis and identification module automatically positions a water meter number area, a digital flow area, a bar code area and a pointer dial area in a collected water meter image based on a constructed Ai visual analysis model, and respectively obtains a corresponding water meter number, digital flow and pointer flow in the collected water meter image based on an image identification technology; and the obtained image recognition result is sent to a production test end according to the confidence coefficient corresponding to each recognition result. Overall image recognition and Ai vision algorithms are adopted for the dial plate, the method is not sensitive to layout arrangement in the dial plate, and the universality of recognition of various water meter dial plates is greatly improved; according to the invention, zero detection is carried out on the sub-dial plate before identification, the completion quality of the previous process is indirectly checked, and the accuracy of subsequent identification is ensured at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of water meter dial recognition, and in particular to an AI vision-based multimodal recognition method and system for water meter dials. Background Art

[0002] There are two main types of OCR recognition technologies currently used in water meter dials (hereinafter referred to as "dials"): one is used to recognize the text area of ​​the dial, directly recognizing the numbers on the dial wheel without recognizing the sub-dial pointer; the other is to directly recognize the plum blossom pointer, set a speed recognition device above the plum blossom pointer, use laser scanning to calculate the rotation of the plum blossom pointer, and then calculate the final value based on the speed ratio of the plum blossom pointer, sub-dial and dial wheel.

[0003] In the first prior art, OCR recognition technology is primarily used for text area recognition, and OCR recognition or similar technologies are not used for sub-dials. This is because the pointers on the sub-dials are randomly installed, making it impossible to directly identify them based on the scanned pointer position. As a result, the sub-dial pointer position cannot be accurately determined. The numeric display of the wheel in the text area is related to the sub-dial pointer position. When the numeric display of the wheel is incomplete, missing strokes, or otherwise abnormal, the sub-dial pointer position cannot be used for auxiliary recognition. As a result, the overall OCR recognition rate of the text area has certain flaws, and the overall recognition rate is below 95%. The second prior art relies heavily on the position of the dial's plum blossom pointer, which lacks versatility. The plum blossom pointer recognition scheme requires the recognition sensor to be precisely positioned above the plum blossom pointer, with certain sensing distance requirements and certain contrast requirements between the plum blossom pointer and the relevant identification background. When using a different type of water meter head, if the plum blossom pointer position moves, the sensor position of the test device must also be changed. If the recognition conditions (such as background contrast) are not met, recognition fails. Therefore, both of the above methods have significant drawbacks. Summary of the Invention

[0004] The purpose of the present invention is to provide a water meter dial multimodal recognition method and system based on AI vision to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solution: a multimodal recognition method for water meter dial based on AI vision, the method comprising: S1. Place the produced water meter on a water meter fixture and collect an image of the water meter on the water meter fixture using a camera; S2. Automatically locate the water meter number area, digital flow area, barcode area, and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, obtain the corresponding water meter number, digital flow, pointer flow, and the confidence level corresponding to each recognition result in the collected water meter image, and send the obtained image recognition results to the production test end. S3. Based on the confidence levels corresponding to the respective recognition results, the production test end determines the recognition operation process. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1, where the water meter is fixed, image captured, and recognized again. Before jumping to S1, the water meter image recognition result and the corresponding water meter image capture result are sent to the manual error correction queue, where they await manual error correction by the corresponding administrator. S4. When the corresponding confidence levels of the water meter image recognition results are greater than or equal to the preset values, the water meter numbers in the water meter image recognition results are called for duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

[0006] Furthermore, in the process of automatically locating the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model, the S2 obtains the minimum rectangular frame area corresponding to the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data, and obtains the image texture features based on the corresponding recognition area type in each minimum rectangular frame area, the recognition area types including water meter number area type, digital flow area type, barcode area type and pointer dial area type; the image texture features based on the corresponding recognition area type in each minimum rectangular frame area are obtained. The physical feature represents the pixel matrix after the corresponding minimum rectangular frame area image is binarized, each pixel in the area image corresponds to an element in the pixel matrix, the elements in the pixel matrix are the binarization results of the grayscale values ​​corresponding to the corresponding pixels, and the positional relationship between the pixels corresponding to different elements in the pixel matrix is ​​the same as the positional relationship between the corresponding pixels in the corresponding area image; the Ai visual analysis model is a summary result of the image texture features of the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data based on the corresponding recognition area type; Call the rectangular frame specifications corresponding to each recognition area type in the constructed Ai visual analysis model, and translate the rectangular frame of the corresponding specifications, and extract the image area within the rectangular frame of the corresponding specifications in the collected water meter image in real time, and record it as the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position; calculate the similarity between the image texture features corresponding to the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position and the image texture features corresponding to each element in the corresponding recognition area type in the constructed Ai visual analysis model, and take the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position when the similarity is greater than the preset similarity value and the similarity is the largest as the image area corresponding to the corresponding recognition area type; the similarity between the image texture features corresponding to the two area images is equal to the ratio of the number of elements with the same element value at the same element position in the pixel point matrix corresponding to the corresponding image texture feature to the total number of elements in the pixel point matrix.

[0007] Furthermore, the specific steps of obtaining the corresponding water meter number in the collected water meter image based on image recognition technology in S2 are as follows: S211, extracting the water meter number area in the collected water meter image; S212: Perform text recognition on the water meter number area in the collected water meter image based on OCR technology to obtain a first water meter number value; S213, extracting the barcode area in the collected water meter image, and binarizing the grayscale value of the barcode area image to obtain a binary barcode image; S214: Based on preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain a barcode value, which is recorded as the second water meter number value; S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value is the same as the second water meter number value, determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number recognition result is 1; if the first water meter number value is different from the second water meter number value, determine that the confidence level of the water meter number recognition result is 0.

[0008] Furthermore, the specific steps of obtaining the corresponding digital flow and pointer flow in the collected water meter image based on image recognition technology in S2 are as follows: S221, extracting digital flow areas and pointer flow areas from the collected water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials; S222. Locate and identify the scale values ​​in the digital flow area on the dial using OCR technology, and use the Hough line detection operator to detect the linear scale in the digital flow area. The intersection of the linear scale and the arc contour of the sub-dial is recorded as the scale point of the corresponding linear scale; each scale point is bound to a scale value; The present invention uses the Hough line detection operator to detect the linear scale in the digital flow area, which includes the following specific implementation steps: 1) acquiring an image and locating the ROI, that is, reading the original image of the water meter disk, determining the coordinates of the scale value area frame through target detection / template matching / contour analysis, obtaining the coordinates of the rectangular area, and then extracting the ROI area (region of interest); 2) image preprocessing, that is, performing grayscale conversion and Gaussian blur noise reduction, and performing Canny edge detection. By adjusting the parameters, the scale lines are cleaned and visible, and the noise is minimized; 3) performing Hough line detection, that is, performing probabilistic Hough transform; and filtering the Hough transform results through angle filtering (adjusting the angle range according to the actual scale direction, such as 75 to 105 degrees for the vertical scale) and length filtering; 4) visualizing the results, that is, drawing the detection results on the ROI and superimposing them on the full image.

[0009] S223, calculating the center point of each sub-dial, and converting each scale point into polar coordinates based on the corresponding disk center point; S224, extracting the pointer vertex position and converting the pointer vertex position into polar coordinates based on the center point of the corresponding disk; S225: Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale points to identify the scale range in which the pointer vertex position is located, and predict the scale value of the corresponding pointer vertex based on the ratio of the arc formed by the pointer vertex position, the center point of the corresponding disk, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints in the scale range in which the pointer vertex position is located and the center point of the corresponding disk; S226, identifying the pointer scale of each sub-dial in turn, summing up the scale values ​​of the pointer vertices in each sub-dial to obtain a final dial value; The corresponding digital flow confidence in the collected water meter image is 1; in the most recent N water meter dial recognition results in the historical data, the average value of the ratio of the absolute value of the difference between the dial value recognized by each water meter and the actual dial value to the corresponding actual dial value is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow confidence.

[0010] Furthermore, during the process of determining the recognition operation process at the production test end in S3, if the confidence level corresponding to the water meter image recognition result is greater than or equal to a preset value, the process jumps to S4; When the confidence level of the water meter image recognition result is less than a preset value, the process jumps to S1. During the process of re-fixing the water meter and collecting and recognizing the image, if the water meter fixing, collecting and recognizing operations are repeated for the same water meter for more than M times, the fixing, collecting and recognizing operations of the corresponding water meter dial are stopped, and the recognition operation process of the corresponding water meter dial is terminated; M is a preset constant. The present invention sets the rule "If the meter fixation, image acquisition, and recognition operations are repeated for the same water meter more than M times, the meter fixation, acquisition, and recognition operations for the corresponding water meter dial are stopped, and the recognition operation process for the corresponding water meter dial is terminated" to prevent steps S1-S3 from falling into an infinite loop. By setting the value M, the maximum number of times step S3 can execute the fixation, acquisition, and recognition operations for the same water meter dial is determined. In the process of waiting for the corresponding administrator to perform manual error correction in S3, if the water meter is fixed and image captured and recognized again, and the confidence levels corresponding to the recognition results obtained are greater than or equal to the preset values, the corresponding water meter image recognition results and the corresponding water meter image capture results sent to the manual error correction processing sequence will be deleted.

[0011] Furthermore, in the process of managing the image recognition results of the corresponding water meter in combination with the duplicate number detection results, when the historical water meter image recognition result data contains the water meter number in the called water meter image recognition result, it is determined that the water meter number in the called water meter image recognition result has a duplicate number, and the recognition operation process of the corresponding water meter dial is stopped; otherwise, the dial value recognized by the corresponding water meter dial is bound to the corresponding water meter dial and saved in the database.

[0012] A water meter dial multimodal recognition system based on AI vision, the system comprising: The water meter fixing and image acquisition module is used to place the produced water meter on the water meter fixing device and collect the water meter image on the water meter fixing device through the camera; The AI ​​visual analysis and recognition module automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, it obtains the corresponding water meter number, digital flow area, pointer flow area, and the confidence level corresponding to each recognition result in the collected water meter image, and sends the obtained image recognition results to the production test end; The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the module jumps to the water meter fixation and image acquisition module to fix the water meter again, collect images, and identify the water meter. Before jumping to the water meter fixation and image acquisition module, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction queue for manual error correction by the corresponding administrator. The duplicate number detection management module is used to call the water meter number in the water meter image recognition result for duplicate number detection when the corresponding confidence level of the water meter image recognition result is greater than or equal to the preset value, and manage the image recognition results of the corresponding water meter in combination with the duplicate number detection result.

[0013] Furthermore, the Ai visual analysis and recognition module includes a regional image positioning unit, a regional image parameter recognition unit and an image recognition result transmission unit. The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model; The regional image parameter recognition unit obtains the corresponding water meter number, digital flow, pointer flow, and confidence level of each recognition result in the collected water meter image based on image recognition technology; The image recognition result transmission unit sends the obtained image recognition result to the production test end.

[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention adopts overall image recognition and AI vision algorithm for the dial, which is insensitive to the layout arrangement inside the dial (such as the position of the plum blossom pointer), greatly improving the versatility of recognition of various water meter dials; (2) The present invention performs a zeroing check on the sub-dial before recognition (checking whether the pointer of each sub-dial is zeroed). If it is not zeroed, it needs to be reset by the previous process. The present invention indirectly checks the completion quality of the previous process and ensures the accuracy of subsequent recognition. At the same time, it can save one detection station and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 Schematic diagram of the structure of the water meter dial multimodal recognition system based on AI vision of the present invention; Figure 21. It is a flow chart of the multimodal recognition method of water meter dial based on AI vision of the present invention; Figure 3 This is a first schematic diagram of an embodiment of the multimodal recognition method for a water meter dial based on AI vision of the present invention; Figure 4 This is a second schematic diagram of an embodiment of the multimodal recognition method for a water meter dial based on AI vision of the present invention; Figure 5 This is the third schematic diagram of an embodiment of the multimodal recognition method for water meter dial based on AI vision of the present invention. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0017] See also Figure 1-Figure 2 , the present invention provides a technical solution: Figure 1 As shown, this embodiment provides a water meter dial multimodal recognition system based on AI vision, and the system includes: The water meter fixing and image acquisition module is used to place the produced water meter on the water meter fixing device and collect the water meter image on the water meter fixing device through the camera; Ai visual analysis and recognition module, the Ai visual analysis and recognition module includes a regional image positioning unit, a regional image parameter recognition unit and an image recognition result transmission unit, The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model; The regional image parameter recognition unit obtains the corresponding water meter number, digital flow, pointer flow, and confidence level of each recognition result in the collected water meter image based on image recognition technology; The image recognition result transmission unit sends the obtained image recognition result to the production test end; The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the module jumps to the water meter fixation and image acquisition module to fix the water meter again, collect images, and identify the water meter. Before jumping to the water meter fixation and image acquisition module, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction queue for manual error correction by the corresponding administrator. The duplicate number detection management module is used to call the water meter number in the water meter image recognition result for duplicate number detection when the corresponding confidence level of the water meter image recognition result is greater than or equal to the preset value, and manage the image recognition results of the corresponding water meter in combination with the duplicate number detection result.

[0018] like Figure 2 As shown, this embodiment provides a multimodal recognition method for a water meter dial based on AI vision, the method comprising: S1. Place the produced water meter on a water meter fixture and collect an image of the water meter on the water meter fixture using a camera; S2. Automatically locate the water meter number area, digital flow area, barcode area, and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, obtain the corresponding water meter number, digital flow, pointer flow, and the confidence level corresponding to each recognition result in the collected water meter image, and send the obtained image recognition results to the production test end. In the process of automatically locating the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model, the S2 obtains the minimum rectangular frame area corresponding to the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data, and obtains the image texture features based on the corresponding identification area type in each minimum rectangular frame area, the identification area types include water meter number area type, digital flow area type, barcode area type and pointer dial area type; the image texture features represent the pixel matrix after binarization processing of the corresponding minimum rectangular frame area image, each pixel in the area image corresponds to an element in the pixel matrix, the elements in the pixel matrix are the binarization results of the grayscale values ​​corresponding to the corresponding pixels, and the positional relationship between the pixels corresponding to different elements in the pixel matrix is ​​the same as the positional relationship between the corresponding pixels in the corresponding area image; the Ai visual analysis model is a summary result of the image texture features based on the corresponding identification area type for the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data; Call the rectangular frame specifications corresponding to each recognition area type in the constructed Ai visual analysis model, and translate the rectangular frame of the corresponding specifications, and extract the image area within the rectangular frame of the corresponding specifications in the collected water meter image in real time, and record it as the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position; calculate the similarity between the image texture features corresponding to the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position and the image texture features corresponding to each element in the corresponding recognition area type in the constructed Ai visual analysis model, and take the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position when the similarity is greater than the preset similarity value and the similarity is the largest as the image area corresponding to the corresponding recognition area type; the similarity between the image texture features corresponding to the two area images is equal to the ratio of the number of elements with the same element value at the same element position in the pixel point matrix corresponding to the corresponding image texture feature to the total number of elements in the pixel point matrix.

[0019] The specific steps of obtaining the corresponding water meter number in the collected water meter image based on image recognition technology in S2 are as follows: S211, extracting the water meter number area in the collected water meter image; S212: Perform text recognition on the water meter number area in the collected water meter image based on OCR technology to obtain a first water meter number value; S213, extracting the barcode area in the collected water meter image, and binarizing the grayscale value of the barcode area image to obtain a binary barcode image; S214: Based on preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain a barcode value, which is recorded as the second water meter number value; S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value is the same as the second water meter number value, determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number recognition result is 1; if the first water meter number value is different from the second water meter number value, determine that the confidence level of the water meter number recognition result is 0.

[0020] The specific steps of obtaining the corresponding digital flow and pointer flow in the collected water meter image based on image recognition technology in S2 are as follows: S221, extracting digital flow areas and pointer flow areas from the collected water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials; S222. Locate and identify the scale values ​​in the digital flow area on the dial using OCR technology, and use the Hough line detection operator to detect the linear scale in the digital flow area. The intersection of the linear scale and the arc contour of the sub-dial is recorded as the scale point of the corresponding linear scale; each scale point is bound to a scale value; S223, calculating the center point of each sub-dial, and converting each scale point into polar coordinates based on the corresponding disk center point; S224, extracting the pointer vertex position and converting the pointer vertex position into polar coordinates based on the center point of the corresponding disk; S225: Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale points to identify the scale range in which the pointer vertex position is located, and predict the scale value of the corresponding pointer vertex based on the ratio of the arc formed by the pointer vertex position, the center point of the corresponding disk, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints in the scale range in which the pointer vertex position is located and the center point of the corresponding disk; S226, identifying the pointer scale of each sub-dial in turn, summing up the scale values ​​of the pointer vertices in each sub-dial to obtain a final dial value; The corresponding digital flow confidence in the collected water meter image is 1; in the most recent N water meter dial recognition results in the historical data, the average value of the ratio of the absolute value of the difference between the dial value recognized by each water meter and the actual dial value to the corresponding actual dial value is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow confidence.

[0021] S3. Based on the confidence levels corresponding to the respective recognition results, the production test end determines the recognition operation process. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1, where the water meter is fixed, image captured, and recognized again. Before jumping to S1, the water meter image recognition result and the corresponding water meter image capture result are sent to the manual error correction queue, where they await manual error correction by the corresponding administrator. During the process of determining the recognition operation process at the production test end in S3, if the confidence level corresponding to the water meter image recognition result is greater than or equal to the preset value, the process jumps to S4; When the confidence level of the water meter image recognition result is less than a preset value, the process jumps to S1. During the process of re-fixing the water meter and collecting and recognizing the image, if the water meter fixing, collecting and recognizing operations are repeated for the same water meter for more than M times, the fixing, collecting and recognizing operations of the corresponding water meter dial are stopped, and the recognition operation process of the corresponding water meter dial is terminated; M is a preset constant. In the process of waiting for the corresponding administrator to perform manual error correction in S3, if the water meter is fixed and image captured and recognized again, and the confidence levels corresponding to the recognition results obtained are greater than or equal to the preset values, the corresponding water meter image recognition results and the corresponding water meter image capture results sent to the manual error correction processing sequence will be deleted.

[0022] S4. When the corresponding confidence levels of the water meter image recognition results are greater than or equal to the preset values, the water meter numbers in the water meter image recognition results are called for duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

[0023] In the process of managing the image recognition results of the corresponding water meter in combination with the duplicate number detection results, S4, when the historical water meter image recognition result data contains the water meter number in the called water meter image recognition result, it is determined that the water meter number in the called water meter image recognition result has a duplicate number phenomenon, and the recognition operation process of the corresponding water meter dial is stopped; otherwise, the dial value recognized by the corresponding water meter dial is bound to the corresponding water meter dial and saved in the database.

[0024] In this embodiment, the pointer on the dial is read and recognized, and the recognition logic is as follows: Step 1: There are four subdials on the instrument (0.1 liter, 0.01 liter, 0.001 liter, and 0.0001 liter). First, use circular contour detection to extract and locate the images of the four dial areas.

[0025] Step 2: Perform regional positioning and numerical recognition on the scale values ​​on the dial, and use the Huffman line detection operator to detect the straight line scale within the scale value area frame. The intersection of the straight line and the arc outline of the sub-dial is the scale point.

[0026] Step 3: Calculate the center point of each sub-dial and convert each scale point into polar coordinates based on the center point.

[0027] Step 4: Detect key points of the pointer vertex and convert the vertex into polar coordinates based on the center point of the disk.

[0028] Step 5: Compare the polar coordinates of the pointer vertex with the polar coordinates of the scale to calculate the scale range of the pointer vertex. Based on the relative proportions, the scale value of the pointer vertex can be estimated.

[0029] Step 6: Identify the pointer scales of the four dials in turn, and sum up the scale values ​​of the four dials to obtain the final dial value.

[0030] The accuracy of pointer reading recognition in this embodiment is very high. Except for the last digit, the accuracy of pointer recognition is 100%. For the recognition of the last digit pointer (0.0001 digit), the estimation method is adopted in this embodiment. First, it is determined which scale value the pointer is close to. If it can be clearly recognized, the nearest scale value is used as the basis. Figure 3 、 Figure 4As shown, when the pointer falls between two adjacent graduation values, Ai can identify the position through camera comparison and then determine the reading.

[0031] like Figure 5 As shown, for the pointer in the circled box, Ai can recognize that it is between 1 and 2, and then based on the judgment of the pointer at the 0.001 liter position below, it can basically determine its corresponding position, making the pointer recognition accuracy for the 0.01 liter position 100%.

[0032] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0033] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. The multimodal recognition method of water meter dial based on AI vision is characterized by: The method comprises: S1. Place the produced water meter on a water meter fixture and collect an image of the water meter on the water meter fixture using a camera; S2. Automatically locate the water meter number area, digital flow area, barcode area, and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, obtain the corresponding water meter number, digital flow, pointer flow, and the confidence level corresponding to each recognition result in the collected water meter image, and send the obtained image recognition results to the production test end. S3. Based on the confidence levels corresponding to the respective recognition results, the production test end determines the recognition operation process. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1, where the water meter is fixed, image captured, and recognized again. Before jumping to S1, the water meter image recognition result and the corresponding water meter image capture result are sent to the manual error correction queue, where they await manual error correction by the corresponding administrator. S4. When the corresponding confidence levels of the water meter image recognition results are greater than or equal to the preset values, the water meter numbers in the water meter image recognition results are called for duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

2. The AI ​​vision-based multimodal water meter dial recognition method according to claim 1 is characterized by: In the process of automatically locating the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model, the S2 obtains the minimum rectangular frame area corresponding to the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data, and obtains the image texture features based on the corresponding identification area type in each minimum rectangular frame area, the identification area types include water meter number area type, digital flow area type, barcode area type and pointer dial area type; the image texture features represent the pixel matrix after binarization processing of the corresponding minimum rectangular frame area image, each pixel in the area image corresponds to an element in the pixel matrix, the elements in the pixel matrix are the binarization results of the grayscale values ​​corresponding to the corresponding pixels, and the positional relationship between the pixels corresponding to different elements in the pixel matrix is ​​the same as the positional relationship between the corresponding pixels in the corresponding area image; the Ai visual analysis model is a summary result of the image texture features based on the corresponding identification area type for the water meter number area, digital flow area, barcode area and pointer dial area in each water meter image in the historical data; Call the rectangular frame specifications corresponding to each recognition area type in the constructed Ai visual analysis model, and translate the rectangular frame of the corresponding specifications, and extract the image area within the rectangular frame of the corresponding specifications in the collected water meter image in real time, and record it as the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position; calculate the similarity between the image texture features corresponding to the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position and the image texture features corresponding to each element in the corresponding recognition area type in the constructed Ai visual analysis model, and take the rectangular frame image area of ​​the corresponding specifications at the corresponding translation position when the similarity is greater than the preset similarity value and the similarity is the largest as the image area corresponding to the corresponding recognition area type; the similarity between the image texture features corresponding to the two area images is equal to the ratio of the number of elements with the same element value at the same element position in the pixel point matrix corresponding to the corresponding image texture feature to the total number of elements in the pixel point matrix.

3. The AI ​​vision-based multimodal water meter dial recognition method according to claim 2 is characterized by: The specific steps of obtaining the corresponding water meter number in the collected water meter image based on image recognition technology in S2 are as follows: S211, extracting the water meter number area in the collected water meter image; S212: Perform text recognition on the water meter number area in the collected water meter image based on OCR technology to obtain a first water meter number value; S213, extracting the barcode area in the collected water meter image, and binarizing the grayscale value of the barcode area image to obtain a binary barcode image; S214: Based on preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain a barcode value, which is recorded as the second water meter number value; S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value is the same as the second water meter number value, determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number recognition result is 1; if the first water meter number value is different from the second water meter number value, determine that the confidence level of the water meter number recognition result is 0.

4. The AI ​​vision-based multimodal water meter dial recognition method according to claim 2, characterized in that: The specific steps of obtaining the corresponding digital flow and pointer flow in the collected water meter image based on image recognition technology in S2 are as follows: S221, extracting digital flow areas and pointer flow areas from the collected water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials; S222. Locate and identify the scale values ​​in the digital flow area on the dial using OCR technology, and use the Hough line detection operator to detect the linear scale in the digital flow area. The intersection of the linear scale and the arc contour of the sub-dial is recorded as the scale point of the corresponding linear scale; each scale point is bound to a scale value; S223, calculating the center point of each sub-dial, and converting each scale point into polar coordinates based on the corresponding disk center point; S224, extracting the pointer vertex position and converting the pointer vertex position into polar coordinates based on the center point of the corresponding disk; S225: Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale points to identify the scale range in which the pointer vertex position is located, and predict the scale value of the corresponding pointer vertex based on the ratio of the arc formed by the pointer vertex position, the center point of the corresponding disk, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints in the scale range in which the pointer vertex position is located and the center point of the corresponding disk; S226, identifying the pointer scale of each sub-dial in turn, summing up the scale values ​​of the pointer vertices in each sub-dial to obtain a final dial value; The corresponding digital flow confidence in the collected water meter image is 1; in the most recent N water meter dial recognition results in the historical data, the average value of the ratio of the absolute value of the difference between the dial value recognized by each water meter and the actual dial value to the corresponding actual dial value is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow confidence.

5. The AI ​​vision-based multimodal water meter dial recognition method according to claim 1, characterized in that: During the process of determining the recognition operation process at the production test end in S3, if the confidence level corresponding to the water meter image recognition result is greater than or equal to the preset value, the process jumps to S4; When the confidence level of the water meter image recognition result is less than a preset value, the process jumps to S1. During the process of re-fixing the water meter and collecting and recognizing the image, if the water meter fixing, collecting and recognizing operations are repeated for the same water meter for more than M times, the fixing, collecting and recognizing operations of the corresponding water meter dial are stopped, and the recognition operation process of the corresponding water meter dial is terminated; M is a preset constant. In the process of waiting for the corresponding administrator to perform manual error correction in S3, if the water meter is fixed and image captured and recognized again, and the confidence levels corresponding to the recognition results obtained are greater than or equal to the preset values, the corresponding water meter image recognition results and the corresponding water meter image capture results sent to the manual error correction processing sequence will be deleted.

6. The AI ​​vision-based multimodal water meter dial recognition method according to claim 1, characterized in that: In the process of managing the image recognition results of the corresponding water meter in combination with the duplicate number detection result, S4 determines that the water meter number in the called water meter image recognition result is duplicated, and stops the recognition operation process of the corresponding water meter dial; Otherwise, the dial value identified by the corresponding water meter dial is bound to the corresponding water meter dial and saved in the database.

7. A water meter dial multimodal recognition system based on AI vision, applying the water meter dial multimodal recognition method based on AI vision according to any one of claims 1 to 6, characterized in that: The system comprises: The water meter fixing and image acquisition module is used to place the produced water meter on the water meter fixing device and collect the water meter image on the water meter fixing device through the camera; The AI ​​visual analysis and recognition module automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, it obtains the corresponding water meter number, digital flow area, pointer flow area, and the confidence level corresponding to each recognition result in the collected water meter image, and sends the obtained image recognition results to the production test end; The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. If the confidence level corresponding to the water meter image recognition result is less than a preset value, the module jumps to the water meter fixation and image acquisition module to fix the water meter again, collect images, and identify the water meter. Before jumping to the water meter fixation and image acquisition module, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction queue for manual error correction by the corresponding administrator. The duplicate number detection management module is used to call the water meter number in the water meter image recognition result for duplicate number detection when the corresponding confidence level of the water meter image recognition result is greater than or equal to the preset value, and manage the image recognition results of the corresponding water meter in combination with the duplicate number detection result.

8. The AI ​​vision-based water meter dial multimodal recognition system according to claim 7 is characterized by: The Ai visual analysis and recognition module includes a regional image positioning unit, a regional image parameter recognition unit and an image recognition result transmission unit. The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed Ai visual analysis model; The regional image parameter recognition unit obtains the corresponding water meter number, digital flow, pointer flow, and confidence level of each recognition result in the collected water meter image based on image recognition technology; The image recognition result transmission unit sends the obtained image recognition result to the production test end.

Images