Industrial pointer type instrument reading method based on deep learning and application

Abstract

The invention provides an industrial pointer type instrument reading method based on deep learning and application thereof, and relates to the field of computer vision and target detection, and the method comprises the steps: firstly constructing a customized image data set containing various types of industrial pointer type instruments; and then designing and realizing an automatic reading algorithm, and finally developing an industrial instrument reading system. According to the implementation method of the system, the independence and the maintainability of each functional module are ensured through modular design, meanwhile, a visual and easy-to-use user interface is provided, and the automatic reading function of the industrial pointer type instrument is achieved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and target detection, and in particular to a method for reading an industrial pointer instrument based on deep learning and an application thereof. Background Art

[0002] Intelligent manufacturing and automated production have become the mainstream trend of industrial development. As a key component of industrial automation systems, industrial instruments undertake the important tasks of measuring, monitoring and controlling various production parameters. Among them, industrial pointer instruments have been widely used in many industries such as electricity, chemical industry, metallurgy, machinery, etc. due to their intuitive and easy-to-read characteristics. However, the traditional manual reading method has many disadvantages, such as low efficiency, susceptibility to human errors, and difficulty in adapting to complex or harsh industrial environments. These problems not only increase production costs, but may also have an adverse impact on production safety and quality control.

[0003] In recent years, with the rapid development of computer vision and deep learning technologies, target detection technology has made significant breakthroughs. In particular, deep learning algorithms have shown excellent performance in image recognition and object detection. These technologies provide new solutions for the automatic reading of industrial pointer instruments. By introducing computer vision and deep learning technologies, automatic detection and reading of industrial pointer instruments can be achieved, thereby overcoming the drawbacks of traditional manual reading methods, improving the accuracy and efficiency of readings, reducing labor costs, and promoting the process of industrial automation and intelligence.

[0004] Although some studies have attempted to apply deep learning to the reading of industrial pointer instruments, these methods often have some problems. For example, some methods rely on complex semantic segmentation or OCR recognition technology, resulting in large computational complexity and slow speed; some methods do not work well when processing tilted or deformed instrument images and have poor robustness; and some methods lack an intuitive user interface, making it inconvenient for users to operate. Therefore, developing an efficient, accurate, robust and user-friendly method for reading industrial pointer instruments has important practical significance and application value. Summary of the invention

[0005] The technical solution adopted by the present invention is as follows: In response to the above technical problems, the present invention proposes a method for reading industrial pointer instruments based on deep learning. The method integrates advanced computer vision and deep learning technologies, and realizes automatic and accurate reading of industrial pointer instruments by optimizing the target detection model. The specific technical solution is as follows: The industrial pointer instrument reading method based on deep learning is characterized by comprising the following steps: Step 1: First, build a custom image dataset containing various types of industrial pointer instruments; Step 2: Design and implementation of automatic reading algorithm; Step 3: Development of industrial instrument reading system.

[0006] Furthermore, the step 1 includes the following steps: Step 101: Collect images of various industrial pointer instruments; Step 102: Use an image annotation tool to annotate key information such as pointers, scales, and numbers in the image.

[0007] Furthermore, the step 2 comprises the following steps: Step 2.1: Model inference and data preprocessing; Step 2.2: Determine the coordinates of the pointer tip; Step 2.3: Gradual positioning of scale areas; Step 2.4: Angle ratio calculation and reading determination.

[0008] Furthermore, in step 2.1, the image is input into a well-trained target detection model for prediction, and the prediction result is processed by a confidence threshold screening mechanism, and the prediction result is screened, data structured, and spatial relationship parsed. The specific process is as follows: Filter Results Setting the confidence threshold , filter low confidence detection frames, the prerequisite for the detection frames to be retained is:

[0009] Data structuring Convert valid detection results into XML files in PASCAL VOC format, ensuring that the output data structure is completely consistent with the training annotations; Spatial relationship analysis For each target involved in the calculation of this method, the geometric center coordinates of its detection box are used to represent its position; assuming that the four coordinates of a target's detection box are , then its geometric center coordinates are: , , .

[0010] Furthermore, in step 2.2, a geometric reasoning model is established, namely, rectangle ABCD; In rectangle ABCD, arrow CA represents the instrument pointer, where A is the pointer head and circle O is the rotation axis of the pointer, which is always located on AC and , OC represents the tail of the pointer, point E is the midpoint of OC, and F is the intersection of diagonals AC and BD. First, determine which vertex on the detection box the tip of the pointer is located at. For distance calculation, the Euclidean distance is used: (2.1) The vertex set of the rectangular detection box V={A,B,C,D}, we analyze the position of point O: When O is located at the intersection F of the diagonals AC and BD, the distances from O to the four vertices are equal, that is, ; For the general case, When the distance function is established , through differential analysis we can get: ,

[0011] This derives the pointer tip positioning criterion: (2.2) If O moves toward C, then the lengths of OA and OD will gradually become longer, and OB and OC will gradually become shorter, with OA being the longest and OC being the shortest. ; If point O coincides with point F, the lever balance principle is introduced for correction: according to the moment balance condition Due to the mass distribution characteristics of the pointer tail, the midpoint E of OC must be located between CF. Therefore, we select the rotation center and the pointer part after it as a whole for labeling, that is, the label is "ass". In this way, the point we choose is the midpoint of the "ass" label detection box. This point must be located in the second half of the pointer and will not overlap with F.

[0012] Further, the steps of step 2.3 are as follows: Digital region proximity matching First, we need to build a set of digital detection boxes , where each element contains the corresponding center coordinates And its own value , then, calculate the pointer tip With all The Euclidean distance of: Finally, filter out , The corresponding values ​​of the two digital type detection boxes , , it can be determined that the pointer tip is between 0.4 and 0.6; Tick ​​mark proximity matching First, we need to build a set of detection boxes for the "scale" category , records the center coordinates of each element and associated numeric values , in order to find the above , The corresponding nearest tick mark is calculated as follows: (2.3) (2.4) Through the above calculation, we can get and The coordinates of the geometric center of the two nearest tick marks.

[0013] Furthermore, the step 2.4 is divided into three steps: reference angle construction, pointer angle calculation and angle correction mechanism; Reference Angle Construction First, construct a rectangle ABCD to represent the detection frame of the instrument panel. Arrows ae represent the pointer. Point a is the center of the dial, and e is the tip of the pointer. The coordinates of point a are , the coordinates of e are , the other two points are the geometric centers of the two thick scale lines obtained above. We record these two points as b and c, and the value corresponding to b is smaller than the value corresponding to c. Assume that when b and c are both on the left side of the central axis of the dial, b is below c; when they are both on the right side, b is below c; when they are on both sides of the central axis, b is on the left side of c. The coordinates of b and c are respectively and , according to the above conditions, we can find that a is the vertex, ab and ac are the angle formed by the two sides. This angle is called the reference angle and is calculated as follows: (2.5) Pointer angle calculation For the second angle, replace point c in the first angle with point e, where the tip of the pointer is located. That is, the angle formed by point b and the position of the pointer head with the center point a as the vertex. This angle is called the pointer angle and is calculated as follows: (2.6) Angle correction mechanism When the calculated angle value When , the supplementary angle operation is performed: (2.7) The center of the dial can be regarded as the center of a circle, so points a, b, c and the pointer head can form a sector. According to the properties of the sector, the proportion of the angle in the sector is the proportion of the area in the sector. The calculation formula for the final reading is as follows: (2.8).

[0014] Furthermore, the step 3 includes a model management function, a user information management function, an image selection function, an image reading function, an abnormal image management function and a phenotype management function.

[0015] Application of industrial pointer instrument reading method based on deep learning in reading industrial pointer instruments 3. Beneficial effects: The technical effects achieved by the present invention are: (1) Improve reading accuracy and efficiency: The deep learning model automatically detects the key information of the pointer instrument and calculates the reading, avoiding human errors and inefficiency. (2) Reduce labor costs: The automated reading system can replace some manual operations, reduce human resource investment, and reduce enterprise operating costs; (3) Enhanced robustness and generalization capabilities: By optimizing the model structure and training strategy, the model's adaptability to complex environments and diverse industrial pointer instruments is improved; (4) Provide an intuitive human-computer interaction interface: Users can conveniently perform operations such as image selection, model loading, and reading prediction through the interface, which improves the convenience and flexibility of use; (5) Support real-time detection and correction: The system can detect and correct the deflection of the input image in real time to ensure the accuracy of the reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of determining the coordinates of the pointer of the present invention; Figure 2 This is a schematic diagram of the pointer position interval determination and angle determination of the present invention. Figure 3 It is the architecture diagram of the industrial pointer instrument reading system of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0018] Dataset Construction First, we build an image dataset containing various types of industrial pointer instruments. The dataset should include detailed annotations of key information such as pointers, scales, and numbers, and the annotation information should be saved in multiple formats such as json, xml, and txt. The specific steps are as follows: Step 101: Collect images of various industrial pointer instruments.

[0019] Step 102: Use image annotation tools to annotate key information such as pointers, scales, and numbers in the image. The specific annotation method is as follows: Because our entire reading method does not involve semantic segmentation tasks, we have innovatively annotated this part of the data set. We mark the dial of the clock as a type of "clock", the pointer as "point", the pointer has a protruding rotation axis, the rotation axis and the pointer part behind it are marked as "ass", the thick scale lines next to the numbers written on the dial (such as 0 to 12 on daily clocks) are marked as "scale", and each number is marked as its own value (for example, 1 is marked as 1, and 0 is marked as 0). The labeling file generates three formats: json, txt, and xml to meet different model training needs. On this basis, in order to improve the detection accuracy of the model for digital targets, we also added some ordinary simple numbers to the data set to improve the accuracy of the final trained model.

[0020] Design and implementation of automatic reading algorithm 2.1 Model Reasoning and Data Preprocessing In this study, we input images into a well-trained object detection model for prediction and use a confidence threshold screening mechanism to process the prediction results. In order to better implement the meter reading method proposed in this paper, it is necessary to screen the prediction results, structure the data, and analyze the spatial relationships. The specific process is as follows.

[0021] Filter Results Setting the confidence threshold , filter low confidence detection boxes. The prerequisite for the detection box to be retained is:

[0022] Data structuring Convert valid detection results into XML files in PASCAL VOC format to ensure that the output data structure is completely consistent with the training annotations.

[0023] Spatial relationship analysis For each target involved in the calculation of this method, the geometric center coordinates of its detection box are used to represent its position. Assume that the four coordinates of the detection box of a target are , then its geometric center coordinates are: , , .

[0024] 2.2 Determination of the coordinates of the pointer tip Through a large number of experimental observations (sample size ), and found that the pointer detection box output by the target detection model has stable geometric characteristics: the detection box is rectangular (the standard deviation of the aspect ratio ) The pointer tip is always at the vertex of the detection box (probability Based on this, we establish Figure 1 The geometric reasoning model shown is rectangle ABCD.

[0025] In rectangle ABCD, arrow CA represents the instrument pointer, where A is the pointer head. Circle O is the rotation axis of the pointer, which is always located on AC and OC represents the tail of the pointer. Point E is the midpoint of OC. F is the intersection of diagonals AC and BD. The first step is to determine which vertex on the detection box the pointer tip is located at. For distance calculation, this article uses Euclidean distance: (2.1) The vertex set of the rectangular detection box V={A,B,C,D}, we analyze the position of point O: When O is located at the intersection F of the diagonals AC and BD, the distances from O to the four vertices are equal, that is, .

[0026] For the general case, When the distance function is established , through differential analysis we can get: ,

[0027] This derives the pointer tip positioning criterion: (2.2) If O moves toward C, then the lengths of OA and OD will gradually increase, while OB and OC will gradually decrease. OA is the longest and OC is the shortest. Therefore, we can conclude In other words, as long as point O moves on the line segment CF that does not include F, the vertex of the detection box farthest from O is the location of the pointer tip. The universality of this criterion is verified by geometric construction and can be extended to the case where the pointer tip is located at the vertices B, C, and D.

[0028] However, a special case must be considered, that is, in the design of very few industrial pointer instruments, point O may coincide with point F. In order to solve this problem, this method introduces the lever balance principle for correction: according to the moment balance condition Due to the mass distribution characteristics of the pointer tail, the midpoint E of OC must be located between CF.

[0029] Therefore, we chose the rotation center and the pointer part after it as a whole for annotation (labeled as "ass"). In this way, the point we chose is the midpoint of the "ass" label detection box, which must be located in the second half of the pointer and will never coincide with F. Figure 1 A diagram showing the tail (ass) of a pointer is shown.

[0030] Figure 1 Schematic diagram of pointer tail (ass) 2.3 Graduation area classification positioning After determining the position coordinates of the pointer head, the goal of this stage is to establish the spatial mapping relationship between the pointer tip and the numbers and dial scales. The specific implementation steps are as follows: Digital region proximity matching In this step, we first need to build a set of digital detection boxes Each element contains the corresponding center coordinates (x i ,y i ) and its own value v i . Then, calculate the pointer tip v tip With all d i The Euclidean distance of: Finally, filter out δ min1 , δ min2 The corresponding values ​​v of the two digital type detection boxes min 、v max .like Figure 3 As shown, after this step, it can be determined that the pointer tip is between 0.4 and 0.6.

[0031] Figure 2 Diagram of dial Tick ​​mark proximity matching In this step, we first need to build a set of detection boxes for the "scale" category , records the center coordinates of each element and associated numeric values (Need to be obtained through nearest neighbor matching). In order to find the above , The corresponding nearest tick mark is calculated as follows: (2.3) (2.4) Through the above calculation, we can get and The coordinates of the geometric center of the two nearest tick marks.

[0032] 2.4 Angle ratio calculation and reading determination The establishment of angle ratio is the core of reading calculation. The work in this stage is mainly divided into three steps: reference angle construction, pointer angle calculation and angle correction mechanism.

[0033] Reference Angle Construction In the previous three stages, the geometric center coordinates of the dial and the thick scale lines have been obtained, so Figure 2 As shown in the figure, a rectangle ABCD is constructed to represent the detection frame of the instrument panel, and arrows ae represent the pointer. Point a is the center of the dial, e is the tip of the pointer, and the coordinates of point a are , the coordinates of e are . The remaining two points are the geometric centers of the two thick scale lines obtained previously. We record these two points as b and c, and the value corresponding to b is smaller than the value corresponding to c (assuming that when b and c are both on the left side of the dial center axis, b is below c; when they are both on the right side, b is below c; when they are on both sides of the center axis, b is on the left side of c.). The coordinates of b and c are and . Based on the above conditions, we can find that a is the vertex, ab and ac are the angle formed by the two sides. This angle is called the reference angle and is calculated as follows.

[0034] (2.5) Pointer angle calculation For the second angle, this method chooses to replace point c in the first angle with point e where the pointer tip is located, that is, the angle formed by point b and the location of the pointer head with the center of the circle (point a) as the vertex. This angle is called the pointer angle, and its calculation method is as follows.

[0035] (2.6) Angle correction mechanism Angles are divided into internal angles and external angles. In this method, only angles less than 180 degrees are used for calculation. When , the supplementary angle operation is performed: (2.7) The center of the dial can be regarded as the center of a circle, so points a, b, c and the pointer can form a sector. According to the properties of the sector, we can conclude that the proportion of the angle in the sector is the proportion of the area in the sector. The calculation formula for the final reading is as follows: (2.8).

[0036] Development of industrial instrument reading system 3.1 Design and implementation of model management functions The model management module is one of the core functional modules of the system. It is mainly used for administrators to upload deep learning model files and provide users with model selection functions. Administrators can upload trained model files to the system through this module. After selecting an image, users can select a suitable model from the uploaded model list for reading prediction. This module is designed to improve the flexibility and scalability of the system and support dynamic loading and use of multiple models.

[0037] In the back-end implementation, the model upload interface / upload_model is designed first. This interface receives the model file uploaded by the administrator through the front-end, stores the file in the uploads / models directory of the server, and returns the response information of successful upload. Secondly, the model list acquisition interface / get_models is designed. This interface reads the list of all uploaded model files from the server directory and returns it to the front-end in json format for users to choose. Finally, the model selection interface / select_model is designed. This interface receives the model name selected by the user, dynamically loads the corresponding model file, and calls the prediction logic to predict the reading of the image selected by the user, and finally returns the prediction result to the front-end.

[0038] In the front-end implementation, the administrator clicks the "Upload Model" button to call the file selection dialog box to select the local model file, and uploads the file to the server through the model upload interface. After the user selects the image, clicks the "Show Results" button, and the system obtains the model list from the server through the model list acquisition interface and displays it in the pop-up model selection interface. After the user selects a model from the list, the system calls the model selection interface to load the model and make a prediction, and finally displays the reading results in the interface.

[0039] 3.2 Design and implementation of user information management function The administrator can click the "Load Users" button on the system interface to trigger the Load_Users() function, which retrieves all user information from the / get_users endpoint of the Flask backend through an HTTP GET request and displays it in a list on the interface. The administrator can select the user to be deleted and click the "Delete User" button to trigger the Delete_User() function, which sends a DELETE request to the / users / <username>Endpoint, execute SQL DELETE statement to delete the specified user record. This function ensures that only administrator users can perform user management operations through the permission control mechanism, effectively improving the security and manageability of the system. At the same time, the front-end interface displays the user list through the Listbox control, allowing administrators to intuitively view and operate user information.

[0040] 3.3 Design and implementation of image selection function (1) Call the camera to take pictures and automatically select the image function The user triggers the "Camera Photo" function through the system interface. First, the system checks whether the user has set the camera save path. If not, the system prompts the user to specify the save location through the "Set Camera Save Path" function. If the save path is set, the system will pop up a camera selection list after the user clicks the "Camera Photo" button. The list lists the camera devices on the user's computer for the user to choose. Then, the win32gui.FindWindowEx() function is used to traverse the window handles on Windows to find the window representing the camera device. For each camera device found, its name is obtained through the win32gui.GetWindowText() function and added to the camera list. After the user selects a camera from the list, the system uses the cv2.VideoCapture class of OpenCV to open the camera device. A frame of image is read from the camera through the cap.read() method. If the read is successful, the system converts the image from BGR format to RGB format for subsequent saving. The converted image is saved to the local file system using the cv2.imwrite() function. And its address is assigned to a global variable, which can be read directly later.

[0041] (2) Implementation of local image selection function After receiving the image file path selected by the user, the system first checks the validity of the file path to ensure that the user has indeed selected a file. If the file path is valid, the system opens the image file and obtains its basic information. The system then feeds back the address of the selected image to the user through a message box. After the user selects an image, the system stores the image path in a global variable selected by the image reading function for subsequent processing.

[0042] 3.4 Design and implementation of image reading function This module is used to obtain the results of instrument readings and is a specific implementation of the reading algorithm proposed in Chapter 3 of this article.

[0043] In terms of the generation of the XML file, first, we initialize an XML root element named annotation. Then add the folder element representing the folder where the image is located to the root element, and set its text content to the specified folder name. Next, add the filename element, and set its text content to the base name of the image file. Subsequently, add a size element and its child elements width, height, and depth, which represent the width, height, and depth of the image, respectively. Then, we traverse the target detection result set result, and for each detected object with a confidence greater than 0.8, create an object element and fill in its name, whether it is truncated, whether it is occluded, and bounding box information, including the coordinates of the upper left and lower right corners of the bounding box. Finally, we define the output folder and file path. After ensuring that the output directory exists, write the constructed XML tree to the file in the specified path. The file name is the image file name plus the ".xml" suffix and is encoded in UTF-8.

[0044] In terms of determining the coordinates of the pointer head, we first parse the xml file of the specified path into detection frame data (stored in detections3) by calling the custom function Parse_xml_to_detections3(). Next, use list derivation to filter out all detection frames tagged with "ass" and "pointer" and store them in the ass_detections and pointer_detections lists respectively. Then, check whether at least one "ass" and one "pointer" detection frame are found at the same time. If the condition is met, it takes the midpoint of the first "ass" detection frame as the reference point, and calls the Find_Farthest_Point() function to find the farthest point from the reference point in the "pointer" detection frame. At this time, the coordinates of this point are the position coordinates of the pointer head. If not enough detection frame types are found, it prints "Not enough detection frame types are found, reading error".

[0045] In terms of judging the position interval of the pointer head reading, we first initialized an empty list called detections to store the object information detected from the XML document, including the center point coordinates and names of the objects. In order to accurately identify object names containing numbers, we defined a regular expression number_pattern for matching. Then, we traverse all object elements in the XML document and perform the following operations for each object: extract its name and compare it. If the name matches the number, further extract the coordinate information of the bounding box bndbox (including xmin, ymin, xmax, ymax), calculate the center point coordinates of the object based on this, and combine the center point coordinates with the name into a tuple and add it to the detections list.

[0046] Next, to store the object information sorted by distance, we initialized another empty list Sorted_Objects_Tuples. By traversing each object in the detections list, we calculated the Euclidean distance from the center point of each object to the given point. Subsequently, we sorted the objects in the Sorted_Objects_Tuples list using the sort method and a lambda function that calculates the distance as the sort key.

[0047] We select the first two objects in the sorted list and store their center point coordinates and names in a new tuple closest_two_objects. For subsequent processing, we convert the names of these two objects from string type to floating point type. The numerical range between the floating point labels of these two objects is the reading range of the pointer.

[0048] In terms of angle calculation and final reading, we define a function called Calculate_angle(), which is designed to calculate the angle between three points point1, point2 and the center of the dial. The function starts by expressing two vectors, one pointing from the center of the dial to point1 and the other pointing to point2. Then, using the dot product and length of the vectors, we can find the cosine of the angle between the two vectors, and then use the inverse cosine function to get the angle (in radians) and convert it to degrees. In order to accurately determine the direction of the angle (i.e., whether it is an acute angle or its supplementary angle), the function also calculates the cross product of the vectors, and decides whether to calculate the supplementary angle based on the sign of the cross product. Similarly, we also use this method to calculate the angle between the center of the dial and the center of the detection box corresponding to the smaller value of the two coarse scales.

[0049] Finally, we get the values ​​corresponding to the thick scale lines through the closest_two_objects array defined above, and define them as first_closest_namef and second_closest_namef respectively. Then, based on the relative size of these two values ​​and the formula proposed in Chapter 3, we calculate the final meter reading.

[0050] 3.5 Design and implementation of abnormal image management function In the backend, we use the SQLite database to store user information and abnormal image records, and design two tables. The users table is used to store basic user information, and the abnormal_images table is used to record abnormal images uploaded by each user and associate the user ID through a foreign key. In order to implement abnormal image management operations, the backend provides multiple API endpoints: / add_abnormal_image is used to receive abnormal images uploaded by users and save them to the server, while recording the user ID and image path in the database / delete_abnormal_image / <int:image_i-d> Used to receive deletion requests, delete the corresponding abnormal image records from the database, and remove the image files from the server; / get_abnormal_images is used to return all abnormal image records of the current user; / download_a-bnormal_image / <int:image_id> Used to provide download function for abnormal images.

[0051] For the front-end part, we designed an intuitive and easy-to-use abnormal image management interface, which is implemented through the Tkinter library. The interface contains an abnormal image list display area, allowing users to intuitively view the abnormal images they have recorded. At the same time, the interface also provides function buttons for adding abnormal images, deleting abnormal images, and downloading abnormal images. Users can trigger corresponding operations by clicking these buttons, and the front-end will call the API provided by the back-end to implement these functions. For example, when the user clicks the "Add Abnormal Image" button, the front-end will pop up a file selection dialog box, allowing the user to select an image file and upload it to the server. At the same time, it will call / add_abnormal_image to record the image information in the database and refresh the abnormal image list on the interface. In each operation, the front-end will check the response status code and error message returned by the back-end to ensure that users receive timely and accurate feedback. 3.6 Design and implementation of phenotypic management functions The phenotype management module is mainly to facilitate users to classify and manage phenotypes. The administrator clicks the "Add meter type" button through the system interface to trigger the Add_Meter_Type() function, which calls the / add_meter_type interface of the Flask backend and submits the meter type name in json format to the server through an HTTP POST request. After receiving the request, the backend executes the SQL INSERT statement, writes the new type name to the meter_types table, and returns the operation result. If the type name already exists, a uniqueness constraint error is returned to ensure data consistency. When deleting a meter type, the administrator selects the target type in the interface list and clicks the "Delete" button to trigger the Delete_Meter_Type(type_id) function, which calls / delete_meter_type / through an HTTP DELETE request.<int:type_id> Interface, the backend synchronously deletes the corresponding records in the meter_types table and the associated meter_images data to achieve cascade deletion.

[0052] For the image corresponding to each meter type, after the administrator selects the specified meter type, clicks the "Add Image" button to trigger the Add_Image_To_Type(type_id) function, and submits the image file and meter type ID to the / add_image_to_type interface through an HTTP POST request. The backend stores the image in the uploads / meter_types / {type_id} directory, inserts the associated record in the meter_images table, and returns the image storage path. To remove an image, the administrator selects the target file in the image list and clicks the "Remove" button to trigger the Remove_Image_From_ Type(image_id) function, called via HTTP DELETE request / remove_image_from_type / <int: image_id> interface, the backend deletes physical files and database records.

[0053] Ordinary users call the Get_Meter_Types() function through the interface, triggering an HTTP GET request to access the / get_meter_types interface. The backend jointly queries the meter_types and meter_images tables, and returns a nested JSON data structure containing the type name and image path. The front end uses the Treeview control to display the meter type and its image list in a tree hierarchy, supporting expansion / collapse operations. When the user clicks the "Download" button, the Download_Image(image_id) function is triggered, calling / download_image / <int:image_id> The interface obtains the image file stream, and uses the filedialog pop-up window to specify the local storage path to complete the download operation.

[0054] In summary, the system implementation method of the present invention ensures the independence and maintainability of each functional module through modular design, while providing an intuitive and easy-to-use user interface, realizing the reading function of industrial pointer instruments, and has wide application in reading industrial pointer instruments.

[0055] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.< / username>

Claims

1. A method for reading industrial pointer instruments based on deep learning, characterized in that: The following steps are involved: Step 1: First, build a custom image dataset containing various types of industrial pointer instruments; Step 2: Design and implementation of automatic reading algorithm; Step 3: Development of industrial instrument reading system.

2. The industrial pointer instrument reading method based on deep learning according to claim 1 is characterized in that: The step 1 includes the following steps: Step 101: Collect images of various industrial pointer instruments; Step 102: Use an image annotation tool to annotate key information such as pointers, scales, and numbers in the image.

3. The industrial pointer instrument reading method based on deep learning according to claim 1 is characterized in that: The step 2 comprises the following steps: Step 2.1: Model inference and data preprocessing; Step 2.2: Determine the coordinates of the pointer tip; Step 2.3: Gradual positioning of scale areas; Step 2.4: Angle ratio calculation and reading determination.

4. The industrial pointer instrument reading method based on deep learning according to claim 3 is characterized in that: In step 2.1, the image is input into a well-trained target detection model for prediction, and the prediction results are processed using a confidence threshold screening mechanism. The prediction results are screened, data structured, and spatial relationship parsed. The specific process is as follows: Filter Results Setting the confidence threshold , filter low confidence detection frames, the prerequisite for the detection frames to be retained is: Data structuring Convert valid detection results into XML files in PASCAL VOC format, ensuring that the output data structure is completely consistent with the training annotations; Spatial relationship analysis For each target involved in the operation of this method, the geometric center coordinates of its detection box are used to represent its position; Assume that the four coordinates of a target detection box are , then its geometric center coordinates are: , , .

5. The industrial pointer instrument reading method based on deep learning according to claim 3 is characterized in that: In the step 2.2, a geometric reasoning model is established, i.e., rectangle ABCD; In rectangle ABCD, arrow CA represents the instrument pointer, where A is the pointer head and circle O is the rotation axis of the pointer, which is always located on AC and , OC represents the tail of the pointer, point E is the midpoint of OC, and F is the intersection of diagonals AC and BD. First, determine which vertex on the detection box the tip of the pointer is located at. For distance calculation, the Euclidean distance is used: （2.1） The vertex set of the rectangular detection box V={A,B,C,D}, we analyze the position of point O: When O is located at the intersection F of the diagonals AC and BD, the distances from O to the four vertices are equal, that is, ; For the general case, When , the distance function is established , through differential analysis we can get: This derives the pointer tip positioning criterion: （2.2） If O moves toward C, then the lengths of OA and OD will gradually become longer, and OB and OC will gradually become shorter, with OA being the longest and OC being the shortest. ; If point O coincides with point F, the lever balance principle is introduced for correction: according to the moment balance condition Due to the mass distribution characteristics of the pointer tail, the midpoint E of OC must be located between CF. Therefore, we select the rotation center and the pointer part after it as a whole for labeling, that is, the label is "ass". In this way, the point we choose is the midpoint of the "ass" label detection box. This point must be located in the second half of the pointer and will not overlap with F.

6. The industrial pointer instrument reading method based on deep learning according to claim 3 is characterized in that: The steps of step 2.3 are as follows: Digital region proximity matching First, we need to build a set of digital detection boxes , where each element contains the corresponding center coordinates And its own value , then, calculate the pointer tip With all The Euclidean distance of: Finally, filter out , The corresponding values ​​of the two digital type detection boxes , , it can be determined that the pointer tip is between 0.4 and 0.6; Tick ​​mark proximity matching First, we need to build a set of detection boxes for the "scale" category , records the center coordinates of each element and associated numeric values , in order to find the above , The corresponding nearest tick mark is calculated as follows: （2.3） （2.4） Through the above calculation, we can get and The coordinates of the geometric center of the two nearest tick marks.

7. The industrial pointer instrument reading method based on deep learning according to claim 3 is characterized in that: The step 2.4 is divided into three steps: reference angle construction, pointer angle calculation and angle correction mechanism; Reference Angle Construction First, construct a rectangle ABCD to represent the detection frame of the instrument panel. Arrows ae represent the pointer. Point a is the center of the dial, and e is the tip of the pointer. The coordinates of point a are , the coordinates of e are , the other two points are the geometric centers of the two thick scale lines obtained above. We record these two points as b and c, and the value corresponding to b is smaller than the value corresponding to c. Assume that when b and c are both on the left side of the central axis of the dial, b is below c; when they are both on the right side, b is below c; when they are on both sides of the central axis, b is on the left side of c. The coordinates of b and c are respectively and , according to the above conditions, we can find that a is the vertex, ab and ac are the angle formed by the two sides. This angle is called the reference angle and is calculated as follows: （2.5） Pointer angle calculation For the second angle, replace point c in the first angle with point e, where the tip of the pointer is located. That is, the angle formed by point b and the position of the pointer head with the center point a as the vertex. This angle is called the pointer angle and is calculated as follows: （2.6） Angle correction mechanism When the calculated angle value When , the supplementary angle operation is performed: （2.7） The center of the dial can be regarded as the center of a circle, so points a, b, c and the pointer head can form a sector. According to the properties of the sector, the proportion of the angle in the sector is the proportion of the area in the sector. The calculation formula for the final reading is as follows: （2.8）。 8. The industrial pointer instrument reading method based on deep learning according to claim 1 is characterized in that: The step 3 includes a model management function, a user information management function, an image selection function, an image reading function, an abnormal image management function and a phenotype management function.

9. Application of the method according to claims 1 to 9 in reading industrial pointer instruments.

Images