Method of industrial instrument identification system based on machine vision

Abstract

The invention provides an industrial instrument identification system based on machine vision. According to the design scheme, the high-definition visual sensor, the OCR algorithm, containerization deployment and multi-protocol support are combined, and efficient and automatic identification of the industrial instrument is achieved; an identification error caused by position change of the camera is solved through a reference image matching algorithm, and the accuracy of an identification area is ensured; meanwhile, an encryption protocol is adopted to guarantee data transmission security, and robustness and accuracy of text recognition are improved through linkage of real-time illumination, background monitoring and multiple cameras; by means of containerization and multi-protocol support, the system has high flexibility, expandability and efficient operation and maintenance capacity and can be seamlessly integrated with existing industrial equipment and platforms, and the efficiency and reliability of industrial intelligent management are improved.

Description

Technical Field

[0001] The invention relates to the field of industrial monitoring, and in particular to a method of an industrial instrument recognition system based on machine vision. Background Art

[0002] In industrial automation and monitoring systems, the installation locations of various instruments (such as flow meters, pressure gauges, steam meters, etc.) are often scattered and the working conditions are complex and harsh, such as high temperature, high humidity, strong electromagnetic interference and other environments, which limits the efficiency and accuracy of manual meter reading. The traditional manual meter reading method not only faces the problems of low data quality and unreliability, but also often leads to frequent data disputes due to traceability difficulties. In addition, manual meter reading is inefficient and has poor data synchronization, which makes it difficult to meet the high requirements of modern industry for real-time and accuracy.

[0003] Many modern instruments use LED carousel display technology to display real-time data by quickly switching between different values. Although this display method is energy-saving, it poses a great challenge to OCR recognition. The rapid switching of values ​​may cause visual persistence, making it difficult for the OCR system to accurately capture and recognize each value. At the same time, some LED indicators have a carousel display phenomenon, and the effective information will not be displayed all the time, but will switch the display according to the carousel cycle, further increasing the difficulty of recognition.

[0004] In industrial automation environments, the installation location of the camera may be difficult to fix due to restrictions on site conditions, and there is a possibility of position movement due to vibration, thermal expansion and contraction, temporary removal for manual inspection, etc. This requires the OCR system to have a high fault tolerance rate and be able to accurately recognize the data on the instrument when the camera position changes slightly. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a method for an industrial instrument recognition system based on machine vision.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method of industrial instrument recognition system based on machine vision, the method is as follows: S1: Unified data reporting and system building: S11: HD Vision Sensor Installation: Install high-definition micro-vision sensors in front of various instruments; S12: Data transmission: Transmit the collected image data to the enterprise intranet via wired or wireless network; S13: protocol upload; Use FTP protocol to upload image data to the company's private server; S14: Storage management; Combined with the Linux system timer, the images that exceed the preset storage period are automatically deleted regularly; S15: Real-time monitoring: Monitor the image folder on the server in real time. Once a new image file is uploaded, it will trigger the linkage processing with the OCR algorithm immediately. S2: Design an efficient OCR algorithm architecture; S21: Client deployment: The client monitors the image folder in step S1 in real time through watchdog. Once a new image file is detected, the client immediately captures the image and converts the captured image into Base64 encoding. The encoded image data is sent to the server through an HTTP POST request. S22: Server deployment: After receiving the image data encoded in step S21, the server first performs image preprocessing, including denoising and contrast enhancement operations; then uses an optimized text detection model to detect the text area in the image, and determines the correct direction of the text through a direction classifier; then uses an efficient text recognition model to recognize the text content and output a high-precision recognition result; Finally, the recognition result is returned to the client through HTTP response; S3: Construction of private deployment and operation and maintenance strategy; S31: Docker container deployment: Use Docker container technology to build a development and deployment environment to ensure that system components can run in isolated containers; package the applications and dependencies related to the visual data acquisition gateway into a Docker image by writing Dockerfile; S32: Container Orchestration and Management: Use Docker Compose or Kubernetes container orchestration tools to manage the deployment, expansion, and update of multiple containers; deploy multiple containers including data acquisition, image processing, and OCR recognition in the visual data acquisition gateway part of location B; S33: Process management and automatic recovery: Use the supervisor process management system to convert the OCR algorithm system into a daemon process to ensure that the service can be automatically restored in the event of a system restart or abnormal situation; configure the supervisor to monitor the status of each key process and immediately restart the corresponding process once an abnormal process termination is detected; S34: Logging and Monitoring Deploy the ELK log and monitoring system to collect and analyze the system operation status in real time, and promptly discover and handle potential problems; in the visual data acquisition gateway part of location B, focus on monitoring key links such as image acquisition, data upload and OCR recognition to ensure the stable operation of the system.

[0007] Preferably, the OCR data should be compatible with the original energy management system of the enterprise. In addition to the HTTP protocol, it also includes the development of two mainstream application layer protocols, Modbus TCP and MQTT. The specific steps are as follows: A1: Multi-protocol support: In addition to the HTTP protocol, the reporting part of the visual data acquisition gateway supports two mainstream application layer protocols, Modbus TCP and MQTT, to meet the data interaction requirements of different application layers; the Modbus TCP protocol is used for data communication with traditional industrial equipment to ensure that the system is compatible with existing industrial automation equipment; the MQTT protocol is used for data communication with the Internet of Things platform and cloud services, supporting lightweight and efficient data transmission; A2: Interface expansion design: In the system architecture design, the interface expansion capability is reserved to support the seamless access of new protocols in the future. Through modular design, new protocols only need to develop corresponding interface modules without large-scale changes to the existing system. A3: Data format conversion and configuration: In the visual data acquisition gateway, a data format conversion module is designed to convert the OCR recognition results into a data format that complies with the ModbusTCP and MQTT protocols, ensuring that the data can be correctly parsed and used by different systems.

[0008] Preferably, in step S1, the position change of the visual sensor may cause the pre-selected text recognition area to exceed the range, resulting in recognition failure. The method for automatically detecting and correcting the text detection frame offset problem caused by the change of the camera position by using a matching algorithm and real-time monitoring of the camera image changes is as follows: C1: Baseline image initialization: In the system initialization stage, a reference image is captured, and a text recognition area is manually or automatically selected on the image; ORB feature point detection and descriptor calculation are performed on the reference image to generate a feature point set and a descriptor set, which are stored as a reference for subsequent comparisons; C2: Real-time image capture: During system operation, the current camera screen is captured in real time periodically or under specific event trigger conditions to generate real-time images; C3: ORB feature point detection and description: Perform ORB feature point detection and descriptor calculation on the real-time image to generate a feature point set and a descriptor set of the real-time image; C4: Feature point matching: Use the LSH approximate nearest neighbor search algorithm to match the feature point descriptors of the reference image and the real-time image; C5: Transformation matrix calculation: The homography transformation matrix from the reference image to the real-time image is calculated from the matching pair through the RANSAC algorithm; C6: ROI correction: Using the calculated transformation matrix, the pre-selected text recognition area in the reference image is mapped to the real-time image to generate a corrected ROI area; C7: OCR recognition: In the corrected ROI area, the OCR algorithm is applied to perform text recognition to obtain the text content; C8: Abnormal detection and alarm: If the transformation matrix calculation fails or the corrected ROI area exceeds the boundary of the real-time image, it is determined that the camera position has changed significantly, which may cause recognition failure; at this time, the system triggers an alarm mechanism to notify the user to adjust the camera position or reinitialize the reference image.

[0009] Preferably, in step S1, all image data are transmitted using an encryption protocol during the upload process.

[0010] Preferably, step C2 also includes using multiple cameras for stereo monitoring, detecting multiple instruments through multiple cameras, changing the shooting angle to simulate the change of camera position, collecting image data at different angles and inputting them into the algorithm so that the algorithm can recognize images at different angles.

[0011] Preferably, the method further includes the step of monitoring the ambient lighting, background changes or moving objects in real time, and the algorithm adaptively adjusts parameters according to the environmental changes, thereby improving the accuracy of text recognition.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: the design scheme of the present invention combines high-definition visual sensors, OCR algorithms, containerized deployment and multi-protocol support to achieve efficient and automatic recognition of industrial instruments; the recognition error caused by camera position changes is solved by the reference image matching algorithm to ensure the accuracy of the recognition area; at the same time, an encryption protocol is used to ensure data transmission security, and real-time lighting, background monitoring and multiple camera linkage are used to improve the robustness and accuracy of text recognition; containerization and multi-protocol support enable the system to have strong flexibility, scalability and efficient operation and maintenance capabilities, and can be seamlessly integrated with existing industrial equipment and platforms, thereby improving the efficiency and reliability of industrial intelligent management. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is the overall schematic diagram of the industrial instrument recognition system based on machine vision; Figure 2 Provide a service deployment diagram for the OCR algorithm; Figure 3 This is a schematic diagram of connecting OCR data online with the enterprise energy management system; Figure 4 Initialize the matching example graph for the reference image. DETAILED DESCRIPTION

[0014] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.

[0015] Please refer to Figures 1 to 4 The present invention provides a method for an industrial instrument recognition system based on machine vision, and the method is specifically as follows: S1: Unified data reporting and system building: S11: HD Vision Sensor Installation: Install high-definition micro vision sensors in front of various instruments; select high-definition micro vision sensors according to application scenarios and measurement accuracy requirements; the resolution, frame rate, and low-light performance of the sensor are parameters that require special attention.

[0016] Install sensors in front of various instruments to ensure that they can clearly capture the target image and avoid interference factors such as light and reflection; consider using a fixed bracket or an automatically adjustable camera support to ensure the accuracy of the camera angle.

[0017] Calibrate the camera to ensure that the captured image is not distorted and that the image quality meets the requirements of subsequent OCR recognition.

[0018] S12: Data transmission: Transmit the collected image data to the enterprise intranet via wired or wireless network; Choose an appropriate wired (such as Ethernet) or wireless (such as Wi-Fi or 4G / 5G) network transmission method; the stability, bandwidth and signal coverage of wireless transmission need to be carefully evaluated.

[0019] Configure a fixed IP address, network port, and related protocols in the device to ensure stable transmission of image data.

[0020] Depending on the sensitivity of the data, encryption protocols (such as TLS) can be used to securely transmit image data to prevent data leakage.

[0021] S13: protocol upload; Use FTP protocol to upload image data to the company's private server; Use the FTP protocol to upload the collected image data to the company's private server. The FTP protocol is usually used for large-volume data transmission, but you need to ensure that you use a secure version (such as FTPS or SFTP) to prevent the data from being intercepted midway.

[0022] Set up an upload script or service to upload image data to the server in batches periodically or in a triggered manner to ensure no data is lost.

[0023] S14: Storage management; Combined with the Linux system timer, the images that exceed the preset storage period are automatically deleted regularly; Use a private server on the corporate intranet for data storage. Evaluate storage requirements based on the size of image data and upload frequency, and configure sufficient disk space and backup mechanisms.

[0024] Combine with Linux timers (such as cron) to automatically delete images that exceed the preset storage period regularly; make sure the images have been processed or backed up before deleting.

[0025] The maximum storage capacity should be considered during design. When the space is close to full, the storage cleanup mechanism is triggered to avoid storage overflow.

[0026] S15: Real-time monitoring: Monitor the image folder on the server in real time. Once a new image file is uploaded, it will trigger the linkage processing with the OCR algorithm immediately. Use a file system monitoring tool such as inotify to monitor the image folder on the server and detect the upload of new image files in real time.

[0027] Once a new image file is detected, the system will automatically trigger the OCR process to ensure that the image can be processed and recognized quickly.

[0028] S2: Design an efficient OCR algorithm architecture; S21: Client deployment: The client monitors the image folder in step S1 in real time through watchdog. Once the creation of a new image file is detected, the image is captured immediately, and the captured image is converted into Base64 encoding. The encoded image data is sent to the server through an HTTP POST request.

[0029] The client application monitors the image upload folder, and upon detecting the creation of a new file, it captures the image and processes it immediately.

[0030] Use Base64 encoding to convert image data into text form to facilitate transmission through the HTTP protocol; Base64 encoding has good compatibility and is suitable for transmitting binary data in HTTP requests.

[0031] The client sends the encoded image data to the server through an HTTP POST request; authentication information or encryption information is included in the request header to ensure secure data transmission.

[0032] S22: Server deployment: After receiving the image data encoded in step S21, the server first performs image preprocessing, including denoising and contrast enhancement operations; then uses an optimized text detection model to detect the text area in the image, and determines the correct direction of the text through a direction classifier; then uses an efficient text recognition model to recognize the text content and output a high-precision recognition result; Use filtering algorithms (such as median filtering and Gaussian filtering) to reduce noise in the image; use contrast enhancement algorithms (such as histogram equalization) to improve image quality and enhance OCR recognition accuracy.

[0033] For text area detection, use the text detection model of the OCR front end (such as EAST or CRAFT algorithm) to locate the text area from the image.

[0034] For direction classification, the text in the image is classified in terms of direction to ensure the correct recognition direction (such as using a direction classifier or a model based on deep learning).

[0035] For text recognition, use an efficient OCR recognition model (such as Tesseract, CRNN, or a custom deep learning model) to perform character recognition on the detected text area and output high-precision text results.

[0036] Finally, the recognition result is returned to the client through HTTP response; S3: Construction of private deployment and operation and maintenance strategy; S31: Docker container deployment: Use Docker container technology to build a development and deployment environment to ensure that system components can run in isolated containers; package the applications and dependencies related to the visual data acquisition gateway into a Docker image by writing Dockerfile; According to system requirements, write Dockerfile to automatically build container images and configure necessary environment variables, dependencies, and startup scripts; through containerized deployment, ensure the consistency of development, testing, and production environments to avoid problems with applications not running properly due to differences in different machines or operating systems.

[0037] S32: Container Orchestration and Management: Use Docker Compose or Kubernetes container orchestration tools to manage the deployment, expansion, and update of multiple containers; deploy multiple containers including data acquisition, image processing, and OCR recognition in the visual data acquisition gateway part of location B; When the system load is high, use container orchestration tools such as Kubernetes to scale horizontally and add more container instances to share the pressure.

[0038] Configure the CI / CD pipeline to achieve continuous integration and continuous deployment of containerized applications; each update is deployed through an automated process to reduce manual intervention.

[0039] S33: Process management and automatic recovery: Use the supervisor process management system to convert the OCR algorithm system into a daemon process to ensure that the service can be automatically restored when the system is restarted or in an abnormal situation; configure the supervisor to monitor the status of each key process and restart the corresponding process immediately once an abnormal process termination is detected.

[0040] S34: Logging and Monitoring: Deploy the ELK log and monitoring system to collect and analyze the system operation status in real time, and promptly discover and handle potential problems; in the visual data acquisition gateway part of location B, focus on monitoring key links such as image acquisition, data upload and OCR recognition to ensure the stable operation of the system.

[0041] Configure monitoring alarm rules to promptly notify operation and maintenance personnel via email, SMS, etc. when the system has abnormalities (such as high OCR recognition failure rate, container resource overload, etc.) Preferably, the OCR data should be compatible with the original energy management system of the enterprise. In addition to the HTTP protocol, it also includes the development of two mainstream application layer protocols, Modbus TCP and MQTT. The specific steps are as follows: A1: Multi-protocol support: In addition to the HTTP protocol, the reporting part of the visual data acquisition gateway supports two mainstream application layer protocols, Modbus TCP and MQTT, to meet the data interaction requirements of different application layers; the Modbus TCP protocol is used for data communication with traditional industrial equipment to ensure that the system is compatible with existing industrial automation equipment; the MQTT protocol is used for data communication with the Internet of Things platform and cloud services, supporting lightweight and efficient data transmission.

[0042] A11: Requirements Analysis and Protocol Selection During the design phase, you first need to analyze the application scenarios and devices to determine the most commonly used and most appropriate protocols; common industrial automation equipment generally supports Modbus TCP, while IoT platforms and cloud services generally prefer the MQTT protocol, so both protocols need to be supported at the same time.

[0043] As a universal communication protocol, HTTP is suitable for Web-based systems and is therefore also retained in the visual data acquisition gateway.

[0044] A12: Protocol stack development and integration For Modbus TCP protocol: Develop Modbus TCP protocol stack to ensure that data can be transmitted in accordance with Modbus standard format; Modbus protocol requires data to be transmitted via TCP / IP, based on client-server mode.

[0045] Design Modbus data register mapping table and develop communication interface with industrial equipment to ensure data reading and writing between devices can be carried out smoothly.

[0046] For the MQTT protocol: Use the existing MQTT protocol stack (such as Eclipse Paho) ​​to implement the message publishing and subscription mechanism; the MQTT protocol is based on the publish-subscribe model, which is suitable for IoT applications and can achieve low-latency and efficient data transmission.

[0047] Set the connection parameters between the device and the IoT platform, including the MQTT server address, port, subscribed topic, message format, etc.

[0048] For HTTP protocol: Develop HTTP client and server functions to ensure that data can be transmitted through standard RESTful interfaces; data is sent to the cloud or received from the cloud through HTTP requests (such as GET and POST).

[0049] Design unified interface specifications to ensure consistency in data formats for interaction with other systems.

[0050] A13: Multi-protocol parallel processing: Design a multi-protocol parallel processing framework in the gateway system; by designing a task scheduler, ensure that messages of different protocols can be processed in parallel without interfering with each other.

[0051] Decoupling between protocols is achieved so that each protocol runs independently without affecting each other, and certain protocols can be flexibly enabled or disabled as needed.

[0052] A14: Data Verification and Debugging In actual operation, debugging is performed by simulating the communication environment of different protocols to verify whether the data interaction of each protocol is stable and reliable.

[0053] For the Modbus TCP protocol, the accuracy of data transmission can be tested by connecting to industrial equipment. For the MQTT protocol, the reception and sending of data can be tested by connecting to the IoT platform. For the HTTP protocol, the correctness and stability of data can be verified through interface testing.

[0054] A2: Interface expansion design: In the system architecture design, interface expansion capabilities are reserved to support seamless access to new protocols in the future; through modular design, new protocols only require the development of corresponding interface modules without the need for large-scale changes to the existing system.

[0055] A21: System architecture design and interface planning: During the system design phase, plan the interface extension points in advance and ensure that the communication between modules is loosely coupled; through modular design, ensure that the access of subsequent protocols can be achieved by adding new modules rather than modifying existing modules.

[0056] Design unified standard interface specifications (such as interface functions, data formats, error handling mechanisms, etc.) for the interfaces of each protocol to ensure that the stability of the existing system will not be destroyed when new protocols are added later.

[0057] A22: Modular design and development: Each protocol is implemented as an independent module; for example, Modbus, MQTT, HTTP, etc. are developed as independent modules. Each module is responsible for protocol parsing and data interaction logic.

[0058] Each protocol module provides a standardized interface through which other modules can interact with the protocol module.

[0059] A unified interface management center is provided in the system, responsible for loading, unloading and switching of different protocol modules.

[0060] A23: Scalability considerations: When designing interfaces, keep them flexible and extensible. For example, support dynamic loading of protocol modules (such as through plug-ins or dynamic link libraries) without restarting the system or modifying existing code on a large scale.

[0061] The protocol modules are selected and configured through the configuration files or interfaces to achieve flexible switching and scheduling of protocols.

[0062] Provide protocol extension documentation and APIs to help future developers more easily access new protocols.

[0063] A24: Compatibility and Backtesting: After adding a new protocol, a comprehensive backtest is conducted to ensure that the existing functions of the system are not affected.

[0064] The system is stress-tested through automated testing tools to ensure that the system can run stably when multiple protocols are running in parallel.

[0065] A3: Data format conversion and configuration: In the visual data acquisition gateway, a data format conversion module is designed to convert the OCR recognition results into a data format that complies with the ModbusTCP and MQTT protocols, ensuring that the data can be correctly parsed and used by different systems.

[0066] A31: Obtaining OCR recognition results: The visual data acquisition gateway first obtains the data in the image through the OCR (optical character recognition) module. The OCR module is responsible for image preprocessing (such as denoising, binarization, etc.) and extracting text information.

[0067] Store the recognition results in a standard data format (such as JSON, XML, or CSV) for subsequent processing.

[0068] A32: Data format conversion: Design a data format conversion module to convert OCR results from a common format (such as JSON) to the format required by each protocol.

[0069] Modbus TCP data format conversion: The Modbus protocol uses a register mapping method, and the OCR text data needs to be converted into a format that Modbus can understand (such as storing data by register address, specifying function codes, etc.).

[0070] The conversion process needs to consider the conversion of different data types, such as the storage methods of numbers, strings, floating-point numbers, etc. in Modbus.

[0071] MQTT data format conversion: Encapsulate the OCR recognized text data into a message body in JSON format and send it to the specified Topic according to the requirements of the MQTT protocol.

[0072] Ensure that the message content meets the requirements of the IoT platform and can be correctly parsed after receiving the message.

[0073] HTTP data format conversion: Package the OCR results in JSON or XML format and send them to the cloud server via HTTP request. Design a RESTful interface to ensure the simplicity and efficiency of data transmission.

[0074] Configure the header and body of the HTTP request to ensure the correctness of the message.

[0075] A33: Data processing and configuration: Provides flexible data processing functions, such as data verification, data screening, data aggregation, etc., to ensure data quality during the conversion process.

[0076] Design a data configuration interface or configuration file to allow users to adjust the OCR result processing method and data format mapping according to actual application requirements.

[0077] Provides logging and debugging functions so that developers can track the entire process of data conversion and ensure that the data is accurate.

[0078] A34: Testing and Verification: Perform unit testing on the data conversion module to verify whether the data format meets the requirements of each protocol.

[0079] Perform integration tests on actual devices and platforms to ensure that OCR recognition results can be correctly parsed and used by Modbus, MQTT, and HTTP protocols.

[0080] Preferably, in step S1, the position change of the visual sensor may cause the pre-selected text recognition area to exceed the range, resulting in recognition failure. The method for automatically detecting and correcting the text detection frame offset problem caused by the change of the camera position by using a matching algorithm and real-time monitoring of the camera image changes is as follows: C1: Baseline image initialization: During the system initialization phase, a reference image is captured and a text recognition area is manually or automatically selected on the image; ORB feature point detection and descriptor calculation are performed on the reference image to generate a feature point set and a descriptor set, which are stored as a reference for subsequent comparisons.

[0081] Manual method: The user selects a rectangular area through the interface, which is the area that needs to be recognized by subsequent OCR. This area is usually a part of the image, such as a text box, logo or number.

[0082] Automatic method: Use a pre-trained text detection algorithm (such as EAST, CTPN, etc.) to automatically identify the text area in the image and mark the area in the reference image.

[0083] C2: Real-time image capture: During system operation, the current camera image is captured in real time periodically or under specific event triggering conditions to generate real-time images.

[0084] Timed capture: You can set a timer to capture images periodically (for example, once a second).

[0085] Event trigger: Capture images when the system detects certain events, such as motion detection or external sensor trigger.

[0086] C3: ORB feature point detection and description: Perform ORB feature point detection and descriptor calculation on the real-time image to generate a feature point set and a descriptor set of the real-time image.

[0087] C4: Feature point matching: The LSH approximate nearest neighbor search algorithm is used to match the feature point descriptors of the reference image and the real-time image.

[0088] In the reference image and the real-time image, the ORB descriptor is used for matching. The descriptors in the reference image and the real-time image are compared by the FLANN matcher to obtain a set of matching pairs; a ratio test (such as the SIFT ratio test) is used to screen out high-quality matching pairs and eliminate false matches.

[0089] C5: Transformation matrix calculation: The homography transformation matrix from the reference image to the real-time image is calculated from the C4 matching pairs using the RANSAC algorithm.

[0090] C6: ROI correction: The calculated transformation matrix is ​​used to map the pre-selected text recognition area in the reference image onto the real-time image to generate a corrected ROI area.

[0091] The pre-selected text recognition region (ROI) in the reference image is mapped to the live image using the calculated homography matrix.

[0092] This mapping process involves coordinate transformation, and by applying a homography matrix, the location of the rectangular area in the reference image can be determined in the live image.

[0093] Generate a new ROI area on the live image, which should be aligned with the ROI area in the reference image to ensure that the text recognition algorithm can accurately locate the target area.

[0094] C7: OCR recognition: In the corrected ROI area, the OCR algorithm is applied to perform text recognition to obtain the text content.

[0095] C8: Abnormal detection and alarm: If the transformation matrix calculation fails or the corrected ROI area exceeds the boundary of the real-time image, it is determined that the camera position has changed significantly, which may cause recognition failure; at this time, the system triggers an alarm mechanism to notify the user to adjust the camera position or reinitialize the reference image.

[0096] Preferably, in step S1, all image data are transmitted using an encryption protocol during the upload process.

[0097] Preferably, step C2 also includes using multiple cameras for stereo monitoring, detecting multiple instruments through multiple cameras, changing the shooting angle to simulate the change of camera position, collecting image data at different angles and inputting them into the algorithm so that the algorithm can recognize images at different angles.

[0098] Preferably, the method further includes the step of monitoring the ambient lighting, background changes or moving objects in real time, and the algorithm adaptively adjusts parameters according to the environmental changes, thereby improving the accuracy of text recognition.

[0099] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.

Claims

1. A method for an industrial instrument recognition system based on machine vision, characterized in that: The method is specifically as follows: S1: Unified data reporting and system building: S11: HD Vision Sensor Installation: Install high-definition micro-vision sensors in front of various instruments; S12: Data transmission: Transmit the collected image data to the enterprise intranet via wired or wireless network; S13: protocol upload; Use FTP protocol to upload image data to the company's private server; S14: Storage management; Combined with the Linux system timer, the images that exceed the preset storage period are automatically deleted regularly; S15: Real-time monitoring: Monitor the image folder on the server in real time. Once a new image file is uploaded, it will trigger the linkage processing with the OCR algorithm immediately. S2: Design an efficient OCR algorithm architecture; S21: Client deployment: The client monitors the image folder in step S1 in real time through watchdog. Once a new image file is detected, the client immediately captures the image and converts the captured image into Base64 encoding. The encoded image data is sent to the server through an HTTP POST request. S22: Server deployment: After receiving the image data encoded in step S21, the server first performs image preprocessing, including denoising and contrast enhancement operations; then uses an optimized text detection model to detect the text area in the image, and determines the correct direction of the text through a direction classifier; then uses an efficient text recognition model to recognize the text content and output a high-precision recognition result; Finally, the recognition result is returned to the client through HTTP response; S3: Construction of private deployment and operation and maintenance strategy; S31: Docker container deployment: Use Docker container technology to build a development and deployment environment to ensure that system components can run in isolated containers; package the applications and dependencies related to the visual data acquisition gateway into a Docker image by writing Dockerfile; S32: Container Orchestration and Management: Use Docker Compose or Kubernetes container orchestration tools to manage the deployment, expansion, and update of multiple containers; deploy multiple containers including data acquisition, image processing, and OCR recognition in the visual data acquisition gateway part of location B; S33: Process management and automatic recovery: Use the supervisor process management system to convert the OCR algorithm system into a daemon process to ensure that the service can be automatically restored in the event of a system restart or abnormal situation; configure the supervisor to monitor the status of each key process and immediately restart the corresponding process once an abnormal process termination is detected; S34: Logging and Monitoring Deploy the ELK log and monitoring system to collect and analyze the system operation status in real time, and promptly discover and handle potential problems; in the visual data acquisition gateway part of location B, focus on monitoring key links such as image acquisition, data upload and OCR recognition to ensure the stable operation of the system.

2. The method of an industrial instrument recognition system based on machine vision as claimed in claim 1, characterized in that: The OCR data must be compatible with the original energy management system of the enterprise. In addition to the HTTP protocol, it also includes the development of two mainstream application layer protocols, ModbusTCP and MQTT. The specific steps are as follows: A1: Multi-protocol support: In addition to the HTTP protocol, the reporting part of the visual data acquisition gateway supports two mainstream application layer protocols, Modbus TCP and MQTT, to meet the data interaction requirements of different application layers; the Modbus TCP protocol is used for data communication with traditional industrial equipment to ensure that the system is compatible with existing industrial automation equipment; the MQTT protocol is used for data communication with the Internet of Things platform and cloud services, supporting lightweight and efficient data transmission; A2: Interface expansion design: In the system architecture design, the interface expansion capability is reserved to support the seamless access of new protocols in the future. Through modular design, new protocols only need to develop corresponding interface modules without large-scale changes to the existing system. A3: Data format conversion and configuration: In the visual data acquisition gateway, a data format conversion module is designed to convert the OCR recognition results into a data format that complies with the Modbus TCP and MQTT protocols, ensuring that the data can be correctly parsed and used by different systems.

3. The method of an industrial instrument recognition system based on machine vision as claimed in claim 1, characterized in that: In step S1, the position change of the visual sensor may cause the pre-selected text recognition area to exceed the range, resulting in recognition failure. The method for automatically detecting and correcting the text detection frame offset problem caused by the change of the camera position by using a matching algorithm and real-time monitoring of the camera image changes is as follows: C1: Baseline image initialization: In the system initialization stage, a reference image is captured, and a text recognition area is manually or automatically selected on the image; ORB feature point detection and descriptor calculation are performed on the reference image to generate a feature point set and a descriptor set, which are stored as a reference for subsequent comparisons; C2: Real-time image capture During system operation, the current camera screen is captured in real time periodically or under specific event trigger conditions to generate real-time images; C3: ORB feature point detection and description Perform ORB feature point detection and descriptor calculation on the real-time image to generate a feature point set and a descriptor set of the real-time image; C4: Feature Point Matching Use the LSH approximate nearest neighbor search algorithm to match the feature point descriptors of the reference image and the real-time image; apply Lowe's ratio test to select more reliable matching pairs; C5: Transformation matrix calculation The homography transformation matrix from the reference image to the real-time image is calculated from the matching pair through the RANSAC algorithm; C6: ROI Correction Using the calculated transformation matrix, the pre-selected text recognition area in the reference image is mapped to the real-time image to generate a corrected ROI area; C7: OCR recognition In the corrected ROI area, the OCR algorithm is applied to perform text recognition to obtain the text content; C8: Anomaly detection and alarm If the transformation matrix calculation fails or the corrected ROI area exceeds the boundary of the real-time image, it is determined that the camera position has changed significantly, which may cause recognition failure; at this time, the system triggers an alarm mechanism to notify the user to adjust the camera position or reinitialize the reference image.

4. The method of an industrial instrument recognition system based on machine vision as claimed in claim 1, characterized in that: In the step S1, all image data are transmitted using an encryption protocol during the upload process.

5. The method of an industrial instrument recognition system based on machine vision as claimed in claim 3, characterized in that: The step C2 also includes using multiple cameras for stereo monitoring, detecting multiple instruments through multiple cameras, changing the shooting angle to simulate the change of camera position, collecting image data at different angles and inputting them into the algorithm so that the algorithm can recognize images at different angles.

6. The method of an industrial instrument recognition system based on machine vision as claimed in claim 3, characterized in that: The method further includes the step of monitoring the ambient lighting, background changes or moving objects in real time, and the algorithm adaptively adjusts parameters according to the environmental changes, thereby improving the accuracy of text recognition.

Images