Multi-camera intelligent monitoring and measurement control method and system

Abstract

This invention discloses a multi-camera intelligent monitoring and measurement control method and system, relating to the fields of industrial automation control, computer vision, and human-computer interaction. It aims to solve the problems of high processing latency, insufficient stability, and susceptibility to measurement errors in existing systems. The method includes multi-camera video acquisition and preprocessing, multi-threaded parallel target detection, target information extraction, offset calculation, closed-loop control execution, and measurement state locking and unlocking. It employs an optimized deep learning model to improve recognition accuracy and avoids resource contention through multi-threaded asynchronous decoupling and mutex locks. The system includes modules for video acquisition, target detection, and main control, adopting a modular and hot-swappable structure. It interacts through a customized bus interface and is configured with a multi-threaded architecture and a dual-state interlocking mechanism. This invention achieves closed-loop linkage between visual detection and control, improving system real-time performance, security, and reliability, and is suitable for industrial production and equipment inspection scenarios.

Description

Technical Field

[0001] This invention relates to the fields of industrial automation control, computer vision, and human-computer interaction, and in particular to an intelligent monitoring and measurement control method and system based on multi-camera visual detection and state interlocking control mechanism. Background Technology

[0002] In industrial production, equipment inspection, and smart manufacturing scenarios, it is often necessary to deploy multiple cameras to monitor different workstations or areas in real time, and combine them with visual inspection algorithms to analyze the target status in order to assist equipment control or measurement decisions and ensure the accuracy and safety of production inspection.

[0003] Existing multi-camera monitoring systems generally adopt a centralized processing architecture, placing video acquisition, image processing, and interface display in the same execution thread or processing flow. This can easily lead to thread blocking and system resource contention, thereby reducing the real-time performance of image processing and affecting detection stability, failing to meet the needs of real-time monitoring and rapid response in industrial scenarios.

[0004] On the other hand, existing measurement and control systems often lack effective state constraint mechanisms in their human-machine interaction design. During the measurement execution process, operators are still allowed to switch measurement modes or key parameters, which may lead to distorted measurement results, abnormal equipment operation, or even safety risks. At the same time, some systems have not achieved closed-loop linkage between visual detection and control execution, and the detection results cannot be efficiently converted into control commands, further affecting the practicality and reliability of the system.

[0005] Therefore, there is an urgent need for an intelligent monitoring and measurement control scheme that can enable parallel processing of multiple cameras, improve system responsiveness, impose security constraints on key configurations during measurement, and construct a closed-loop control link to address the shortcomings of existing technologies. Summary of the Invention

[0006] The purpose of this invention is to overcome the defects in the existing technology and provide a multi-camera intelligent monitoring and measurement control method and system. By constructing a multi-threaded video processing architecture and a measurement state interlocking mechanism, it solves the problems of high processing latency, insufficient system stability, and susceptibility to interference from misoperation in the existing technology. At the same time, it realizes closed-loop linkage between visual detection and control execution, thereby improving the real-time performance, security and reliability of the industrial monitoring system and meeting the actual needs of industrial automated production detection.

[0007] To achieve the above-mentioned objectives, this invention provides a multi-camera intelligent monitoring and measurement control method, the specific steps of which are as follows:

[0008] 1. Video Acquisition and Preprocessing: Acquire video data from at least two cameras, and perform preprocessing operations such as denoising, scaling, and correction on the video frames to improve the accuracy of subsequent target detection;

[0009] 2. Multi-threaded parallel detection: An independent target detection thread is created for each camera. Each detection thread adopts a dynamic scheduling mechanism using a thread pool to achieve asynchronous decoupling from the video acquisition thread, enabling each detection thread to run in parallel and avoiding blocking problems caused by single-threaded processing. At the same time, a mutex lock mechanism is used to avoid resource contention between threads, significantly improving detection efficiency and system stability.

[0010] 3. Target information extraction: Based on the YOLO8 model optimized and trained according to this invention (optimization direction is small target recognition in industrial scenes, adjusting network anchor box size and loss function weights), the preprocessed video frames are analyzed in real time to accurately obtain the target's category information, three-dimensional spatial position and pose angle;

[0011] 4. Offset Calculation: Based on the extracted three-dimensional spatial position of the target, combined with the preset industrial measurement reference coordinate system, a coordinate transformation algorithm (formula: ΔX=X target-X reference, ΔY=Y target-Y reference, ΔZ=Z target-Z reference) is used to accurately calculate the spatial offset of the target object in the reference coordinate system;

[0012] 5. Closed-loop control execution: The main control module generates measurement auxiliary parameters or equipment adjustment parameters based on the offset and sends them to the external execution device to realize closed-loop control between visual inspection and control execution;

[0013] 6. Measurement State Locking: Upon receiving the measurement start command, the system enters the measurement state and locks the current measurement mode and key configuration parameters through a hierarchical interlocking logic based on a state machine (divided into three levels: pre-locking, formal locking, and emergency unlocking). At the same time, through a dual mechanism of hardware level shielding and software command interception, it blocks commands such as mode switching and parameter modification to prevent accidental operation from interfering with the measurement process.

[0014] 7. Lockout: After the measurement is completed, the system automatically unlocks the lockout, restores the system's parameter configuration and mode switching capabilities, and waits for the next measurement command.

[0015] This invention also provides a multi-camera intelligent monitoring and measurement control system for implementing the above-mentioned multi-camera intelligent monitoring and measurement control method. The system includes: a video acquisition module, a target detection module, a main control module, a graphical user interface module, a display management module, a measurement control module, and a status interlock module. Each module interacts with data through a customized bus interface to ensure the real-time performance and accuracy of data transmission. Furthermore, each module adopts a hot-swappable design for easy individual maintenance and upgrades.

[0016] In one embodiment, the system further includes a processor and a memory; the memory stores computer-executable instructions, and when the processor executes the instructions, it implements the above-described method steps, ensuring the coordinated operation of the various modules of the system.

[0017] The specific functions of each module and the system thread architecture are as follows:

[0018] 1. Thread Architecture: The main thread of this system is the UI thread, responsible for event loop capture and sensing various operation events in the UI. The system sets up two UI sub-threads: Sub-thread 1 is the control thread, mainly responsible for external device control and execution of the measurement process; Sub-thread 2 is the target detection thread, responsible for sending the detected target data to the control thread, guiding the control thread to control the external device to complete target alignment, and providing a guarantee for the measurement operation. The threads adopt a hybrid communication mechanism of "signals and slots + shared memory". Signals and slots are used for command transmission, and shared memory is used for large data transmission of target detection, which avoids thread conflicts and ensures the accuracy of data transmission and the stability of thread operation.

[0019] 2. Video Acquisition Module: Primarily used to acquire video streams from external cameras. This system is programmed in Python and uses OpenCV library functions to achieve real-time acquisition of external video streams. Combined with preprocessing operations, it provides high-quality video data for subsequent object detection.

[0020] 3. Object Detection Module: This module contains multiple independent detection threads (corresponding one-to-one with the number of cameras). It uses the optimized YOLO8 model trained according to this invention to locate and identify specific objects. This model optimizes the backbone network structure and increases the number of feature fusion layers to address the characteristics of small target size and large lighting variations in industrial scenarios. It can accurately extract the object's category information, spatial location, and pose angle, and encapsulate the above information in a preset format before sending it to the main control module. Since the YOLO model detection process requires a large number of comparison and inference operations, integrating it into the UI thread or control thread would cause thread blocking. Therefore, this invention deploys the YOLO detection task in an independent object detection thread with higher priority than the UI thread to ensure efficient parallel operation of each thread.

[0021] 4. Main Control Module: Corresponds to the system's control thread, with built-in functions for capturing and sending external control serial port data frames. It is used to receive external commands and output control commands to drive external devices to operate. This system is equipped with a left tool robotic arm, a right tool robotic arm, and a bottom trolley. The main control module can control the above devices to perform specified actions based on the target category, spatial position, and attitude angle returned by the target detection thread, thereby completing the target alignment operation and facilitating subsequent measurement work.

[0022] 5. Graphical User Interface Module: Developed based on the PyQt image library, this module is the core interface for user interaction with the system. The UI includes video display areas for three cameras: the left camera, the central main camera, and the right camera, with a 2:5:2 aspect ratio to balance local observation and overall view. Function buttons are located at the bottom of the UI and below the main display area for quick access to various operations. Tabs and stack windows are located in the lower left and right corners of the UI, including areas for displaying debugging information, measurement operations, and external device operations, enabling categorized display and operation of various functions and enhancing the human-computer interaction experience.

[0023] 6. Display Management Module: As part of the UI control thread, it supports switching the screen of the central display area via buttons on the UI interface; when it is necessary to operate the left-side tools, the left camera screen can be switched to the main display area for easy observation of operation details; when it is necessary to view the global field of view, the central main camera screen can be switched to the main display area to meet the observation needs of different scenarios.

[0024] 7. Measurement Control Module: This is an external circuit device that communicates with the main system module via the 485 serial port protocol. After the control thread moves the device to the designated measurement position, the user can select the measurement mode in the measurement operation area of ​​the UI interface. After clicking "Confirm," the system triggers a status lock. The measurement control module receives status data and measurement mode instructions sent by the control thread through a signal and slot mechanism, executes the corresponding measurement operation according to the instructions, and after the measurement is completed, feeds back the measurement data to the control thread through the 485 serial port communication protocol. The control thread judges and processes the data, and then sends it to the UI main thread through the signal and slot mechanism to achieve real-time display of the measurement data.

[0025] 8. Status Interlock Module: Used to implement safety constraints during the measurement process, it adopts a dual protection design of hardware and software interlocks. After the user completes status confirmation and command sending on the measurement control panel, the system status automatically enters the formal locked state. The locking signal is synchronously fed back to the hardware level, cutting off the circuit path of the mode switching button. At the same time, the software level intercepts all mode switching commands until the user manually unlocks the device through dual verification (password verification + physical button confirmation). In addition, this module sets up an interlock mechanism for voltage measurement and current measurement buttons. Through GPIO pin level mutual exclusion control, the two buttons cannot be selected at the same time, avoiding equipment failure or measurement data distortion caused by simultaneously activating two measurement modes.

[0026] Compared with the prior art, the present invention has at least the following beneficial effects:

[0027] 1. A multi-threaded parallel processing architecture is adopted, with an independent target detection thread configured for each camera, which effectively reduces the risk of thread blocking, reduces system resource contention, and significantly improves the real-time performance of visual detection;

[0028] 2. By using a state interlocking mechanism and a measurement mode mutual exclusion design, the modification of key parameters and the accidental switching of measurement modes during the measurement process are prevented, thus ensuring the continuity of the measurement process and the consistency of data.

[0029] 3. Achieve synchronization and consistency between the interface state and the internal control logic. Through communication methods such as signals, slots, and mutexes, ensure the coordinated operation of each thread and module, thereby improving system stability.

[0030] 4. Construct a closed-loop control link of "visual inspection - parameter calculation - control execution" to efficiently convert target detection results into equipment control commands, thereby improving the system's practicality and control accuracy;

[0031] 5. The system adopts a modular structure design, with each module having independent functions and clear interfaces, which facilitates subsequent function expansion, maintenance and upgrades;

[0032] 6. Optimize human-computer interaction design by improving user convenience and reducing operational difficulty through reasonable UI layout, screen switching functions, and categorized operation areas;

[0033] 7. Improve the overall security and reliability of industrial monitoring and measurement systems to meet the actual needs of various scenarios such as industrial production and equipment testing. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall system structure, used to illustrate the connection relationships and collaborative logic of the various modules of the system of the present invention;

[0035] Figure 2 This is a schematic diagram of the layout of a multi-camera monitoring interface, used to show the display ratio of the three cameras and the distribution of each functional area in the UI interface;

[0036] Figure 3 This is a schematic diagram of the measurement mode interlock process, used to illustrate the specific procedures for measurement state locking, mutual exclusion control, and unlocking.

[0037] The accompanying drawings are for illustrative purposes only and do not constitute a limitation on the scope of protection of the present invention. Detailed Implementation

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can understand and implement it.

[0039] In one specific embodiment, the workflow of the multi-camera intelligent monitoring and measurement control system of the present invention is as follows:

[0040] 1. After the system starts up, the processor executes the computer-executable instructions stored in the memory to drive the initialization of each module. The video acquisition module starts to acquire video data from the left camera, the central main camera, and the right camera respectively, and performs preprocessing operations such as noise reduction and scaling on the acquired video frames.

[0041] 2. The system dynamically creates target detection threads corresponding to the number of cameras (3 in this embodiment). Each target detection thread independently calls the YOLO8 model to perform target detection on the preprocessed video frames, extract the target's category information, spatial position and pose angle, and send the detection results to the main control module (control thread) through the signal and slot mechanism.

[0042] 3. The main control module parses the received detection results. On the one hand, it overlays the target information onto the corresponding camera video screen on the UI interface for users to view in real time. On the other hand, it calculates the offset based on the deviation between the target's spatial position and the reference coordinate system, generates equipment adjustment parameters, and controls the movement of the left tool robot arm, the right tool robot arm, and the bottom carriage to complete the target alignment operation.

[0043] 4. Users select the measurement mode (voltage measurement or current measurement, the two cannot be selected at the same time) through the measurement operation area of ​​the UI interface, and issue a measurement start command after clicking confirm;

[0044] 5. The state interlock module triggers a state machine switch, the system enters the measurement state, locks the current measurement mode and related configuration parameters, blocks any commands for mode switching or parameter modification, and ensures that the measurement process is not disturbed; at the same time, the main control module sends the measurement mode command to the measurement control module through the 485 serial communication protocol.

[0045] 6. The measurement control module executes the corresponding measurement operation according to the instructions. After the measurement is completed, the measurement data is fed back to the main control module through the 485 serial communication protocol. The main control module judges and processes the data, and then sends it to the UI main thread through the signal and slot mechanism. The measurement data is displayed in real time in the debugging information display area.

[0046] 7. After the measurement is completed, the user issues an unlock command through the UI interface. The system unlocks and restores functions such as parameter configuration and mode switching, waiting for the next measurement or monitoring command.

[0047] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A multi-camera intelligent monitoring and measurement control method and system, characterized in that, Includes the following steps: S1. The video acquisition module simultaneously acquires video data from at least two cameras, and the processor processes the video data to generate corresponding video frames; S2. Create an independent target detection thread for each camera. Each detection thread adopts a dynamic scheduling mechanism using a thread pool to achieve asynchronous decoupling from the video acquisition thread, process the corresponding video frames in parallel, and avoid resource contention between threads through a mutex lock mechanism. S3. In the target detection thread, an optimized deep learning target detection model (optimized for small target recognition in industrial scenes, adjusting network anchor box size and loss function weights) is used to perform image preprocessing and target detection on video frames to obtain the target's category information, three-dimensional spatial position and pose angle. Based on the target's three-dimensional spatial position and a preset industrial measurement reference coordinate system, a coordinate transformation algorithm is used to calculate the spatial offset of the target object in the reference coordinate system. S4. The target detection result is sent to the main control thread through an asynchronous communication mechanism. The main control thread performs fusion processing on the detection result and generates device control parameters based on the target offset and sends them to the execution device to form a closed-loop control based on the visual detection result, controlling the external device to perform target alignment operation. S5. Upon receiving the measurement start command, the system enters the measurement state and locks the measurement mode parameters through a hierarchical interlocking logic based on a state machine (divided into three levels: pre-lock, formal lock, and emergency unlock). S6. In measurement mode, the measurement mode switching command or the mode switching signal is blocked through a dual mechanism of hardware level shielding and software command interception to prevent the mode from being changed during the measurement process. S7. Based on the user's operation, select the corresponding measuring tool and place it at the designated target point; S8. After receiving the measurement end command, unlock the measurement mode through dual verification (password verification + physical button confirmation) and restore the measurement mode switching function.

2. The method according to claim 1, characterized in that: The measurement modes include DC or AC modes, as well as voltage or current measurement modes. The voltage measurement and current measurement modes are mutually exclusive through GPIO pin level control and cannot be selected simultaneously.

3. A multi-camera intelligent monitoring and measurement control system, characterized in that, It includes: a video acquisition module, a target detection module, a main control module, a display management module, a measurement control module, a status interlock module, and a graphical user interface module, among which: S1. The target detection module includes multiple target detection threads, each thread corresponds to one camera, and the thread priority is higher than the UI thread. Target detection is performed using a deep learning target detection model. S2. The main control module is used to receive and fuse the detection results, generate control parameters according to the target offset, and control external devices to complete target alignment and measurement auxiliary operations. S3. The state interlock module adopts a dual protection design of hardware interlock and software interlock, which is used to lock the measurement mode parameters and prohibit mode switching through hierarchical interlock logic during the measurement process; S4. The modules interact with each other through a data communication interface to achieve coordinated operation of video processing, control execution and user interaction. Each module adopts a hot-swappable design for easy individual maintenance and upgrades.

4. The system according to claim 3, characterized in that: It also includes a processor and a memory; the memory stores computer-executable instructions, which, when executed by the processor, implement the steps of the method described in claim 1.

Images