A multi-device heterogeneous integrated software device management system and method

Abstract

This application discloses a multi-device heterogeneous integrated software device management system and method, relating to the field of hardware device management. The system includes: a core framework layer providing unified device management, registration, base class interfaces, and configuration interface specifications; the core framework layer includes a device manager, a device registrar, a device base class, and a device configuration interface base class; a device abstraction layer defining a unified operation interface for physical hardware devices; a device implementation layer encapsulating hardware SDKs or communication protocols from various manufacturers; a hardware layer providing physical hardware devices; the device manager dynamically creating device instances in the device implementation layer based on device signatures through the device registrar, and passing device attributes to the device implementation layer through the configuration serialization interface of the device base class; and the device implementation layer communicating with the physical hardware devices in the hardware layer by calling the manufacturer's SDK or communication protocol. This application significantly reduces the complexity of device access and improves the system's scalability and maintainability.

Description

Technical Field

[0001] This application relates to the field of hardware device management, and in particular to a software device management system and method for heterogeneous integration of multiple devices. Background Technology

[0002] In the field of industrial vision inspection and automation control, it is often necessary to integrate various types of hardware devices to complete complex inspection and control tasks. Common hardware devices include industrial cameras, light source controllers, barcode readers, PLCs, motion controllers, serial port devices, etc. These devices are usually provided by different manufacturers, each using its own independent software development kit (SDK) or communication protocol. To achieve device integration and utilization, the following solutions are commonly used for hardware device management: 1. Direct call solution from the vendor's SDK.

[0003] This solution relies on independent SDK development packages provided by each equipment manufacturer, such as Hikvision's Robotics SDK, Basler Pylon SDK, and Cognex SDK. Developers must write separate interface call code for each device, making it impossible to reuse code across different manufacturers. Because the devices are independent of each other and lack a unified control and management mechanism, system integration becomes highly complex and development efficiency is low.

[0004] 2. Device driver layer solution.

[0005] This solution is based on industry standard protocols such as OPC (OLE for Process Control) and is mainly applicable to traditional industrial equipment such as PLCs. However, for intelligent devices that have been widely used in recent years (such as industrial cameras and barcode readers), the support capability of this type of protocol is limited, making it difficult to meet the requirements of modern inspection systems for the integration of heterogeneous devices.

[0006] 3. Device-independent configuration file scheme.

[0007] This solution requires writing independent configuration files and parsing modules for each device, resulting in inconsistent configuration formats and making the system difficult to maintain and expand. Furthermore, the lack of a unified device scheduling and management mechanism weakens the collaborative capabilities between devices.

[0008] In summary, the existing technology has the following shortcomings in practical applications: High complexity of device access: For each new device accessed, corresponding SDK call code needs to be written. Developers need to learn multiple SDK interfaces and specifications, resulting in low code reuse rate and high maintenance cost.

[0009] Disorganized equipment management: There is a lack of a unified equipment management mechanism, making it difficult to monitor equipment status, resulting in fragmented handling of anomalies and low efficiency in fault location.

[0010] Integration of heterogeneous devices is difficult: different manufacturers and different types of devices have inconsistent interfaces, lack linkage mechanisms, inconsistent data formats, and great difficulty in interaction.

[0011] Poor scalability: Adding new device types requires modifying a large amount of existing code, lacks a flexible plugin mechanism, and is difficult to adapt to rapidly changing business needs.

[0012] Configuration management is not standardized: device configurations are scattered, formats are inconsistent, parsing logic is complex, and there is a lack of a unified configuration persistence mechanism.

[0013] Therefore, how to provide a detection software framework and device management method that can uniformly manage multiple heterogeneous devices, reduce access complexity, and improve system scalability and maintainability is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0014] The purpose of this application is to provide a software device management system and method for multi-device heterogeneous integration, so as to realize a complete device management solution from device abstraction, registration, management, configuration persistence, status monitoring to unified invocation, significantly reducing the complexity of device access and improving the scalability and maintainability of the system.

[0015] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a software device management system for heterogeneous integration of multiple devices, comprising: The core framework layer provides a unified specification for device management, registration, base class interfaces, and configuration interfaces. The core framework layer includes a device manager, a device registrar, a device base class, and a device configuration interface base class. The device abstraction layer, connected to the core layer of the framework, is used to define a unified operation interface for physical hardware devices; the physical hardware devices include cameras, data communication devices, light source controllers, barcode scanners, and drivers. The device implementation layer, connected to the device abstraction layer, is used to encapsulate the hardware SDKs or communication protocols of various manufacturers and complete the interaction with physical hardware devices through the unified operation interface defined by the device abstraction layer or the core layer of the framework. The hardware layer is used to provide physical hardware devices; The device manager dynamically creates device instances in the device implementation layer based on device signatures through the device registrar, and passes device attributes to the device implementation layer through the configuration serialization interface of the device base class. The device implementation layer communicates with the physical hardware device in the hardware layer by calling the vendor SDK or communication protocol. The device manager in the core framework layer interacts with the device abstraction layer through the unified operation interface defined by the device base class to complete the unified invocation of the physical hardware device.

[0016] In one embodiment, the device base class defines at least the following: device type enumeration, connection status enumeration, unified operation interface, data acquisition interface, attribute read / write interface, and configuration serialization interface.

[0017] In one embodiment, the configuration serialization interface uses JSON format to persist device configuration: the device base class defines an export configuration virtual method to convert the attributes recorded in the current physical hardware device object class into JSON, and defines a load configuration virtual method to restore the physical hardware device object class attributes from the JSON; when saving the configuration, the device manager traverses all physical hardware device object classes, calls the export configuration virtual method of each physical hardware device object class to collect configuration information and write it to the configuration file, and reads the configuration file to complete the creation and initialization of physical hardware device object class instances during startup loading.

[0018] In one embodiment, the device registrar includes: The mapping table module is used to store a mapping table from device signatures to device creation functions; The template registration module is used to obtain the device signature by using the static method module signature of template T, and to wrap the default constructor of template T into a creation function and store it in the mapping table; The creation function module is used to use the device creation method to find and call the corresponding device creation function based on the device signature, and return a smart pointer to the new physical hardware device object class.

[0019] In one embodiment, the device manager includes: The device list module stores all created device instances; The configuration file reading module is used to read device configuration entries from a JSON-formatted configuration file. The device creation module is used to create a device instance by calling the device creation method of the device registrar according to the device signature in the configuration entry, and to call the virtual method of loading configuration of the device base class to fill the configuration parameters into the device object class; The device query module is used to query physical hardware device object class instances by device ID, device type, and / or device signature; The timer module is used to periodically check the connection status of physical hardware devices and perform automatic reconnection operations on physical hardware device objects that are offline.

[0020] In one embodiment, the device manager uses a read-write lock mechanism to manage device state: at least one read-write lock is set in the device base class to protect device connection state, device attributes, and error message list, respectively; read operations use read locks to allow concurrent reading by multiple threads; write operations use write locks; a state change signal is emitted when the device state changes, and an error signal is emitted when an error occurs.

[0021] In one embodiment, the device abstraction layer includes: a camera abstraction unit, a data communication abstraction unit, a light source controller abstraction unit, a barcode scanner abstraction unit, and a driver abstraction unit; The camera abstraction unit defines operation interfaces including a start acquisition interface, a stop acquisition interface, an image acquisition interface, and an exposure parameter setting interface; the data communication abstraction unit supports communication protocols including serial communication, TCP / IP communication, UDP communication, Modbus communication, WebSocket communication, and PLC communication; the light source controller abstraction unit defines operation interfaces including a brightness adjustment interface and a strobe control interface; the barcode scanner abstraction unit defines operation interfaces including a barcode scanning interface and a QR code scanning and parsing interface; the driver abstraction unit defines operation interfaces including a motor control interface and a motion control card control interface.

[0022] Secondly, this application provides a device management method, which is applied to the aforementioned multi-device heterogeneous integrated software device management system, comprising: The device abstraction steps include designing a unified device base class and defining general device specifications. The device base class should include at least a device type enumeration, a connection status enumeration, and a unique device identifier, and define a unified operation interface, a data acquisition interface, an attribute read / write interface, and a configuration serialization interface. The device registration process involves establishing a mapping table between device signatures and device creation functions, registering the creation function of the target device class to the mapping table using a template registration method, and dynamically creating device object class instances based on the device signature. In the device loading step, the device manager reads the JSON format configuration file, iterates through the device configuration entries in the configuration file, extracts the device signature from each device configuration entry, calls the dynamic creation method in the device registration step to generate the corresponding device instance, calls the virtual method of loading configuration of the corresponding device instance to fill the configuration parameters into the device object, and adds the corresponding device instance to the device list. The device invocation steps involve obtaining a pointer to the target device through the query interface provided by the device manager, directly calling the unified operation interface defined in the device base class, and executing the overridden methods in the specific implementation class of the target device through polymorphism to complete the operation on the physical hardware device. The status monitoring steps include starting a timer to periodically check the connection status of each physical hardware device in the device list; for devices that are offline, an automatic reconnection operation is performed; when the device status changes, a status change signal is emitted; when a device error occurs, an error signal is emitted.

[0023] In one embodiment, the device signature is the return value of the static method module signature provided for each device class, used to uniquely identify the physical hardware device type; the template registration method registers devices automatically through a static registration mechanism when the program starts.

[0024] In one embodiment, the automatic reconnection operation includes: when the device status switches from online to offline, the device manager starts a reconnection timer, calls the connection operation of the current physical hardware device at preset time intervals, and issues an alarm signal after the current physical hardware device returns to online status or reaches the maximum number of reconnections.

[0025] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a multi-device heterogeneous integrated software device management system and method. The system includes a core framework layer for providing unified device management, registration, base class interfaces, and configuration interface specifications. The core framework layer includes a device manager, a device registrar, a device base class, and a device configuration interface base class. A device abstraction layer connected to the core framework layer defines a unified operation interface for physical hardware devices. A device implementation layer connected to the device abstraction layer encapsulates hardware SDKs or communication protocols from various vendors and interacts with physical hardware devices through the unified operation interface defined by the device abstraction layer or the core framework layer. A hardware layer provides physical hardware devices. The device manager in the core framework layer dynamically creates device instances in the device implementation layer based on device signatures through the device registrar and passes device attributes to the device implementation layer through the configuration serialization interface of the device base class. The device implementation layer communicates with the physical hardware devices in the hardware layer by calling vendor SDKs or communication protocols. The device manager in the core framework layer interacts with the device abstraction layer through the unified operation interface defined by the device base class to complete the unified invocation of physical hardware devices. This application implements a complete device management solution, from device abstraction, registration, management, configuration persistence, status monitoring to unified invocation, which significantly reduces the complexity of device access and improves the scalability and maintainability of the system. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram of a multi-device heterogeneous integrated software device management system provided in an embodiment of this application; Figure 2 An overall architecture diagram of a multi-device heterogeneous integrated software device management system provided in an embodiment of this application; Figure 3 A flowchart illustrating device registration and dynamic creation is provided as an embodiment of this application; Figure 4 A flowchart illustrating a method for calling a device via a unified interface, as provided in an embodiment of this application; Figure 5 This is a state transition diagram of the device connection state provided in an embodiment of this application; Figure 6 A flowchart of device configuration management provided in one embodiment of this application; Figure 7 A flowchart illustrating the device configuration persistence process provided in one embodiment of this application; Figure 8 An inheritance hierarchy diagram of a device class provided in one embodiment of this application; Figure 9 This is a flowchart illustrating the collaborative operation of multiple devices in a practical AI detection application, as provided in one embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] In one exemplary embodiment, such as Figure 1 As shown, a multi-device heterogeneous integrated software device management system is provided, including: a framework core layer, which provides a unified specification for device management, registration, base class interface and configuration interface, the framework core layer including device manager, device registrar, device base class and device configuration interface base class.

[0031] In one embodiment, the device base class defines at least the following: device type enumeration, connection status enumeration, unified operation interface, data acquisition interface, attribute read / write interface, and configuration serialization interface.

[0032] In one embodiment, the configuration serialization interface uses JSON format to persist device configuration: the device base class defines a virtual method toJson() (export configuration) to convert the current physical hardware device attributes into QJsonObject (JSON), and defines a virtual method load(const QJsonObject&) (load configuration) to restore the physical hardware device object class attributes from the QJsonObject; when saving the configuration, the device manager iterates through all physical hardware device object classes, calls the toJson() virtual method of each physical hardware device object class to collect configuration information and write it to the configuration file, and reads the configuration file to complete the creation and initialization of physical hardware device object class instances during startup loading.

[0033] In one embodiment, the device registrar includes: The mapping table module is used to store a mapping table from device signatures to device creation functions.

[0034] The template registration module is used to obtain the device signature using the static method signature() (module signature) of template T, and to wrap the default constructor of template T into a creation function and store it in the mapping table.

[0035] The creation function module is used to use the create(const QString&signature) (create device) method to find and call the corresponding device creation function based on the device signature, and return a smart pointer to the new physical hardware device object class.

[0036] In one embodiment, the device manager includes: The Device List module stores all created device instances.

[0037] The configuration file reading module is used to read device configuration entries from a JSON-formatted configuration file.

[0038] The device creation module is used to create a device instance by calling the create(const QString&signature) method of the device registrar based on the device signature in the configuration entry, and to call the load(const QJsonObject&) virtual method of the device base class to fill the configuration parameters into the device object.

[0039] The device query module is used to query physical hardware devices by device ID, device type, and / or device signature.

[0040] The timer module is used to periodically check the connection status of physical hardware devices and perform automatic reconnection operations on offline physical hardware devices.

[0041] In one embodiment, the device manager uses a read-write lock mechanism to manage device state: at least one QReadWriteLock is set in the device base class to protect the device connection state, device attributes, and error message list, respectively; read operations use read locks, allowing concurrent reading by multiple threads; write operations use write locks; when the device state changes, a stateChanged signal is emitted, and when an error occurs, an errorOccurred signal is emitted.

[0042] The device abstraction layer, connected to the core layer of the framework, is used to define a unified operation interface for physical hardware devices; the physical hardware devices include cameras, data communication devices, light source controllers, barcode scanners, and drivers.

[0043] In one embodiment, the device abstraction layer includes: a camera abstraction unit, a data communication abstraction unit, a light source controller abstraction unit, a barcode scanner abstraction unit, and a driver abstraction unit.

[0044] The camera abstraction unit defines operation interfaces including a start acquisition interface, a stop acquisition interface, an image acquisition interface, and an exposure parameter setting interface; the data communication abstraction unit supports communication protocols including serial communication, TCP / IP communication, UDP communication, Modbus communication, WebSocket communication, Siemens PLC communication, and Mitsubishi PLC communication; the light source controller abstraction unit defines operation interfaces including a brightness adjustment interface and a strobe control interface; the barcode scanner abstraction unit defines operation interfaces including a barcode scanning interface and a QR code scanning and parsing interface; the driver abstraction unit defines operation interfaces including a motor control interface and a motion control card control interface.

[0045] The device implementation layer, connected to the device abstraction layer, is used to encapsulate hardware SDKs or communication protocols from various manufacturers. Through the unified operation interface defined by the device abstraction layer or the core layer of the framework, it completes the interaction with physical hardware devices.

[0046] The hardware layer is used to provide physical hardware devices.

[0047] The device manager dynamically creates device instances in the device implementation layer based on device signatures through the device registrar, and passes device attributes to the device implementation layer through the configuration serialization interface of the device base class. The device implementation layer communicates with the physical hardware device in the hardware layer by calling the vendor SDK or communication protocol. The device manager in the core framework layer interacts with the device abstraction layer through the unified operation interface defined by the device base class to complete the unified invocation of the physical hardware device.

[0048] The following section uses a complete industrial vision software equipment management system as an example to illustrate in detail the specific implementation method of the multi-device heterogeneous integration software equipment management system of this application.

[0049] like Figure 2 As shown, the software equipment management system of this application comprises four layers from top to bottom: a core framework layer, a device abstraction layer, a device implementation layer, and a hardware layer. The core framework layer is connected to the application layer, which includes AI detection algorithms, business logic, and a device invocation layer. The core framework layer further includes a device manager, a device registrar, a device base class, and a device configuration interface base class. The device abstraction layer includes camera abstraction units, data communication abstraction units, light source controller abstraction units, barcode scanner abstraction units, and driver abstraction units. The device implementation layer includes specific device implementation units from various manufacturers, such as Basler camera implementation units, Hikvision camera implementation units, Modbus master station implementation units, and TCP server implementation units. The hardware layer includes physical hardware such as industrial cameras, PLC devices, serial port devices, light source controllers, barcode readers, and robots. Figure 2 It shows the hierarchical relationship between each layer and the data flow.

[0050] The core framework layer is the system's central hub, responsible for the unified management and scheduling of all devices. The Device Manager, as the core management unit, maintains a global device list, providing functions for device registration, querying, status monitoring, and lifecycle management. The Device Registry maintains a mapping table from device signatures to device creation functions, supporting dynamic device creation. The Device Base Class defines the common interfaces that all devices must implement, including device type enumeration, connection status enumeration, unified operation interface, data acquisition interface, attribute read / write interface, and configuration serialization interface. The Device Configuration Interface Base Class provides a unified configuration panel specification for each device type, ensuring a consistent user experience across different device parameter configuration interfaces.

[0051] The specific process of device registration and dynamic creation mechanism, such as Figure 3 As shown, firstly, each specific device class calls the template registration method through a static registration mechanism to register its device signature and creation function mapping relationship in the device registrar's mapping table. When the system needs to create a device instance, the device manager calls the device registrar's creation method based on the device signature in the configuration file. The device registrar looks up the corresponding creation function in the mapping table, executes it, and returns a pointer to the newly created device instance. The inheritance hierarchy of device classes is shown below. Figure 8As shown, the top layer is the Device abstract base class, which defines the common interface for all devices; the second layer consists of device type abstract classes, including Camera, DataCom, LightController, Barcode Scanner, and Driver abstract classes, which inherit from the Device base class; the third layer consists of specific vendor implementation classes, such as HikCamera, BaslerCamera, ModbusMaster, and Siemens PLC Communication (TcpClient), which inherit from their respective device type abstract classes. Figure 8 It clearly demonstrates the complete inheritance relationship from abstract to concrete.

[0052] First, each specific device class needs to implement a static method `signature()`, which returns a unique device identifier of type string. For example, the Hikvision camera class returns "HikCamera", the Basler camera class returns "BaslerCamera", and the Modbus master class returns "ModbusMaster". This device signature is the key identifier of the entire registration mechanism.

[0053] The device registrar internally maintains a mapping table from strings to function objects. The keys of this mapping table are device signatures, and the values ​​are creation functions that accept no arguments and return smart pointers to newly created device instances.

[0054] Template registration method: registerDevice <t>(Registering the device) is the core of implementing static registration. This method accepts a specific device class T as a template parameter. Inside the function, it first calls the module signature T::signature() to obtain the signature of the device class, and then constructs a lambda expression as the creation function. This lambda expression internally creates a new instance of class T using the new operator or the make_shared function. Finally, the key-value pair of the signature and the creation function is inserted into the mapping table.

[0055] To achieve automatic registration upon program startup, this application employs static initialization. In the implementation file of each specific device class, a static boolean variable is defined, and its initial value is determined by calling `registerDevice`. <t>This is done by setting static variables. Since static variables are initialized when the program loads, all device classes will automatically complete registration before the main function executes. This method requires no modification to any core framework code; when adding a new device, simply create a new device class file and add the static registration code to it, and the framework will recognize and manage it.

[0056] When the system needs to create a device instance, the Device Manager reads the device signature from the configuration file, calls the device registrar's create device method, and passes the device signature as a parameter. The create device method searches for the creation function corresponding to the signature in the mapping table. If found, it calls the function and returns a pointer to the newly created device instance; otherwise, it returns a null pointer and logs an error.

[0057] This dynamic creation mechanism allows the device manager to decide which devices to create at runtime based on the configuration, without needing to know all device types in advance, greatly enhancing the system's flexibility and scalability.

[0058] The device abstraction layer, located below the core framework layer, defines unified abstract interfaces for different types of devices. The camera abstraction unit defines common operations for all camera devices, including starting acquisition, stopping acquisition, image acquisition, setting exposure time, and setting gain. The data communication abstraction unit defines unified read / write interfaces for communication methods such as serial port, TCP / IP, UDP, Modbus, and WebSocket. The light source controller abstraction unit defines operations such as brightness adjustment, flicker control, and trigger mode setting. The barcode scanner abstraction unit defines operations such as barcode scanning, QR code parsing, and trigger scanning. The driver abstraction unit defines operations such as motor motion control, point-to-point motion, continuous motion, and I / O control.

[0059] The device implementation layer encapsulates the hardware SDKs and specific communication protocols implemented by various manufacturers. Taking industrial cameras as an example, the Hikvision camera implementation unit encapsulates the Hikvision robot SDK, packaging functions such as device connection, parameter configuration, and image acquisition into methods conforming to the device base class interface specification. Similarly, the Basler camera implementation unit encapsulates the Basler Pylon SDK. Taking data communication as an example, the Modbus master implementation unit encapsulates the standard functions of the Modbus protocol, supporting communication with slave devices such as PLCs via serial port or TCP. Each specific device implementation class inherits from the corresponding device type abstract class and overrides all pure virtual functions.

[0060] The hardware layer includes actual physical hardware devices, such as industrial cameras, PLCs, serial port devices, light source controllers, barcode readers, and robots. These devices connect to the industrial control computer via physical interfaces such as Ethernet, USB, and serial ports, receiving control commands from the software system and returning acquired data or status information.

[0061] When the system starts, the Device Manager reads the configuration file and, based on the device signature in the configuration file, calls the creation method of the device registrar to dynamically generate the corresponding device implementation layer object. The device registrar looks up the creation function corresponding to the device signature in the mapping table, executes it, and returns the newly created device instance. The Device Manager then calls the virtual method for loading the configuration of this device instance, filling the device object with configuration parameters (such as IP address, port number, communication parameters, etc.). During the initialization process, the device instance calls the connection function of the vendor's SDK based on the configuration parameters to establish an actual connection with the physical hardware device at the hardware layer. At this point, the device has completed loading and initialization and is available for upper-layer calls.

[0062] This embodiment takes loading a Hikvision industrial camera into an industrial vision inspection system as an example to explain in detail the specific process of device loading and initialization. When the system starts, the device manager reads the JSON format configuration file and parses out the configuration entries for each device, including the device signature and device attribute parameters. The device manager calls the device registrar to dynamically create the corresponding device instance based on the device signature. Subsequently, the device manager calls the virtual method of loading configuration for the device instance to populate the configuration parameters into the device object. Finally, the device manager adds the device instance to the device list and establishes connections for status change signals and error signals.

[0063] The system's device configuration file uses JSON format and is named devices.json. This file contains an array named "devices", where each element corresponds to the configuration information of a device. Taking a Hikvision camera as an example, its configuration entries include: a device module signature field with the value "HikCamera"; a device ID field with the value "camera_front"; and a device attribute object containing specific camera parameters such as IP address, exposure time, and gain.

[0064] When the system starts up, Device Manager first calls the configuration file reading module to open the devices.json file and read all its contents. Then, it uses a JSON parsing library to parse the file contents into a JSON object and obtain the "devices" array.

[0065] The Device Manager iterates through each configuration entry in the "devices" array. For the currently accessed camera configuration entry, the Device Manager first extracts the value "HikCamera" from the device module signature field, and then calls the device registrar's create device method, passing this signature as a parameter. The device registrar searches for the creation function corresponding to "HikCamera" in the mapping table, executes the function, creates and returns a smart pointer to a Hikvision camera instance.

[0066] After obtaining a device instance, the Device Manager calls the instance's virtual method for loading configuration, passing the entire configuration entry (including the device ID and device properties) as a JSON object. Internally, the virtual method first calls the base class's virtual method for loading configuration, extracting the device ID from the JSON object and storing it in a member variable. Then, it parses the device property object, reading parameters such as IP address, exposure duration, and gain one by one, and writes these parameters into the device property dictionary by calling the device property setting method.

[0067] After the Device Manager completes the property population, it calls the connection method of the device instance (by passing the ACTION_CONNECT action through the device interface). Internally, the connection method calls the Hikvision SDK's connection functions MV_CC_CreateHandle and MV_CC_OpenDevice based on the configured IP address to establish a connection with the actual physical camera. Upon successful connection, the device status changes from unknown to online, and the device instance transmits a status change signal to notify other modules in the system that the camera is ready.

[0068] Finally, the Device Manager adds the device instance to the global device list and connects the device instance's state change signals and error signals to the corresponding slot functions of the Device Manager for unified state monitoring and exception handling in the future.

[0069] The specific process of device configuration management and the persistence of device configuration using JSON format, such as... Figure 6 and Figure 7 As shown, configuration management includes two processes: configuration loading and configuration saving. The configuration loading process is as follows: The Device Manager reads the devices.json configuration file, parses the JSON object, iterates through each device configuration entry, creates a device instance based on the device signature, and calls the virtual method to load the configuration and populate the configuration parameters. The configuration saving process is as follows: After the user modifies device parameters in the configuration interface, they call the device instance's "Set Device Attributes" method to update the attributes. The Device Manager iterates through all device instances, calls the virtual method to export the configuration for each instance to collect configuration information, and writes the entire configuration to the devices.json configuration file.

[0070] In traditional device configuration management, configuration information is often scattered throughout the code or uses various configuration files in different formats, leading to chaotic, difficult-to-maintain, and difficult-to-migrate configuration management. This application adopts a unified JSON format to centrally manage the configuration information of all devices, and implements configuration serialization and deserialization through two virtual methods defined in the device base class: export configuration and load configuration.

[0071] The virtual method for exporting configuration converts the current state and properties of the device instance into a JSON object. In its implementation, the device base class already implements general serialization logic: converting the device ID and the entire property dictionary into a JSON object. Derived classes can extend this method as needed to add device-specific configuration information. For example, the camera class can add camera-specific parameters such as resolution and pixel format to the base class's serialization functionality.

[0072] The virtual method for loading configuration performs the opposite operation, restoring the state and properties of the device instance from the JSON object. The device base class first extracts and sets the device ID from the JSON object, then iterates through each key-value pair in the property dictionary, calling the device property set method to restore the properties one by one. When a derived class overrides the virtual method for loading configuration, it should first call the base class's virtual method for loading configuration, and then process the configuration information specific to the derived class.

[0073] In the system configuration saving process, when a user modifies device parameters in the configuration interface and clicks the save button, the Device Manager first iterates through all device instances in the device list. For each device instance, it calls its virtual method to export configuration to obtain the device's configuration JSON object, collecting all device configuration objects into an array. Then, the Device Manager creates a root JSON object and assigns this array to the "devices" field. Finally, this root JSON object is formatted as a string and written to the devices.json file.

[0074] During the system startup configuration loading process, the Device Manager reads the devices.json file, parses it into a root JSON object, and obtains the "devices" array. For each configuration entry in the array, it extracts the device module signature field, calls the device registrar to create the corresponding device instance, and then calls the virtual method of loading configuration for that instance, passing in the configuration entry to complete the initialization of device properties.

[0075] This unified JSON configuration persistence mechanism brings the following technical benefits: First, all device configurations are centrally managed, making them easy to view and modify; second, the configuration format is standardized, allowing for management using a version control system; third, it supports importing and exporting configurations, facilitating project migration and deployment; and fourth, the configurations are in plain text format, providing excellent human readability.

[0076] When device operation is required, the system first obtains a pointer to the target device through the device query interface provided by the device manager (e.g., querying by device ID). Since all device pointers are of the device base class type, the caller only needs to invoke the unified operation interface defined in the device base class (e.g., operating the device or retrieving device data). At runtime, the system automatically executes the overridden methods in the specific device implementation class based on polymorphism. These methods internally call the vendor's SDK or communicate directly with the physical hardware device to complete the actual operation. In this way, the upper-layer logic does not need to care about which manufacturer or type of device is being used at the bottom layer, achieving true hardware independence.

[0077] The specific process of calling the device through a unified interface, such as Figure 4 As shown, the upper-layer business logic obtains the pointer to the target device through the device query interface of the device manager; then it directly calls the unified operation interface defined in the device base class (such as operating the device or obtaining device data); due to the polymorphism mechanism, the method overridden in the specific device implementation class is actually executed; the specific device implementation class internally calls the manufacturer's SDK or communicates directly with the physical hardware, and returns the result after completing the actual operation.

[0078] In traditional device invocation methods, different types and manufacturers of devices have completely different interfaces. For example, the image acquisition function for Hikvision cameras is named MV_CC_StartGrabbing, the image acquisition function for Bassler cameras is named PylonGrab, and the read / write functions for Modbus devices are completely different. Developers need to learn the specific API for each device and write a lot of redundant code.

[0079] This application completely solves this problem through the design of a unified operation interface. For example... Figure 4 As shown, the device base class defines two core unified interfaces: operating the device and acquiring device data. The operating device interface is used to perform device operation functions, such as connecting, disconnecting, starting data acquisition, stopping data acquisition, turning on the light source, and turning off the light source. The acquiring device data interface is used to obtain data from the device, such as images, barcodes, and sensor values.

[0080] Taking image acquisition from an industrial camera as an example, when the upper-layer business logic needs to acquire an image, it first obtains the device pointer of the camera by passing in the device ID "camera_front" through the device manager's "Get Device Instance" method. The type of this pointer is DevicePtr, which is a pointer to the device base class, not a pointer to a specific camera class.

[0081] Subsequently, the upper-level logic calls the device operation method of the device pointer, passing in the action constant "Start Acquisition". Since this pointer actually points to a Hikvision camera instance (or Basler camera instance), the runtime polymorphism mechanism will call the overridden device operation method in the Hikvision camera class (or Basler camera class). This overridden method internally calls the actual acquisition function of the vendor's SDK, such as MV_CC_StartGrabbing or PylonGrab.

[0082] After data acquisition is complete, the upper-layer logic calls the method to obtain device data, passing in the data key "get image". Similarly, the polymorphism mechanism calls the overridden method to obtain device data in the specific camera class. This method internally calls the image acquisition function of the vendor's SDK, such as MV_CC_GetImage or PylonRetrieveImage, to obtain the image data and convert it into a unified image data format (such as QImage or cv::Mat) before returning it.

[0083] With this design, the upper-level business logic calling code remains completely consistent regardless of whether the underlying camera is a Hikvision or a Basler camera. When it is necessary to replace the Hikvision camera with a Basler camera, only the device module signature field in the configuration file needs to be modified, without modifying any business logic code.

[0084] This application also provides a device management method, which is applied to the above-mentioned multi-device heterogeneous integrated software device management system, including: Step S1: Device abstraction step, design a unified device base class, define general device specifications; the device base class includes at least device type enumeration, connection status enumeration, device unique identification code, and defines a unified operation interface, data acquisition interface, attribute read and write interface, and configuration serialization interface.

[0085] Step S2: Device registration step, establish a mapping table between device signature and device creation function, register the creation function of the target device class to the mapping table through the template registration method, and dynamically create device object class instance according to device signature.

[0086] In one implementation, the device signature is the return value of the static method `signature()` provided for each device class, used to uniquely identify the physical hardware device type; the template registration method `registerDevice` <t>It is executed automatically through a static registration mechanism when the program starts.

[0087] Step S3: Device loading step. The device manager reads the JSON format configuration file, iterates through the device configuration entries in the configuration file, extracts the device signature from each device configuration entry, calls the dynamic creation method in the device registration step to generate the corresponding device instance, and then calls the virtual method load(const QJsonObject&) of the corresponding device instance to fill the configuration parameters into the device object, and adds the corresponding device instance to the device list.

[0088] Step S4: Device invocation step. Obtain the pointer of the target device through the query interface provided by the device manager, directly call the unified operation interface defined in the device base class, and execute the overridden method in the specific implementation class of the target device through polymorphism to complete the operation on the physical hardware device.

[0089] Step S5: Status monitoring step. Start a timer to periodically check the connection status of each physical hardware device in the device list. For devices with an offline connection status, perform an automatic reconnection operation. When the device status changes, transmit a status change signal. When a device error occurs, transmit an error signal.

[0090] In one embodiment, the automatic reconnection operation includes: when the device status switches from online to offline, the device manager starts a reconnection timer, calls the connection operation of the current physical hardware device at preset time intervals, and issues an alarm signal after the current physical hardware device returns to online status or reaches the maximum number of reconnections.

[0091] This embodiment details the specific process of device status monitoring and automatic reconnection mechanism, such as... Figure 5 As shown, the device connection status includes four types: unknown, offline, connecting, and online. The device is in the unknown state when the system starts; it enters the connecting state after a connection operation is performed; if the connection is successful, it enters the online state; if the connection fails, it enters the offline state; if a heartbeat timeout or communication abnormality occurs in the online state, it switches to the offline state; in the offline state, an automatic reconnection is triggered by a timer, and it re-enters the connecting state; when the user actively disconnects, it switches from the online state to the offline state.

[0092] Device status management is crucial for the stable operation of industrial monitoring systems. When the system starts, device instances are created but not yet connected to physical devices; at this point, the devices are in an unknown state. After the device manager invokes the device's connection operation, the device status switches to "connecting." If the connection is successful, the device status switches to "online" and transmits a status change signal; if the connection fails, the device status switches to "offline," and an error message is logged and an error signal is transmitted.

[0093] The Device Manager internally maintains a timer that triggers a timeout event every second. Each time the timer expires, the Device Manager iterates through all devices in the device list and calls a heartbeat detection operation for each device. For devices that are online, the heartbeat detection is implemented by calling the device operation method and passing in a "ping" action. If the return result is false or a timeout occurs, the device is considered offline, its status changes from online to offline, and it emits the corresponding status change signal and error signal.

[0094] For devices in offline status, the Device Manager initiates an automatic reconnection mechanism. Specifically, the Device Manager maintains a reconnection timer and a reconnection count variable for each offline device. Upon first entering offline status, the reconnection timer starts, triggering a reconnection attempt every 5 seconds. During each reconnection attempt, the Device Manager invokes the device's connection operation. If the connection is successful, the device status returns to online, the reconnection timer stops, and the reconnection count is reset to zero. If the connection fails, the reconnection count is incremented by 1. When the reconnection count reaches the maximum number of attempts (e.g., 12 times, or 1 minute), reconnection attempts stop, and a persistent error signal is emitted, prompting maintenance personnel for manual intervention.

[0095] When a device encounters an error, the device instance adds the error message to its internal error message list and emits an error signal. Upon receiving this signal, the Device Manager can determine the subsequent handling strategy based on the severity of the error: for recoverable errors (such as communication timeouts), it automatically performs a reconnection operation; for unrecoverable errors (such as physical device disconnection), it logs the error and notifies maintenance personnel.

[0096] In addition, the device manager provides an error query interface, allowing upper-level business logic to call the device's error information retrieval method at any time to obtain the current error list for fault diagnosis and location.

[0097] This embodiment uses an Automatic Optical Inspection (AOI) system for PCB boards as an example to illustrate in detail the specific process of collaborative work among various heterogeneous devices, such as... Figure 9 As shown, a typical process includes: the PLC controls the conveyor belt to stop; the light source controller adjusts the brightness and turns on the light source; the camera captures images and acquires image data; the barcode reader scans and parses the barcode information; the image processing algorithm detects and analyzes the captured images; the PLC controls the conveyor belt to start and performs sorting actions based on the detection results. Figure 9 All devices in the application are invoked through the unified interface of this application framework, and the coordination between devices is controlled by the business logic layer.

[0098] The AOI system needs to integrate the following equipment: an industrial camera for capturing PCB images, a ring light controller for providing illumination, a barcode reader for reading the barcode information of the PCB board, a PLC for controlling the start and stop of the conveyor belt and the movement of the sorting cylinders, and a robot for picking up defective boards and placing them in the defective product area.

[0099] When the system starts, the Device Manager creates instances of the four devices mentioned above sequentially according to the devices.json configuration file, completing the connection and initialization of each device. For example, the camera's configuration includes IP address 192.168.1.100, exposure time 10000 microseconds, and gain 0; the light source controller's configuration includes serial port COM3 and initial brightness value 80; the barcode reader's configuration includes TCP port 5000; and the PLC's configuration includes Modbus TCP address 192.168.1.200.

[0100] When the PCB board moves on the conveyor belt and triggers the photoelectric sensor, the detection process begins. First, the business logic layer obtains the pointer of the PLC device through the device manager, calls the device operation method and passes in the "stop" action. The PLC executes the instruction to stop the conveyor belt and precisely positions the PCB board below the camera's field of view.

[0101] Next, the business logic layer obtains a pointer to the light source controller device, calls the device property setting method to set the brightness to 80%, and then calls the device operation method with the "turn on the light" action to light up the ring light source. To ensure the stability of the light source, the system briefly delays for 50 milliseconds.

[0102] Then, the business logic layer obtains a pointer to the camera device, calls the device operation method with the "take a picture" action as input, and the camera performs single-frame acquisition. After acquisition, it calls the device data acquisition method with the "image" key-value pair as input to obtain the acquired image data. This image data is then passed to the image processing algorithm module for defect detection.

[0103] At the same time, the business logic layer obtains a pointer to the barcode reader device, calls the device data acquisition method and passes in the "barcode" key value. The barcode reader scans and returns the barcode information on the PCB board for traceability and recording.

[0104] After the image processing algorithm completes the detection, it returns the detection result (pass or fail). Based on the detection result, the business logic layer controls the sorting action through the PLC device: if the detection is passable, the PLC controls the conveyor belt to start and send the PCB board to the next process; if the detection is failable, the PLC stops the conveyor belt and sends a signal to the robot, which then performs a gripping action to remove the defective product from the conveyor belt.

[0105] Finally, the business logic layer obtains a pointer to the light source controller device, calls the device operation method and passes in the "turnOff" action to turn off the light source to save energy and extend its lifespan.

[0106] Throughout the collaborative workflow, all device calls are completed through a unified device base class interface, eliminating the need to concern oneself with the specific communication protocols and SDK differences of each device. For example, the code controlling the light source's on / off state is completely consistent with the code for camera image acquisition, greatly simplifying the development of business logic. Furthermore, if a change in camera brand or PLC model is required in the future, only the configuration file needs to be modified and the corresponding device implementation class added; no modification to any business logic code is required, fully demonstrating the scalability and maintainability of this application's framework.

[0107] Industrial vision inspection systems typically involve multi-threaded concurrent access. For example, one thread might handle image acquisition, another image processing, and a third device status monitoring. In such a multi-threaded environment, it is crucial to ensure the thread safety of device status and attribute data.

[0108] This application incorporates a thread-safe mechanism within its device base class. Specifically, the device base class defines three read-write lock objects, QReadWriteLock, to protect the device connection state variables, the device property dictionary, and the error message list, respectively.

[0109] When multiple threads need to read device status simultaneously (for example, a status monitoring thread is checking if the device is online, while a business logic thread also wants to know the device status), these read operations can be executed concurrently. Each read operation requests a read lock on the status lock before accessing the status variable, and releases the read lock after reading is complete. Because the read lock is a shared lock, multiple read operations can hold it simultaneously without blocking each other.

[0110] When a state change is required (e.g., changing the status from unknown to online after a device has successfully connected), a write operation requests a write lock. A write lock is an exclusive lock; when one thread holds the write lock, all other threads' read and write lock requests are blocked until the write lock is released. This ensures that other threads do not read inconsistent state data during a state change.

[0111] A similar mechanism is used for reading and writing device attributes. When business logic needs to modify device parameters (such as adjusting camera exposure time), the method for setting device attributes acquires a write lock on the attribute lock, which is released after modification. When other threads call the method to retrieve device attributes and read attribute values, they acquire a read lock on the attribute lock, allowing multiple read operations to occur concurrently.

[0112] The error message list is protected using the same mechanism. When a device error occurs, the `Log Error Information` method acquires a write lock on the error lock and adds the error message to the list. When the `Get Error Information` method is called to retrieve all error messages, it acquires a read lock on the error lock and returns a copy of the error list.

[0113] This read-write lock mechanism offers higher concurrency performance compared to traditional mutexes because read operations account for the vast majority of scenarios, and concurrent execution of read locks significantly reduces thread waiting time. Meanwhile, the exclusivity of write operations ensures data consistency, preventing data corruption or logical errors caused by race conditions.

[0114] This application has the following beneficial effects: Reduce device integration complexity: With a unified device base class and registration mechanism, adding a new device only requires implementing the device class and registering it, reducing the amount of integration code by more than 80%, and eliminating the need for developers to learn how to use various SDKs.

[0115] Improve equipment management efficiency: Achieve centralized management of equipment through equipment management classes, support multiple ways to query equipment, monitor equipment status in real time, automatically alarm for anomalies, and improve operation and maintenance efficiency by more than 50%.

[0116] Simplify the integration of heterogeneous devices: Through a unified interface design, cameras, PLCs, and light sources all use the same calling method, and the AI ​​detection algorithm layer does not need to care about the differences in the underlying devices.

[0117] Enhanced system scalability: Supports plug-in extensions; simply create a new device class and register it without modifying the core framework code; supports hot-swappable devices.

[0118] Improve the standardization of configuration management: All device configurations are stored in a unified JSON format, supporting the import and export of configurations, which facilitates project management and version control.

[0119] To ensure multi-threaded safety: A read-write lock mechanism is used to allow multiple threads to concurrently read the device status, and write operations are automatically locked to ensure data consistency.

[0120] Enhance device status observability: Provide a complete signaling mechanism, including status change signals, error message signals, etc., to support real-time monitoring of device status and quick problem location.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.< / t> < / t> < / t>

Claims

1. A software device management system for heterogeneous integration of multiple devices, characterized in that, include: The core framework layer provides a unified specification for device management, registration, base class interfaces, and configuration interfaces. The core framework layer includes a device manager, a device registrar, a device base class, and a device configuration interface base class. The device abstraction layer, connected to the core layer of the framework, is used to define a unified operation interface for physical hardware devices; the physical hardware devices include cameras, data communication devices, light source controllers, barcode scanners, and drivers. The device implementation layer, connected to the device abstraction layer, is used to encapsulate the hardware SDKs or communication protocols of various manufacturers and complete the interaction with physical hardware devices through the unified operation interface defined by the device abstraction layer or the core layer of the framework. The hardware layer is used to provide physical hardware devices; The device manager dynamically creates device instances in the device implementation layer based on device signatures through the device registrar, and passes device attributes to the device implementation layer through the configuration serialization interface of the device base class. The device implementation layer communicates with the physical hardware device in the hardware layer by calling the vendor SDK or communication protocol. The device manager in the core framework layer interacts with the device abstraction layer through the unified operation interface defined by the device base class to complete the unified invocation of the physical hardware device.

2. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The device base class defines at least the following: device type enumeration, connection status enumeration, unified operation interface, data acquisition interface, attribute read / write interface, and configuration serialization interface.

3. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The configuration serialization interface uses JSON format to persist device configurations: the device base class defines an export configuration virtual method to convert the attribute information recorded in the current physical hardware device object class into JSON, and defines a load configuration virtual method to restore the physical hardware device attributes from the JSON; when saving the configuration, the device manager traverses all physical hardware device object classes, calls the export configuration virtual method of each physical hardware device object class to collect configuration information and write it to the configuration file, and reads the configuration file to complete the creation and initialization of physical hardware device object class instances during startup loading.

4. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The device registrar includes: The mapping table module is used to store a mapping table from device signatures to device creation functions; The template registration module is used to obtain the device signature by using the static method module signature of template T, and to wrap the default constructor of template T into a creation function and store it in the mapping table; The creation function module is used to use the device creation method to find and call the corresponding device creation function based on the device signature, and return a smart pointer to the new physical hardware device object class.

5. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The device manager includes: The device list module stores all created device instances; The configuration file reading module is used to read device configuration entries from a JSON-formatted configuration file. The device creation module is used to create a device instance by calling the device creation method of the device registrar according to the device signature in the configuration entry, and to call the virtual method of loading configuration of the device base class to fill the configuration parameters into the device object class; The device query module is used to query physical hardware device object class instances by device ID, device type, and / or device signature; The timer module is used to periodically check the connection status of physical hardware devices and perform automatic reconnection operations on physical hardware device objects that are offline.

6. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The device manager uses a read-write lock mechanism to manage device status: at least one read-write lock is set in the device base class to protect the device connection status, device attributes, and error message list, respectively; read operations use read locks, allowing multi-threaded concurrent reading; write operations use write locks; when the device status changes, a status change signal is emitted, and when an error occurs, an error signal is emitted.

7. The software device management system for heterogeneous integration of multiple devices according to claim 1, characterized in that, The device abstraction layer includes: a camera abstraction unit, a data communication abstraction unit, a light source controller abstraction unit, a barcode scanner abstraction unit, and a driver abstraction unit; The camera abstraction unit defines operation interfaces including a start acquisition interface, a stop acquisition interface, an image acquisition interface, and an exposure parameter setting interface; the data communication abstraction unit supports communication protocols including serial communication, TCP / IP communication, UDP communication, Modbus communication, WebSocket communication, and PLC communication; the light source controller abstraction unit defines operation interfaces including a brightness adjustment interface and a strobe control interface; the barcode scanner abstraction unit defines operation interfaces including a barcode scanning interface and a QR code scanning and parsing interface; the driver abstraction unit defines operation interfaces including a motor control interface and a motion control card control interface.

8. A software device management method for heterogeneous integration of multiple devices, characterized in that, The device management method described herein is applied to the multi-device heterogeneous integrated software device management system as described in any one of claims 1 to 7, comprising: The device abstraction steps include designing a unified device base class and defining general device specifications. The device base class should include at least a device type enumeration, a connection status enumeration, and a unique device identifier, and define a unified operation interface, a data acquisition interface, an attribute read / write interface, and a configuration serialization interface. The device registration process involves establishing a mapping table between device signatures and device creation functions, registering the creation function of the target device class to the mapping table using a template registration method, and dynamically creating device object class instances based on the device signature. In the device loading step, the device manager reads the JSON format configuration file, iterates through the device configuration entries in the configuration file, extracts the device signature from each device configuration entry, calls the dynamic creation method in the device registration step to generate the corresponding device instance, calls the virtual method of loading configuration of the corresponding device instance to fill the configuration parameters into the device object, and adds the corresponding device instance to the device list. The device invocation steps involve obtaining a pointer to the target device through the query interface provided by the device manager, directly calling the unified operation interface defined in the device base class, and executing the overridden methods in the specific implementation class of the target device through polymorphism to complete the operation on the physical hardware device. The status monitoring steps involve periodically checking the connection status of each physical hardware device in the device list. For devices that are offline, an automatic reconnection operation is performed. When the device status changes, a status change signal is transmitted. When a device encounters an error, an error signal is transmitted.

9. The software device management method for heterogeneous integration of multiple devices according to claim 8, characterized in that, The device signature is the return value of the static method module signature provided for each device class, used to uniquely identify the physical hardware device type; the template registration method registers devices automatically through the static registration mechanism when the program starts.

10. The software device management method for heterogeneous integration of multiple devices according to claim 8, characterized in that, The automatic reconnection operation includes: when the device status changes from online to offline, the device manager starts a reconnection timer, calls the connection operation of the current physical hardware device at preset time intervals, and issues an alarm signal when the current physical hardware device returns to online status or reaches the maximum number of reconnections.

Images