Device scanning method, device and storage medium in visual inspection process

Abstract

The application discloses a device scanning method and device in a visual detection process, an apparatus and a storage medium, relates to the field of visual detection, and is applied to a target industrial inference device and comprises the following steps: determining a target index list based on the device running states of target cameras; determining a target device index based on a preset device index list, judging whether the target device index exists in the target index list, if the target device index exists, determining a next target device index based on the preset device index list, and jumping to the step of judging whether the target device index exists in the target index list; if the target device index does not exist, performing device query on a device node of the target device index pair to determine target device information of the device node; after completing the iteration of each device index in the preset device index list, updating a device list of the target industrial inference device to complete a safety scanning operation corresponding to the target industrial inference device. The application realizes the completion of device scanning without interfering with the running cameras.

Description

Technical Field

[0001] This invention relates to the field of visual inspection, and particularly to equipment scanning methods, apparatus, devices, and storage media in the visual inspection process. Background Technology

[0002] Industrial vision inspection systems typically require 24 / 7 continuous operation for real-time video inference from multiple cameras. To ensure system reliability, a watchdog module needs to periodically scan the device list to detect newly connected devices or restore faulty devices. However, traditional device scanning methods require turning on each camera for testing, which interrupts ongoing inference tasks, leading to video stream interruptions, loss of inference results, and severely impacting production continuity. Therefore, existing systems usually disable device scanning during inference, but this prevents faulty devices from automatically recovering, requiring manual intervention to restart the system, resulting in high maintenance costs. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a device scanning method, apparatus, device, and storage medium in the visual inspection process, which can complete device scanning without interfering with a running camera. The specific solution is as follows: In a first aspect, this application discloses a device scanning method in a visual inspection process, applied to a target industrial inference device, comprising: The device operating status of each target camera corresponding to the target industrial inference device is detected, and a target index list is determined based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera; A target device index is determined based on a preset device index list. It is then determined whether the target device index exists in the target index list. If the target device index exists in the target index list, the next target device index is determined based on the preset device index list, and the process jumps to the step of determining whether the target device index exists in the target index list. The preset device index list is used to store device indexes within a preset index range. If the target device index does not exist in the target index list, then a device query operation is performed on the device node corresponding to the target device index to determine the target device information corresponding to the device node; After traversing each device index in the preset device index list, the device list corresponding to the target industrial inference device is updated based on the target device information to complete the security scan operation corresponding to the target industrial inference device.

[0004] Optionally, the step of detecting the device operating status of each target camera corresponding to the target industrial inference device, and determining the target index list based on the device operating status of each target camera, includes: The camera manager of the target industrial inference device is used to obtain a set of running camera threads, and the device running status of the target camera corresponding to each camera thread in the set of running camera threads is determined to be running. The device indexes corresponding to each camera thread in the set of running camera threads are summarized to determine the target index list.

[0005] Optionally, after performing a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node, the method further includes: Based on the target device information, the USB port path, manufacturer identifier, product identifier, and serial number corresponding to the device node are determined, and the target interface number suffix of the USB port path is removed to determine the normalized port path corresponding to the device node; the normalized port path is used to characterize the physical port corresponding to the device node; The target hash is determined based on the normalized port path, the manufacturer identifier, the product identifier, and the serial number, and the target device identifier corresponding to the device node is determined based on the target hash. Accordingly, after updating the device list corresponding to the target industrial inference device based on the target device information, the process further includes: The camera to be recovered is determined based on the configuration information of the target industrial inference device; the camera to be recovered is the target camera that should be in operation as recorded in the configuration information and whose current operating state is not in operation. The target device identifier corresponding to the camera to be recovered is compared with each target device identifier recorded in the updated device list to obtain the corresponding target comparison result; If the target comparison result indicates that there is a target device identifier in the updated device list that is the same as the target device identifier corresponding to the camera to be recovered, then the device index of the device node corresponding to the target device identifier is determined based on the device list, and the target recovery task corresponding to the camera to be recovered is executed using the device index of the device node corresponding to the target device identifier.

[0006] Optionally, determining the target hash based on the normalized port path, the vendor identifier, the product identifier, and the serial number includes: If the serial number does not meet the preset availability conditions, then a first target hash is determined based on the normalized port path, the manufacturer identifier, and the product identifier; If the normalized port path does not meet the preset availability condition, then a second target hash is determined based on the manufacturer identifier, the product identifier, and the device node path corresponding to the device node. If the normalized port path, the manufacturer identifier, the product identifier, and the serial number do not meet the preset availability conditions, then a hash operation is performed on the device node path to determine the third target hash.

[0007] Optionally, the device scanning method in the visual inspection process further includes: If the target camera is detected to be a faulty camera, it is determined whether the faulty device index corresponding to the faulty camera has changed. If the faulty device index corresponding to the faulty camera has not changed, the camera thread corresponding to the faulty camera is terminated and the corresponding device resources are released. The target inference task corresponding to the faulty camera is restarted based on the faulty device index. The faulty device index is the device index of the device node corresponding to the faulty camera.

[0008] Optionally, the device scanning method in the visual inspection process further includes: If the faulty device index corresponding to the faulty camera changes, or the target inference task corresponding to the faulty camera fails to restart, a forced scan operation is performed on the faulty camera to obtain the corresponding forced scan result. Based on the target device identifier corresponding to the faulty camera and the forced scan result, the current faulty device index corresponding to the faulty camera is determined, so as to restart the target inference task corresponding to the faulty camera based on the current faulty device index corresponding to the faulty camera.

[0009] Optionally, detecting the device operating status of each target camera corresponding to the target industrial inference device includes: The device operating status of each target camera corresponding to the target industrial inference device is detected based on a preset scanning cycle. Accordingly, after updating the device list corresponding to the target industrial inference device based on the target device information to complete the security scanning operation corresponding to the target industrial inference device, the method further includes: The preset scanning cycle is dynamically adjusted based on the updated device list, so as to scan the device operating status of each target camera based on the adjusted preset scanning cycle.

[0010] Secondly, this application discloses a device scanning apparatus for a visual inspection process, applied to a target industrial inference device, comprising: The index list determination module is used to detect the device operating status of each target camera corresponding to the target industrial inference device, and determine the target index list based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera; An index determination module is used to determine a target device index based on a preset device index list, determine whether the target device index exists in the target index list, and if the target device index exists in the target index list, determine the next target device index based on the preset device index list, and jump to the step of determining whether the target device index exists in the target index list; the preset device index list is used to store the device indexes within a preset index range; The node query module is used to perform a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node if the target device index does not exist in the target index list; The list update module is used to update the device list corresponding to the target industrial inference device based on the target device information after completing the traversal of each device index in the preset device index list, so as to complete the security scanning operation corresponding to the target industrial inference device.

[0011] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the device scanning method in the aforementioned visual inspection process.

[0012] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned device scanning method in the visual inspection process.

[0013] In this application, during device scanning in the visual inspection process, the target industrial inference device detects the device operating status of each target camera corresponding to the target industrial inference device and determines a target index list based on the device operating status of each target camera. The target index list is used to store the device index corresponding to the target camera whose device operating status is running. The device index is a number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera. The target device index is determined based on the preset device index list. It is determined whether the target device index exists in the target index list. If the target device index exists in the target index list, the next target device index is determined based on the preset device index list, and the process jumps to the step of determining whether the target device index exists in the target index list. The preset device index list is used to store the device indexes within a preset index range. If the target device index does not exist in the target index list, a device query operation is performed on the device node corresponding to the target device index to determine the target device information corresponding to the device node. After completing the traversal of each device index in the preset device index list, the device list corresponding to the target industrial inference device is updated based on the target device information to complete the security scanning operation corresponding to the target industrial inference device. As can be seen, the device scanning method in the visual inspection process provided in this embodiment first determines the device index corresponding to the target camera that is running and summarizes it into a target index list before performing device scanning. During the traversal of the preset device index list, target device indexes that exist in the target index list are skipped directly without performing any device opening or device query operations on them, thereby avoiding device resource conflicts between the scanning process and the ongoing video acquisition and inference tasks, and achieving zero interference to the inference task. At the same time, for target device indexes that do not exist in the target index list, the device query operation is performed normally and the device list is updated based on the obtained target device information. This enables the target industrial inference device to continuously discover newly connected devices and track the recovery status of faulty devices during the operation of the inference task, achieving a balance between the continuity of device management and the stability of the inference task, and providing a reliable basis for the automatic recovery of faulty devices. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0015] Figure 1This is a flowchart of a device scanning method in a visual inspection process disclosed in this application; Figure 2 This is a schematic diagram of a device scanning method in a specific visual inspection process disclosed in this application; Figure 3 This is a schematic diagram of a specific process for obtaining the index of an operating device disclosed in this application; Figure 4 This is a schematic diagram of a specific equipment recovery process disclosed in this application; Figure 5 This is a schematic diagram of the structure of a scanning device in a visual inspection process disclosed in this application; Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In industrial vision inspection systems, continuous 24 / 7 operation is typically required for real-time video inference from multiple cameras. To ensure system reliability, a watchdog module needs to periodically scan the device list to detect newly connected devices or restore faulty devices. However, traditional device scanning methods require turning on each camera for testing, which interrupts the ongoing inference task, leading to video stream interruptions, loss of inference results, and severely impacting production continuity. Therefore, existing systems typically disable device scanning during inference, but this prevents faulty devices from automatically recovering, requiring manual intervention to restart the system, resulting in high maintenance costs. To address these technical issues, this application discloses a device scanning method in the vision inspection process that can complete device scanning without interfering with running cameras.

[0018] See Figure 1 As shown, this embodiment of the invention discloses a device scanning method in a visual inspection process, applied to a target industrial inference device, comprising: Step S11: Detect the device operating status of each target camera corresponding to the target industrial inference device, and determine a target index list based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera.

[0019] In this embodiment, the target industrial inference device is the computing device deployed in the industrial field to perform real-time video inference tasks. For example, it can be an edge computing device equipped with a neural network processing unit (NPU), specifically an edge inference device built on an embedded platform such as the RK3588. Of course, the target industrial inference device can also be other computing devices with video acquisition and inference capabilities; this embodiment does not limit this. The target camera is a video acquisition device connected to the target industrial inference device. For example, it can be a UVC (USB Video Class) camera connected via a USB (Universal Serial Bus) interface. The number of target cameras can be one or more; this embodiment also does not limit this.

[0020] In a Linux operating system environment, the V4L2 (Video for Linux 2) subsystem assigns a corresponding device node to each connected camera. The device node is represented as ` / dev / videoX`, where `X` is the number assigned to that device node by the operating system, which is the device index described in this embodiment. It should be noted that the device index is dynamically assigned by the operating system kernel during device enumeration. Its specific value depends on the order in which the devices are detected by the kernel. Therefore, after a system restart or device re-enumeration, cameras on the same physical port may obtain different device indices. This is one of the important problems faced by device management in industrial vision inspection systems. Furthermore, in vision inspection systems built based on computer vision libraries such as OpenCV, video acquisition is typically performed by calling the `cv2.VideoCapture(device_index)` interface to open the camera corresponding to the device index. Therefore, the device index can also be called the OpenCV device index.

[0021] Furthermore, the target industrial inference device can be equipped with functional modules such as a camera manager, a watchdog module, and an inference engine. The camera manager manages the camera threads (CameraThread) corresponding to each target camera. Each camera thread is responsible for video acquisition and inference scheduling for one target camera. When a camera thread is created, its constructor receives a device index parameter (device_index) and stores this device index as a member variable of the thread object (e.g., self.device_index). The watchdog module monitors the operating status of each target camera and executes the device scanning method provided in the visual inspection process of this application embodiment. The inference engine performs inference calculations on video frames based on the NPU, for example, it can implement parallel inference of multiple video streams based on an RKNN model pool. Figure 2 As shown, the device scanning method in the visual inspection process disclosed in this embodiment is applied to the target industrial inference device, and can be executed by the watchdog module in the target industrial inference device.

[0022] In this embodiment, as Figure 3 As shown, before the device scan begins, the target cameras currently in operation (i.e., performing video acquisition and inference tasks) are identified, and the device indices corresponding to these target cameras are summarized into a target index list. This allows for skipping device indices in the target index list during subsequent scanning, preventing the scanning operation from touching the running target cameras from the outset. In one specific implementation, the device operating status of each target camera corresponding to the target industrial inference device is detected, and the target index list is determined based on the device operating status of each target camera. This includes: obtaining a set of running camera threads using the camera manager of the target industrial inference device, determining the device operating status of the target cameras corresponding to each camera thread in the set of running camera threads as "running," and summarizing the device indices corresponding to each camera thread in the set of running camera threads to determine the target index list. Specifically, the camera manager maintains all currently active camera threads, which together constitute a set of running camera threads. Since each camera thread corresponds to a target camera performing a video acquisition and inference task, the device operating status of the target cameras corresponding to each camera thread in the set of running camera threads can be determined as "running." Based on this, as... Figure 3As shown, the watchdog module iterates through all active camera thread objects in the set of running camera threads, reads the device index member variable (i.e., camera_thread.device_index) stored in each camera thread object, and can extract the device index corresponding to each running target camera. By summarizing the extracted device indexes, a target index list can be obtained.

[0023] For example, in a specific application scenario, the target industrial inference device currently has three target cameras performing inference tasks. The watchdog module retrieves the three currently running camera thread objects from the camera manager and extracts their respective device indices: the target camera with the target device identifier "88cfcf82c6b0" has a device node number of 0 in the current system, so the device index of this camera thread is 0; the target camera with the target device identifier "def456ghi789" has a device node number of 2 in the current system, so the device index of this camera thread is 2; the target camera with the target device identifier "abc123def456" has a device node number of 5 in the current system, so the device index of this camera thread is 5; thus, the target index list running_camera_indices=[0, 2, 5] is obtained. The target device identifier (Device ID) is a 12-bit hexadecimal string generated by hashing information such as the device's USB port path, manufacturer identifier, product identifier, and serial number, used to stably identify the same physical camera across restarts and enumerations.

[0024] It's important to note that the device index stored in the camera thread originates from the device scan during system startup: At system startup, by traversing each device node within a preset index range (e.g., / dev / video0 to / dev / video19), a device query command (e.g., udevadm info) is called for each device node to obtain its unique identifier. Simultaneously, the / dev / videoX number (i.e., the device index) corresponding to that device node in the current system is recorded. When creating a camera thread for the target camera, this number is passed as the device_index parameter to the camera thread's constructor, and the camera thread stores it as a member variable. It's also worth noting that the configuration information of the target industrial inference device stores the target camera's target device identifier, not its device index. At system startup, the dynamic mapping from the target device identifier to the device index is completed by looking up the current device index corresponding to the target device identifier without opening the device (e.g., using the get_device_index_without_opening(unique_id) method). This design is adopted because device indexes are dynamically allocated by the operating system kernel during device enumeration. After a system restart, cameras on the same physical port may obtain different device indexes, while the target device identifier has stability across restarts and enumerations. Correspondingly, the target index list obtained by the watchdog module reflects the device index values ​​actually allocated by the operating system kernel in the current running session.

[0025] Step S12: Determine the target device index based on the preset device index list, determine whether the target device index exists in the target index list, and if the target device index exists in the target index list, determine the next target device index based on the preset device index list, and jump to the step of determining whether the target device index exists in the target index list; the preset device index list is used to store the device indexes within the preset index range.

[0026] In this embodiment, the preset device index list is used to limit the range of device indexes covered by the device scan. The preset index range can be set according to the number of device nodes that the target industrial inference device may connect to. For example, the preset index range can be set to 0 to 19, and the corresponding preset device index list is [0, 1, 2, ..., 19], corresponding to device nodes / dev / video0 to / dev / video19. Of course, the preset index range can also be set to other value ranges according to actual business needs, and this embodiment does not limit this.

[0027] During device scanning, each device index in the preset device index list is sequentially used as a target device index, and it is determined whether the current target device index exists in the target index list: If the target device index exists in the target index list, it means that the target camera corresponding to the target device index is currently running. In this case, the target device index is skipped, that is, no device opening or device query operation is performed on the device node corresponding to the target device index, and the next target device index is determined based on the preset device index list, and the above determination step is repeated; if the target device index does not exist in the target index list, the next step is performed. The above skipping process can be implemented by the following pseudocode: for device_index in range(20): if skip_running_cameras and device_index in running_camera_indices: continue # Skip the index of running camera devices else: device_info = query_device_info(device_index) Here, `skip_running_cameras` is the inference status check flag, a global boolean variable updated by the camera thread in the camera manager when starting or stopping the inference task. When any target camera is performing an inference task, the inference status check flag is set to true (i.e., `skip_running_cameras=True`), and the scanning process executes the aforementioned skip logic. `running_camera_indices` is the aforementioned target index list. Taking the aforementioned application scenario as an example, when the scanning process traverses to device indices 0, 2, and 5, it is determined that they exist in the target index list, and `continue` is executed directly to skip them without generating any operations for the corresponding device nodes. Only the device nodes corresponding to device indices 1, 3, 4, and 6 to 19 are subjected to subsequent device query operations.

[0028] Understandably, since no device opening or device query operations are performed on the device index corresponding to the target camera in operation, the video stream of the running target camera will not be interfered with in any way. This fundamentally avoids the problem of inference frame loss and video stream interruption caused by opening device detection in the traditional full scan method. This is also the core of achieving zero interference in the inference task in this embodiment.

[0029] Step S13: If the target device index does not exist in the target index list, then perform a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node.

[0030] In this embodiment, for target device indexes that do not exist in the target index list, i.e., device indexes that are not currently running, a device query operation is performed on their corresponding device nodes to determine the target device information corresponding to the device node. As a feasible implementation, the device query operation can be implemented using system command queries without opening the device file; for example, the udevadm command can be called to query device nodes. udevadm info --query=all --name= / dev / video1 The `udevadm` command relies on the operating system's udev device management mechanism, directly reading device information from the device attribute database maintained by the operating system kernel. The entire query process does not require opening the ` / dev / videoX` device file, thus avoiding resource consumption. Even if the device node corresponding to the target device index is actually being used by another process, the query operation will not conflict with its resources. In this way, device information can be obtained while minimizing the resource consumption of the scanning operation.

[0031] In this embodiment, the target device information may include multiple device attribute fields corresponding to the device node, such as the ID_PATH field representing the USB port path, the ID_VENDOR_ID field representing the manufacturer identifier, the ID_MODEL_ID field representing the product identifier, and the ID_SERIAL_SHORT field representing the serial number. Considering that in actual industrial scenarios, the device index is dynamically allocated by the operating system kernel during device enumeration, system restarts, device re-plugging or unplugging, or USB re-enumeration may all cause the device index of the same physical camera to change. Therefore, the device index alone cannot reliably identify the target camera across restarts and enumerations. After performing a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node, this embodiment further includes: determining the USB port path, manufacturer identifier, product identifier, and serial number corresponding to the device node based on the target device information, and removing the target interface number suffix from the USB port path to determine the normalized port path corresponding to the device node; the normalized port path is used to represent the physical port corresponding to the device node; and a target hash is determined based on the normalized port path, manufacturer identifier, product identifier, and serial number, so as to determine the target device identifier corresponding to the device node based on the target hash.

[0032] Specifically, the target device information returned by the udevadm command can be parsed using a preset parsing method (such as the _parse_udevadm_output method) to extract the following key attribute fields: The first item is the ID_PATH field, which is the USB port path. Its format is, for example, pci-0000:00:14.0-usb-0:10:1.0, which can be extracted from the output of the udevadm command using the regular expression "E:ID_PATH=(.+)". The USB port path is generated by the Linux kernel's USB subsystem based on the physical port the device is actually plugged into. As long as the device is plugged into the same USB port, this field will not change due to system restarts, thus exhibiting good stability. The second field is the ID_VENDOR_ID field, which is the manufacturer identifier, a four-byte hexadecimal USB manufacturer ID, such as 32e4, which can be extracted from the output of the udevadm command using the regular expression "E: ID_VENDOR_ID=(.+)". The third field is the ID_MODEL_ID field, which is the product identifier, a four-byte hexadecimal USB product ID, such as 0410. The fourth field is the ID_SERIAL_SHORT field, which is the serial number string provided by the USB device. It should be noted that some low-end UVC cameras have a serial number of "0" or an empty string, in which case the serial number field is unavailable.

[0033] Furthermore, the USB port path is normalized by removing the target interface number suffix from the end of the USB port path to obtain a normalized port path. For example, the suffix (e.g., ":1.0") can be removed using the regular expression `re.sub(r':\d+\.\d+$', '', id_path)`, thus normalizing `pci-0000:00:14.0-usb-0:10:1.0` to `pci-0000:00:14.0-usb-0:10`. The normalized port path is used to represent the physical port corresponding to the device node, i.e., a stable physical port identifier.

[0034] It is understandable that the reason for normalizing the USB port path in this embodiment is that a camera on the same physical USB port may generate multiple / dev / videoN device nodes in the system due to having multiple functional interfaces (for example, UVC devices usually have both a video capture interface and a metadata interface). The only difference in the original USB port path of these device nodes is the suffix of the interface number at the end (such as: XX). After removing the target interface number suffix through normalization, all interface nodes belonging to the same physical camera are mapped to the same normalized port path, and thus to the same target device identifier, avoiding the ambiguity of the same physical camera being identified as multiple different devices.

[0035] In this embodiment, after obtaining the normalized port path, the target hash is determined based on the normalized port path, manufacturer identifier, product identifier, and serial number, and the target device identifier corresponding to the device node is determined based on the target hash. For example, the normalized port path, manufacturer identifier, product identifier, and serial number can be concatenated, an MD5 hash operation can be performed on the concatenated result, and the first 12 hexadecimal characters of the hash result can be taken as the target device identifier; for example, concatenating pci-0000:00:14.0-usb-0:10, 32e4, 0410 with SN12345, performing an MD5 hash operation, and taking the first 12 bits can obtain the target device identifier "88cfcf82c6b0".

[0036] Furthermore, considering that the attribute fields of the actual device may be missing or invalid, this embodiment also provides a multi-level fallback target hash determination strategy to ensure that a definite target device identifier can be generated for the device node under any circumstances. As a feasible implementation, it includes: if the serial number does not meet the preset availability conditions, then a first target hash is determined based on the normalized port path, manufacturer identifier, and product identifier; if the normalized port path does not meet the preset availability conditions, then a second target hash is determined based on the manufacturer identifier, product identifier, and the device node path corresponding to the device node; if the normalized port path, manufacturer identifier, product identifier, and serial number all do not meet the preset availability conditions, then a hash operation is performed on the device node path to determine a third target hash. The preset availability conditions are used to determine whether the corresponding field is available. For example, the preset availability conditions can be set to the corresponding field being non-empty and not an invalid value; taking the serial number as an example, the serial number meeting the preset availability conditions means that the serial number is not an empty string and is not "0". The device node path is the path of the device node in the operating system, such as / dev / video2.

[0037] In one specific implementation, the determination of the target hash follows a multi-level backoff priority order as follows: First priority (complete features): When the normalized port path, manufacturer identifier, product identifier, and serial number all meet the preset availability conditions, the normalized port path, manufacturer identifier, product identifier, and serial number are used together as hash input to determine the target hash; for example, pci-0000:00:14.0-usb-0:10, 32e4, 0410 are concatenated with SN12345, an MD5 hash operation is performed, and the first 12 bits are taken to obtain the target device identifier "88cfcf82c6b0"; Second priority (when serial number is missing): When the serial number does not meet the preset availability conditions (such as the serial number of some low-end UVC cameras being "0" or an empty string), the first target hash is determined based on the normalized port path, manufacturer identifier, and product identifier; for example, after concatenating pci-0000:00:14.0-usb-0:10, 32e4 and 0410, an MD5 hash operation is performed and the first 12 bits are taken to obtain the target device identifier "a3b2c9d8e1f0"; Third priority (when USB topology information is unavailable): When the normalized port path does not meet the preset availability conditions, the second target hash is determined based on the manufacturer identifier, product identifier and device node path; for example, 32e4, 0410 and / dev / video2 are concatenated and then an MD5 hash operation is performed and the first 12 bits are taken as the target device identifier; Fourth priority (final backup): When the normalized port path, vendor identifier, product identifier, and serial number do not meet the preset availability conditions, perform a hash operation on the device node path itself to determine the third target hash; for example, perform an MD5 hash operation on / dev / video2 and take the first 12 bits as the target device identifier.

[0038] Through the above multi-level fallback strategy, in this embodiment, regardless of the actual availability of the device attribute fields, a definite and as stable as possible target device identifier can be generated for the device node, ensuring the robustness and full coverage of the device identifier mechanism.

[0039] Step S14: After traversing each device index in the preset device index list, update the device list corresponding to the target industrial inference device based on the target device information to complete the security scan operation corresponding to the target industrial inference device.

[0040] In this embodiment, after traversing all device indices in the preset device index list, the target device information obtained from each device query operation is summarized, and the device list corresponding to the target industrial inference device is updated based on the target device information. The device list records the currently available device nodes of the target industrial inference device and their corresponding device attributes. For example, the device list can record the mapping relationship between the target device identifier and the device index corresponding to each device node (i.e., a mapping in the form of {Device ID: current system index}). Since the entire scanning process skips the device index corresponding to the target camera that is currently running, and only performs device query operations without opening device files on non-running device nodes, this scanning operation achieves zero interference with the ongoing inference task; therefore, it is called a security scanning operation.

[0041] Based on the generation of the target device identifier, such as Figure 4 As shown, this embodiment also provides a device recovery method based on security scan results. As a feasible implementation, after updating the device list corresponding to the target industrial inference device based on the target device information, the method further includes: determining the camera to be recovered based on the configuration information of the target industrial inference device; the camera to be recovered is a target camera that should be in operation as recorded in the configuration information and whose current device operation state is not in operation; comparing the target device identifier corresponding to the camera to be recovered with each target device identifier recorded in the updated device list to obtain the corresponding target comparison result; if the target comparison result indicates that there is a target device identifier in the updated device list that is the same as the target device identifier corresponding to the camera to be recovered, then determining the device index of the device node corresponding to the target device identifier based on the device list, and using the device index of the device node corresponding to the target device identifier to execute the target recovery task corresponding to the camera to be recovered.

[0042] Specifically, the configuration information of the target industrial inference equipment records each target camera and its configuration parameters using the target device identifier as the key. That is, the configuration information stores the mapping relationship between the target device identifier and configuration parameters (such as exposure, brightness, frame rate, etc.). After the security scan operation is completed, the updated device list records the mapping relationship between each target device identifier and its current device index. The watchdog module identifies target cameras that should be in operation according to the configuration information but are currently not in operation as cameras to be recovered. It then compares the target device identifier corresponding to each camera to be recovered with the target device identifiers recorded in the updated device list one by one to obtain the target comparison result.

[0043] It is important to note that when confirming whether a device can be recovered, the comparison is made using the target device ID, not the device index number. This is because after a device malfunctions and is re-enumerated, its device index may have changed; the original index may be occupied by another device or may be invalid. Only the target device ID can stably identify the same physical camera across enumerations. If the target comparison result indicates that the updated device list contains a target device ID identical to the one corresponding to the camera to be recovered, it means the camera has been brought back online. In this case, the current device index corresponding to that target device ID (i.e., the new device index just obtained during this security scan) is read from the device list, and the target recovery task for the camera to be recovered is executed using this device index. For example, a video capture object is created using this device index as a parameter (i.e., calling cv2.VideoCapture(new_index)), the configuration parameters corresponding to the target device ID are loaded, a new camera thread is created and started, thereby automatically recovering the inference task corresponding to the camera to be recovered. If the target comparison results indicate that there is no identical target device identifier in the updated device list, it means that the device does not currently exist in the system at all, and a manual alarm can be triggered to notify the operation and maintenance personnel to intervene.

[0044] It should also be noted that the parameter passed to the video capture object is the new device index obtained from the security scan operation, not the old device index recorded in the configuration information. This is because the original device index may have been occupied by another device or become invalid after a failure. The process of finding the current device index from the target device identifier can be implemented using the get_device_index(unique_id) method, which performs a device scan, finds the device index corresponding to the target device identifier in the scan results, and returns it.

[0045] Furthermore, considering that target cameras inevitably experience various malfunctions during long-term continuous operation of industrial inference equipment, this embodiment also provides a differentiated recovery process for malfunctioning cameras to further enhance the system's self-healing capability. Statistical analysis of six consecutive months of operational data (a total of 47 malfunction events) from a real production environment reveals that target camera malfunctions in industrial vision inspection systems can be categorized into two main types: Type A faults are software-layer faults. After such faults occur, the device node number (i.e., the faulty device index) remains unchanged. This type of fault occurred 32 times within the statistical period, accounting for 68% of all faults. Specifically, it includes the following four types: Type A1 is a frame read timeout fault, which occurred 16 times within the statistical period. This manifests as the video capture object's frame read interface (e.g., cv2.VideoCapture.read()) continuously returning null values ​​(None) for more than a preset duration (e.g., 10 seconds). The cause is usually instantaneous USB bandwidth contention leading to frame drops by the UVC driver. Type A2 is a decoding failure fault, which occurred 8 times within the statistical period. This manifests as the frame read interface returning frame data, but the frame is empty. Or it may be corrupted (e.g., an empty byte array), usually caused by an MJPEG decoder malfunction; Class A3 is an inference thread crash fault, which occurred 5 times in the statistical period. It manifests as the inference worker thread throwing a runtime error due to an abnormal inference handle (e.g., an abnormal RKNN handle). It is usually caused by an abnormal computing unit state after the NPU has been running under high load for a long time; Class A4 is a handle failure fault, which occurred 3 times in the statistical period. It manifests as the video acquisition object's status query interface (isOpened()) returning a true value, but the actual device is no longer usable. It is usually caused by an inconsistency in the internal state of the UVC driver.

[0046] Type B faults are hardware-level faults. These faults may cause changes in device node numbers. They occurred 15 times during the statistical period, accounting for 32% of all faults. Specifically, they include the following three types: Type B1 is a power supply jitter fault, occurring 8 times during the statistical period. This manifests as a USB camera disconnecting due to a momentary power shortage at the USB port and then automatically reconnecting. The operating system kernel treats this as a device removed and then reinserted, potentially assigning it a new device node number. Type B2 is a firmware hang fault, occurring 5 times during the statistical period. This manifests as the USB camera's firmware becoming unresponsive. The kernel's USB subsystem kicks the device out and re-enumerates it after a timeout. This USB re-enumeration inevitably triggers a change in device node number. Type B3 is a reconnection fault after being kicked off the bus, occurring 2 times during the statistical period. This manifests as the USB controller briefly suspending under abnormal conditions such as high temperature, causing all devices to disconnect and then recover.

[0047] In response to the two types of faults mentioned above, this embodiment determines whether the fault device index corresponding to the fault camera has changed when the target camera is detected to be a fault camera. If the fault device index corresponding to the fault camera has not changed, the camera thread corresponding to the fault camera is terminated and the corresponding device resources are released. The target inference task corresponding to the fault camera is restarted based on the fault device index. The fault device index is the device index of the device node corresponding to the fault camera.

[0048] This implementation primarily addresses the aforementioned Type A software layer faults, specifically scenarios where the faulty device index remains unchanged. Specifically, the watchdog module can monitor whether a target camera is faulty by performing health checks on each target camera (e.g., checking heart rate, frame rate, etc. using the `_check_camera_health` method). Upon detecting a faulty camera, it determines whether the corresponding faulty device index has changed. If not, it executes a camera-level recovery process (e.g., using the `_restart_camera` method), which may include the following steps: Step 1: Terminate the camera thread corresponding to the faulty camera and release the corresponding device resources, that is, call the stop interface of the camera thread (camera_thread.stop()), wait for the video acquisition thread to end (join acquisition thread), wait for the inference worker thread to end (join inference worker thread), wait for the save worker thread to end (join save worker thread), and call the release interface of the video acquisition object (cap.release()) to release the device handle; Step 2: Wait for the preset release time (e.g., wait 1 second, i.e., time.sleep(1)) to ensure that the V4L2 device is completely released by the operating system; Step 3: Since the faulty device index has not changed, the video capture object is recreated directly using the original faulty device index, i.e., cv2.VideoCapture(device_index) is called; Step 4: Reset the fault counter corresponding to the faulty camera, that is, set camera_state.failure_count to 0; Step 5: Create and start a new camera thread (camera_thread.start()) to resume the target inference task corresponding to the faulty camera.

[0049] In actual fault handling scenarios, the total time for the above-mentioned Type A fault recovery process is about 3 to 5 seconds. The recovery speed is fast and the impact range is small. The entire recovery process only involves the faulty camera itself, and the other operating target cameras are completely unaffected.

[0050] Furthermore, as a feasible implementation method, the device scanning method in the visual inspection process provided in this application embodiment may further include: if the fault device index corresponding to the faulty camera changes, or the target inference task corresponding to the faulty camera fails to restart, then a forced scanning operation is performed on the faulty camera to obtain the corresponding forced scanning result, and the fault device index currently corresponding to the faulty camera is determined based on the target device identifier corresponding to the faulty camera and the forced scanning result, so as to restart the target inference task corresponding to the faulty camera based on the fault device index currently corresponding to the faulty camera.

[0051] Furthermore, as a feasible implementation method, the device scanning method in the visual detection process provided in this embodiment may further include: if the fault device index corresponding to the faulty camera changes, or the target inference task corresponding to the faulty camera fails to restart, then a forced scanning operation is performed on the faulty camera to obtain the corresponding forced scanning result, and the fault device index currently corresponding to the faulty camera is determined based on the target device identifier corresponding to the faulty camera and the forced scanning result, so as to restart the target inference task corresponding to the faulty camera based on the fault device index currently corresponding to the faulty camera.

[0052] This implementation primarily addresses the aforementioned Type B hardware layer faults, specifically scenarios where the faulty device index may change, and Type A fault recovery processes that fail to restart. The forced scan operation refers to a full device scan that skips constraints such as scan cycle limits, scan result caching limits, and running index limits. This can be triggered, for example, by setting the `force_rescan=True` parameter (i.e., calling `camera_device_manager.scan_cameras(force_rescan=True)`). Specifically, the global recovery process for Type B faults (e.g., implemented via the `_trigger_global_restart` method) may include the following steps: Step 1: Send a stop signal to all camera threads; Step 2: Wait for all camera threads to complete and exit; Step 3: Clean up all NPU inference handles of the inference engine and recreate them, for example by calling inference_engine.reload_rknn_pool(), to eliminate any possible inference handle state anomalies; Step 4: Perform a forced scan operation, that is, call camera_device_manager.scan_cameras(force_rescan=True) to force a full device scan and obtain the forced scan results; Step 5: Search for the target device identifier corresponding to the faulty camera in the forced scan results, that is, compare the target device identifier recorded in the configuration information with the target device identifiers recorded in the forced scan results; Step 6: If the same target device identifier is found, obtain the new device node number corresponding to the target device identifier in the current system, that is, the faulty device index corresponding to the faulty camera; it should be emphasized that the target device identifier is compared here, not the device index number. Step 7: Create a video capture object using the newly acquired faulty device index (instead of the old device index recorded in the configuration information), i.e., call cv2.VideoCapture(new_index); Step 8: Create a new camera thread and pass the new faulty device index as the device_index parameter; Step 9: Start a new camera thread and resume the target inference task corresponding to the faulty camera.

[0053] In actual fault handling scenarios, the total time for the above-mentioned Type B fault recovery process is approximately 20 to 28 seconds. If the same target device identifier is not found in the forced scan results based on the target device identifier corresponding to the faulty camera, it means that the device does not exist in the system at all (e.g., the USB physical port is damaged, causing the device to be unable to be enumerated). In this case, retries can be stopped after the maximum number of restarts is reached, and a manual alarm (such as an SMS alarm) can be triggered to notify the operation and maintenance personnel to intervene.

[0054] To facilitate understanding, three typical fault recovery cases during actual production environment operation are given below.

[0055] Case 1: The target camera with the target device identifier "a3b2c9d8e1f0" (original device index 3) triggered a Class B1 fault due to power supply jitter. After USB re-enumeration, the operating system kernel allocated device index 7 for it (the original device index 3 was occupied by other devices). The watchdog module performed a global recovery process: After performing a forced scan operation, the forced scan results found that the target device identifier "a3b2c9d8e1f0" currently corresponds to the device node / dev / video7. Then, a video capture object (i.e., VideoCapture(7)) was created using the new device index 7. The recovery was successful, and the total recovery time was 22 seconds. The system automatically resumed operation without manual restart.

[0056] Case 2: A target camera with the device identifier "88cfcf82c6b0" (original device index 0) triggered a Class B2 fault due to firmware hang. After the kernel removed the device and re-enumerated it, the new device node number was still 0 (the original number was not occupied by other devices). The watchdog module determined that the faulty device index had not changed and executed the camera-level recovery process (i.e., the _restart_camera method): directly reconstructing the video capture object using the original device index 0, with a total recovery time of 4 seconds.

[0057] Case 3: A target camera with the target device identifier "abc123def456" malfunctioned due to a damaged USB physical port (oxidation of the gold fingers). The watchdog module attempted automatic recovery three times, all of which failed (after each forced scan, the device could not be found at the USB port path corresponding to the target device identifier). After reaching the maximum number of restarts, it stopped retrying and triggered an SMS alarm to notify maintenance personnel to replace the hardware. This case is one of the four failures that failed to automatically recover out of 47 failure events within the statistical period. This type of failure belongs to the category of hardware physical damage failures that require manual intervention.

[0058] In summary, the comparison and recovery logic during the fault recovery process (e.g., implemented through the `_check_camera_health` and `_handle_camera_failure` methods) can be summarized in the following five points: First, the configuration information stores the mapping from the target device identifier to configuration parameters (such as exposure, brightness, frame rate, etc.); second, after a fault occurs, a scan is performed, and the scan result is a mapping from the target device identifier to the current system device index; third, the recovery judgment is based on whether the target device identifier in the configuration information appears in the target device identifier list of the scan results; fourth, if it appears, the current device index corresponding to the target device identifier is read (i.e., the newly obtained device index is scanned), and a video capture object is created using this new device index; fifth, if it does not appear, it indicates that the device is completely non-existent, triggering a manual alarm.

[0059] To balance the timeliness of device discovery with system performance overhead, this embodiment also provides a dynamic adjustment mechanism for the scanning cycle. As a feasible implementation, the process of detecting the device operating status of each target camera corresponding to the target industrial inference device may specifically include: the watchdog module detecting the device operating status of each target camera corresponding to the target industrial inference device based on a preset scanning cycle; correspondingly, after updating the device list corresponding to the target industrial inference device based on the target device information to complete the security scanning operation corresponding to the target industrial inference device, it may further include: dynamically adjusting the preset scanning cycle based on the updated device list, and scanning the device operating status of each target camera based on the adjusted preset scanning cycle.

[0060] Specifically, the watchdog module can periodically perform the aforementioned security scan operation within a monitoring loop (such as the `_monitor_loop` method) according to a preset scan cycle. For example, the preset scan cycle can be set to 120 seconds, meaning a security scan is performed every 120 seconds to balance the timeliness of device discovery with system performance overhead. Furthermore, the preset scan cycle can be dynamically and adaptively adjusted based on system status: during normal system operation, a longer scan cycle (e.g., 120 seconds) is used to reduce system overhead; when an anomaly is detected based on the updated device list (e.g., a device is missing from the device list, or there are cameras awaiting recovery), the scan cycle is shortened for rapid retry, thereby accelerating the discovery and recovery of faulty devices.

[0061] Furthermore, embodiments of this application may also include one or more combinations of the following enhancement mechanisms: First, the scan failure retry mechanism: add the device index that fails in a single scan to the retry queue. After a preset retry time (such as 10 seconds), only the device index in the retry queue is scanned again, avoiding the waste of overhead caused by repeatedly scanning all device indexes. Second, the scan result caching mechanism: the results of successful scans are cached for a preset cache duration (e.g., 60 seconds). When a duplicate scan occurs within the cache validity period, the cache is directly hit and the cached scan result is returned, thereby reducing the overhead of udevadm system calls. Third, a forced scan coverage mechanism: a forced scan interface (such as the API endpoint / api / system / rescan) is provided. Through this interface, a forced scan can be triggered that skips all of the above restrictions (including skipping restrictions on running indexes, scan cycle restrictions, scan result caching restrictions, etc.), which is used for manual maintenance scenarios.

[0062] To avoid device resource conflicts caused by concurrent scanning by multiple modules, this embodiment can also have a watchdog module exclusively manage device scanning permissions, which can be achieved through the following three-layer mechanism: The first layer is a global mutex mechanism: the watchdog module maintains a scan mutex object (such as a _scan_lock object of type threading.Lock), acquires the scan mutex before performing a device scan, and releases the scan mutex after the scan is completed; other modules (such as the scan_cameras method of the camera manager) control the execution of device scans through the scan permission parameter (allow_scan), and skip the scan directly when allow_scan=False. This is the finest-grained concurrency control. The second layer is the inference state check flag mechanism: before performing a scan, a global Boolean variable (i.e., the inference flag) is checked. If any target camera is in inference, a skip flag is set (i.e., skip_running_cameras=True), skipping the device indices corresponding to the running target cameras (such as device indices 0, 2, and 5), and only scanning the device indices that are not in a running state (such as device indices 1, 3, 4, and 6 to 19). This flag is updated by the camera thread in the camera manager when starting or stopping the inference task. The third layer is an automatic scanning mechanism during idle periods: the watchdog module checks the inference status of all target cameras periodically (e.g., every 120 seconds) during the monitoring loop. Only when the pause flag (detected_pause) is true (i.e., all target cameras have paused inference or the system is in an idle state) will a full scan be performed without skipping any device indexes. This ensures that the most comprehensive device discovery operation is only performed within the safe window.

[0063] Furthermore, in this embodiment, the watchdog module is the only module in the target industrial inference device that performs device scanning. Other modules (such as the camera manager, inference engine, WebSocket manager, etc.) obtain the scan results indirectly through the interface provided by the watchdog module, without initiating their own scans, thus fundamentally avoiding conflicts caused by concurrent scanning by multiple modules.

[0064] To facilitate overall understanding, the device scanning method in the visual inspection process provided in this embodiment will be described below using a complete implementation scenario. The scenario is as follows: The target industrial inference device is running an inference task with three target cameras (device indices 0, 2, and 5). At this time, the target camera corresponding to device index 1 malfunctions and stops. The watchdog module needs to refresh the device list through a security scan operation to discover the recovery status of the malfunctioning device and automatically restore the device.

[0065] Step 1: Obtain the running device indexes: The watchdog module obtains the three currently running camera thread objects from the camera manager, extracts their corresponding device indices, and summarizes them to obtain the target index list running_camera_indices=[0, 2, 5]; Step 2, perform a security scan: traverse all device indices (0 to 19) in the preset device index list, skipping device indices 0, 2, and 5 (without performing any device opening or query operations), and performing normal device query operations on device indices 1, 3, 4, and 6 to 19; Step 3, Device Information Query (without opening the device): For a device index that is not running (such as device index 1), call the command udevadm info --query=all --name= / dev / video1 to query the device information, parse the target device identifier, and update the device list; Step 4, Faulty Device Recovery: After the security scan is completed, the camera to be recovered is identified based on the configuration information. Its target device identifier is compared with the target device identifiers in the updated device list. If a match is found, the current device index is determined based on the device list, and the target recovery task corresponding to the camera to be recovered is automatically started.

[0066] The final result is that the fault of the target camera corresponding to device index 1 is automatically recovered within 30 seconds, and the inference tasks corresponding to device indices 0, 2, and 5 are completely unaffected throughout the scanning and recovery process, achieving zero-downtime recovery of the system.

[0067] Furthermore, to verify the effectiveness of the device scanning method in the visual inspection process provided in this embodiment, the method was continuously tested in a real production environment for 6 months. The statistical data is as follows: there were a total of 47 failure events during the statistical period, including 32 Class A software layer failures and 15 Class B hardware layer failures; 43 failures were successfully recovered automatically, with an automatic recovery success rate of 43 / 47=91.5%; there were 4 failures that required manual intervention for recovery, all of which were due to physical damage to the USB hardware port and the need to replace the camera hardware; the average time from failure occurrence to automatic recovery was 28 seconds.

[0068] Furthermore, ablation comparison experiments were conducted on three device scanning strategies under the same fault injection conditions. The results are as follows: When using the no-scan strategy (i.e., the traditional strategy of prohibiting device scanning during inference operation), the automatic fault recovery rate was 0%, and the proportion of faulty devices permanently offline was 100%; when using the full-scan strategy (i.e., the traditional optimized strategy of performing a full scan to enable device detection during inference operation), the fault recovery rate was 72.3%, but it caused a 38% drop in the average frame rate of the running camera, and in actual deployment, it usually reverted to the no-scan strategy due to excessive interference; when using the security scanning strategy provided in this application, the fault recovery rate was 91.5%, and the frame rate of the running camera dropped to 0%. It can be seen that compared with the traditional watchdog scheme that prohibits scanning during inference operation, resulting in a fault recovery rate of 0%, and the full-scan scheme with a fault recovery rate of 72.3% but seriously interferes with the running camera, this application achieves the highest fault recovery rate without interfering with the inference task at all.

[0069] As can be seen, the device scanning method in the visual inspection process provided in this embodiment has at least the following beneficial effects: First, zero interference: device scanning is performed during the inference task without affecting the ongoing video acquisition and inference tasks, ensuring the continuity of the production system; Second, automatic fault recovery: the watchdog module can detect and recover faulty devices during the inference task, increasing the automatic fault recovery rate from 0% in traditional methods to over 90% (91.5% in actual testing), significantly reducing the need for manual intervention; Third, timely device discovery: newly connected devices can be detected and configured in a timely manner during the inference task without stopping the inference task, improving system flexibility; Fourth, minimized resource consumption: only non-running devices are queried, and system commands are used for querying instead of opening device files, resulting in minimal resource consumption; Fifth, improved system reliability: parallel execution of device management and inference tasks is achieved, significantly improving system availability, and reducing the single fault recovery time from approximately 15 minutes to less than 30 seconds; Sixth, reduced operation and maintenance costs: the need for manual intervention is reduced by approximately 80%, lowering operation and maintenance costs and improving production efficiency. The method provided in this embodiment can be widely applied to visual inspection systems that require continuous operation 24 / 7, such as intelligent manufacturing, security monitoring, and autonomous driving.

[0070] See Figure 5 As shown, this application discloses a device scanning apparatus in a visual inspection process, applied to a target industrial inference device, comprising: The index list determination module 11 is used to detect the device operating status of each target camera corresponding to the target industrial inference device, and determine the target index list based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera; The index determination module 12 is used to determine a target device index based on a preset device index list, determine whether the target device index exists in the target index list, and if the target device index exists in the target index list, determine the next target device index based on the preset device index list, and jump to the step of determining whether the target device index exists in the target index list; the preset device index list is used to store the device indexes within a preset index range; The node query module 13 is used to perform a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node if the target device index does not exist in the target index list; The list update module 14 is used to update the device list corresponding to the target industrial inference device based on the target device information after completing the traversal of each device index in the preset device index list, so as to complete the security scanning operation corresponding to the target industrial inference device.

[0071] As can be seen, the device scanning method in the visual inspection process provided in this embodiment first determines the device index corresponding to the target camera that is running and summarizes it into a target index list before performing device scanning. During the traversal of the preset device index list, target device indexes that exist in the target index list are skipped directly without performing any device opening or device query operations on them, thereby avoiding device resource conflicts between the scanning process and the ongoing video acquisition and inference tasks, and achieving zero interference to the inference task. At the same time, for target device indexes that do not exist in the target index list, the device query operation is performed normally and the device list is updated based on the obtained target device information. This enables the target industrial inference device to continuously discover newly connected devices and track the recovery status of faulty devices during the operation of the inference task, achieving a balance between the continuity of device management and the stability of the inference task, and providing a reliable basis for the automatic recovery of faulty devices.

[0072] In one specific implementation, the index list determination module 11 may include: The index list determination unit is used to obtain a set of running camera threads using the camera manager of the target industrial inference device, determine the device running status of the target camera corresponding to each camera thread in the set of running camera threads as running, and summarize the device indexes corresponding to each camera thread in the set of running camera threads to determine the target index list.

[0073] In one specific embodiment, the device may further include: The device information processing module is used to determine the USB port path, manufacturer identifier, product identifier, and serial number corresponding to the device node based on the target device information, and to remove the target interface number suffix from the USB port path to determine the normalized port path corresponding to the device node; the normalized port path is used to characterize the physical port corresponding to the device node. The device identifier determination module is used to determine a target hash based on the normalized port path, the manufacturer identifier, the product identifier, and the serial number, so as to determine the target device identifier corresponding to the device node based on the target hash; Accordingly, the device may further include: The camera to be recovered determination module is used to determine the camera to be recovered based on the configuration information of the target industrial inference device; the camera to be recovered is the target camera that should be in operation as recorded in the configuration information and whose current operating state is not in operation. The identifier comparison module is used to compare the target device identifier corresponding to the camera to be recovered with each target device identifier recorded in the updated device list to obtain the corresponding target comparison result; The recovery task execution module is used to determine the device index of the device node corresponding to the target device identifier based on the device list if there is a target device identifier in the updated device list that is the same as the target device identifier corresponding to the camera to be recovered, and to execute the target recovery task corresponding to the camera to be recovered using the device index of the device node corresponding to the target device identifier.

[0074] In one specific implementation, the device identification module may include: The first hash determination unit is used to determine a first target hash based on the normalized port path, the manufacturer identifier, and the product identifier if the serial number does not meet the preset availability conditions. The second hash determination unit is used to determine a second target hash based on the manufacturer identifier, the product identifier, and the device node path corresponding to the device node if the normalized port path does not meet the preset availability conditions. The third hash determination unit is used to perform a hash operation on the device node path to determine a third target hash if the normalized port path, the manufacturer identifier, the product identifier, and the serial number do not meet the preset availability conditions.

[0075] In one specific embodiment, the device may further include: The first inference task restart module is used to determine whether the fault device index corresponding to the fault camera has changed if the target camera is detected to be a fault camera, and if the fault device index corresponding to the fault camera has not changed, terminate the camera thread corresponding to the fault camera and release the corresponding device resources, and restart the target inference task corresponding to the fault camera based on the fault device index; the fault device index is the device index of the device node corresponding to the fault camera.

[0076] In one specific embodiment, the device may further include: The second inference task restart module is used to perform a forced scan operation on the faulty camera to obtain the corresponding forced scan result if the faulty device index corresponding to the faulty camera changes or the target inference task corresponding to the faulty camera fails to restart, and to determine the faulty device index currently corresponding to the faulty camera based on the target device identifier corresponding to the faulty camera and the forced scan result, so as to restart the target inference task corresponding to the faulty camera based on the faulty device index currently corresponding to the faulty camera.

[0077] In one specific implementation, the index list determination module 11 may include: The operation status determination unit is used to detect the device operation status of each target camera corresponding to the target industrial inference device based on a preset scanning cycle. Accordingly, the device may further include: The scanning cycle adjustment module is used to dynamically adjust the preset scanning cycle based on the updated device list, so as to scan the device operating status of each target camera based on the adjusted preset scanning cycle.

[0078] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0079] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the device scanning method during the visual inspection process disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0080] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0081] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include an operating system 221, computer programs 222, etc., and the storage method can be temporary storage or permanent storage.

[0082] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the device scanning method in the visual inspection process executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0083] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the device scanning method in the aforementioned visual inspection process. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0085] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0087] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0088] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A device scanning method in a visual inspection process, characterized in that, Applied to target industrial inference equipment, including: The device operating status of each target camera corresponding to the target industrial inference device is detected, and a target index list is determined based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera; A target device index is determined based on a preset device index list. It is then determined whether the target device index exists in the target index list. If the target device index exists in the target index list, the next target device index is determined based on the preset device index list, and the process jumps to the step of determining whether the target device index exists in the target index list. The preset device index list is used to store device indexes within a preset index range. If the target device index does not exist in the target index list, then a device query operation is performed on the device node corresponding to the target device index to determine the target device information corresponding to the device node; After traversing each device index in the preset device index list, the device list corresponding to the target industrial inference device is updated based on the target device information to complete the security scan operation corresponding to the target industrial inference device.

2. The device scanning method in the visual inspection process according to claim 1, characterized in that, The step of detecting the device operating status of each target camera corresponding to the target industrial inference device, and determining the target index list based on the device operating status of each target camera, includes: The camera manager of the target industrial inference device is used to obtain a set of running camera threads, and the device running status of the target camera corresponding to each camera thread in the set of running camera threads is determined to be running. The device indexes corresponding to each camera thread in the set of running camera threads are summarized to determine the target index list.

3. The device scanning method in the visual inspection process according to claim 2, characterized in that, After performing a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node, the method further includes: Based on the target device information, the USB port path, manufacturer identifier, product identifier, and serial number corresponding to the device node are determined, and the target interface number suffix of the USB port path is removed to determine the normalized port path corresponding to the device node; the normalized port path is used to characterize the physical port corresponding to the device node; The target hash is determined based on the normalized port path, the manufacturer identifier, the product identifier, and the serial number, and the target device identifier corresponding to the device node is determined based on the target hash. Accordingly, after updating the device list corresponding to the target industrial inference device based on the target device information, the process further includes: The camera to be recovered is determined based on the configuration information of the target industrial inference device; the camera to be recovered is the target camera that should be in operation as recorded in the configuration information and whose current operating state is not in operation. The target device identifier corresponding to the camera to be recovered is compared with each target device identifier recorded in the updated device list to obtain the corresponding target comparison result; If the target comparison result indicates that there is a target device identifier in the updated device list that is the same as the target device identifier corresponding to the camera to be recovered, then the device index of the device node corresponding to the target device identifier is determined based on the device list, and the target recovery task corresponding to the camera to be recovered is executed using the device index of the device node corresponding to the target device identifier.

4. The device scanning method in the visual inspection process according to claim 3, characterized in that, The step of determining the target hash based on the normalized port path, the vendor identifier, the product identifier, and the serial number includes: If the serial number does not meet the preset availability conditions, then a first target hash is determined based on the normalized port path, the manufacturer identifier, and the product identifier; If the normalized port path does not meet the preset availability condition, then a second target hash is determined based on the manufacturer identifier, the product identifier, and the device node path corresponding to the device node. If the normalized port path, the manufacturer identifier, the product identifier, and the serial number do not meet the preset availability conditions, then a hash operation is performed on the device node path to determine the third target hash.

5. The device scanning method in the visual inspection process according to claim 3, characterized in that, Also includes: If the target camera is detected to be a faulty camera, it is determined whether the faulty device index corresponding to the faulty camera has changed. If the faulty device index corresponding to the faulty camera has not changed, the camera thread corresponding to the faulty camera is terminated and the corresponding device resources are released. The target inference task corresponding to the faulty camera is restarted based on the faulty device index. The faulty device index is the device index of the device node corresponding to the faulty camera.

6. The device scanning method in the visual inspection process according to claim 5, characterized in that, Also includes: If the faulty device index corresponding to the faulty camera changes, or the target inference task corresponding to the faulty camera fails to restart, a forced scan operation is performed on the faulty camera to obtain the corresponding forced scan result. Based on the target device identifier corresponding to the faulty camera and the forced scan result, the current faulty device index corresponding to the faulty camera is determined, so as to restart the target inference task corresponding to the faulty camera based on the current faulty device index corresponding to the faulty camera.

7. The device scanning method in the visual inspection process according to claim 1, characterized in that, The detection of the device operating status of each target camera corresponding to the target industrial inference device includes: The device operating status of each target camera corresponding to the target industrial inference device is detected based on a preset scanning cycle. Accordingly, after updating the device list corresponding to the target industrial inference device based on the target device information to complete the security scanning operation corresponding to the target industrial inference device, the method further includes: The preset scanning cycle is dynamically adjusted based on the updated device list, so as to scan the device operating status of each target camera based on the adjusted preset scanning cycle.

8. A device scanning apparatus for a visual inspection process, characterized in that, Applied to target industrial inference equipment, including: The index list determination module is used to detect the device operating status of each target camera corresponding to the target industrial inference device, and determine the target index list based on the device operating status of each target camera; the target index list is used to store the device index corresponding to the target camera whose device operating status is running, and the device index is the number assigned by the operating system of the target industrial inference device to the device node corresponding to the target camera; An index determination module is used to determine a target device index based on a preset device index list, determine whether the target device index exists in the target index list, and if the target device index exists in the target index list, determine the next target device index based on the preset device index list, and jump to the step of determining whether the target device index exists in the target index list; the preset device index list is used to store the device indexes within a preset index range; The node query module is used to perform a device query operation on the device node corresponding to the target device index to determine the target device information corresponding to the device node if the target device index does not exist in the target index list; The list update module is used to update the device list corresponding to the target industrial inference device based on the target device information after completing the traversal of each device index in the preset device index list, so as to complete the security scanning operation corresponding to the target industrial inference device.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the device scanning method in the visual inspection process as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the device scanning method in the visual inspection process as described in any one of claims 1 to 7.

Images