Function inspection system and method for smart terminal device USB device

Abstract

The application discloses a kind of function inspection system and method of intelligent terminal equipment USB device, solve the problem that artificial reading chart is inefficient, error-prone, cannot be traced back, realize the closed-loop detection of principle diagram to full-process automation.The function inspection system includes: image acquisition module for acquiring the principle diagram image of equipment to be inspected;Image preprocessing module for pre-processing principle diagram image;Visual recognition and topology analysis module for identifying function module, extracting module model and port identification, extracting connection line segment to construct the signal flow topology structure from main control chip to each USB port;Structured list generation module for generating structured detection list according to topology structure;MES integration and task scheduling module for associating detection list with production work order and distributing to equipment to be inspected;Test execution and feedback module for automatically executing USB test on equipment to be inspected, comparing actual results with detection list and generating test report.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation testing and production information technology, and specifically refers to a functional testing system and method for USB devices in smart terminal equipment. Background Technology

[0002] Android-based smart terminal devices (such as interactive whiteboards and industrial PCs) typically integrate multiple USB hubs, USB switches, and USB ports to provide a rich array of USB peripheral interfaces and functions. During manufacturing, to ensure product quality, these USB devices must undergo functional testing to verify that their connections meet design requirements. Traditional testing methods primarily rely on manual operation: inspectors first consult the device's schematic diagram (see...). Figure 1 The manual inspection process involves manually checking the number of USB hubs, USB ports, and Type-C ports with USB interfaces, and then comparing this information with the actual device list enumerated by the equipment testing software. Currently, this manual inspection method has several drawbacks: First, the schematic diagrams often contain a large number of components and complex connections, making manual verification time-consuming and inefficient, especially in multi-model mixed production scenarios that severely impact production cycle time; second, manual reading of diagrams is prone to omissions and errors due to fatigue or negligence, making accuracy difficult to guarantee; third, the test results are usually stored in paper records or spreadsheets, making automatic summarization, analysis, and traceability difficult, and failing to form an effective data loop with the production information system.

[0003] In recent years, OCR technology has been widely used for document digitization. However, general OCR mainly targets natural scene text or standard documents, lacking the specialized ability to analyze the unique text layout (such as internal module annotations and port labels) and connection topology of engineering diagrams, and thus cannot understand the logical connection relationships between devices. In existing technologies, the recognition results have not been automatically integrated with the production line testing process, still requiring manual intervention to convert the recognition results into test instructions, failing to fundamentally solve the problem of low automation in inspection.

[0004] Therefore, there is an urgent need for an intelligent inspection method that can automatically parse the principle block diagram, extract the USB device connection relationship, and deeply integrate with the manufacturing execution system (MES). Summary of the Invention

[0005] The main objective of this invention is to provide a functional testing system and method for USB devices in smart terminal equipment, which solves the problems of low efficiency, error-proneness, and lack of traceability in the prior art due to manual diagram reading, and realizes closed-loop testing with full automation from schematic diagram to production line.

[0006] To achieve the above objectives, one solution of the present invention is: A functional testing system for USB devices in smart terminal devices, comprising: The image acquisition module is used to acquire the schematic diagram image of the device to be tested. The image preprocessing module is used to preprocess the principle block diagram image; The visual recognition and topology analysis module is used to identify functional modules in the pre-processed image using a YOLO object detection model with added CBAM attention mechanism, extract module model and port identifier using the PaddleOCR framework, extract connecting line segments using the LSD line segment detection algorithm, and construct the signal flow topology from the main control chip to each USB port through geometric matching and breadth-first search. A structured list generation module is used to generate a structured detection list based on the topology. The MES integration and task scheduling module is used to associate the inspection list with the production work order and distribute it to the equipment to be inspected; Test execution and feedback module: Used to automatically perform USB tests on the device to be tested, compare the actual results with the test checklist and generate a test report.

[0007] Preferably, the visual recognition and topology analysis module is also used to eliminate isolated modules and / or line segments that are not connected to the main control chip, and to assign channel identifiers to the USB switch chip according to the pin connection relationship.

[0008] Preferably, the MES integration and task scheduling module is also used to automatically obtain the corresponding principle block diagram image from the manufacturing execution system according to the production work order, and automatically send the test report back to the manufacturing execution system.

[0009] The second solution of the present invention is: A method for functionally verifying a USB device in a smart terminal device includes the following steps: Step S1: Obtain the schematic diagram image of the device to be tested; Step S2: Preprocess the principle block diagram image; Step S3: For the preprocessed image, the YOLO object detection model with CBAM attention mechanism is used to identify the functional modules in the image, the PaddleOCR framework is used to extract the module model and port identifier, the LSD line segment detection algorithm is used to extract the connecting line segments, and the signal flow topology from the main control chip to each USB port is constructed through geometric matching and breadth-first search. Step S4: Generate a structured detection list based on the topology; Step S5: Associate the inspection list with the production work order and distribute it to the equipment to be inspected; Step S6: Automatically perform USB testing on the device to be tested, compare the actual results with the test list, and generate a test report.

[0010] Preferably, in step S3, after constructing the topology, isolated modules and / or line segments that are not connected to the main control chip are removed, and channel identifiers are assigned to the USB switch chip according to the pin connection relationship.

[0011] Preferably, in step S1, the corresponding principle block diagram image is automatically obtained from the manufacturing execution system according to the production work order; in step S6, the test report is automatically sent back to the manufacturing execution system.

[0012] Preferably, in step S2, image preprocessing includes one or more of grayscale conversion, binarization, noise reduction, and tilt correction.

[0013] Preferably, in step S6, the USB test includes USB port scanning and / or USB device enumeration.

[0014] Preferably, in step S3, geometric matching is achieved by using OpenCV to spatially match line segments with module pins.

[0015] Preferably, in step S3, a breadth-first search traverses all connected paths and records the complete path from the main control chip to each USB port.

[0016] After adopting the above technical solution, the present invention has the following technical effects: (1) It realizes fully automatic analysis of the principle block diagram, eliminating the need for manual diagram reading and greatly improving inspection efficiency; (2) By using the combined algorithm of YOLO+OCR+LSD, modules, text and connection relationships are accurately identified, avoiding missed detections and false detections; (3) Deep integration with MES enables automatic scheduling of testing tasks, result feedback and data traceability, forming a production closed loop. Attached Figure Description

[0017] Figure 1 A schematic diagram of the principle drawn by the designer in the existing technology.

[0018] Figure 2 This is a flowchart of a specific embodiment of the present invention. Detailed Implementation

[0019] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0020] This invention discloses a functional testing system for USB devices in smart terminal equipment, comprising the following modules: Image acquisition module: Acquires schematic diagram images of the equipment to be inspected (supports PNG, JPG, PDF and other formats), or obtains the corresponding schematic diagram file from the MES system library based on the work order.

[0021] Image preprocessing module: preprocesses the block diagram image, specifically by performing grayscale conversion, binarization, noise reduction, and tilt correction to obtain a clear and processable binary image.

[0022] Visual Recognition and Topology Analysis Module: This module uses a YOLOv11n object detection model with added CBAM attention mechanism to recognize functional modules in the block diagram. It combines an optimized OCR engine for system block diagrams to extract module models and port identifiers. The LSD (Line Segment Detector) algorithm is used to extract all connecting lines. OpenCV geometric analysis is used to achieve spatial matching between lines and module pins. Starting from the SOC chip, a breadth-first search is used to traverse all connected paths to automatically construct a complete logical connection topology. Based on this, modules and lines not connected to the SOC are removed, and the multi-path-to-single Switch chip is analyzed path by path.

[0023] Structured Inventory Generation Module: Converts the parsed "module-port-connected device" relationship into a standardized structured inspection inventory.

[0024] MES Integration and Task Scheduling Module: Automatically associates the list with the production work order in MES and schedules and distributes it to the corresponding equipment to be inspected.

[0025] Test execution and feedback module: Automatically performs USB port scanning and device enumeration on the device side, compares the actual results with the test list and generates a report, which is then automatically sent back to MES.

[0026] refer to Figure 2 As shown, the present invention also discloses a method for functional testing of USB devices in smart terminal devices, comprising the following steps: Step S1: Obtain the schematic diagram image of the device to be tested.

[0027] Step S2: Preprocess the block diagram image (e.g., grayscale conversion, binarization, noise reduction, and tilt correction).

[0028] Step S3: For the preprocessed image, a YOLO object detection model with CBAM attention mechanism is used to identify functional modules in the image. The PaddleOCR framework is used to extract module model numbers and port identifiers. The LSD line segment detection algorithm is used to extract connecting line segments, and several OpenCV implementations are used to achieve spatial matching between line segments and module pins. A breadth-first search is then used to construct the signal flow topology from the main control chip to each USB port. Isolated modules and / or line segments not connected to the main control chip are removed, and channel identifiers are assigned to the USB switch chips based on pin connection relationships. Specifically, the breadth-first search traverses all connected paths and records the complete path from the main control chip to each USB port.

[0029] Step S4: Generate a structured checklist based on the topology.

[0030] Step S5: Associate the inspection list with the production work order and distribute it to the equipment to be inspected.

[0031] Step S6: Automatically perform USB tests (such as USB port scanning, USB device enumeration, etc.) on the device to be tested, compare the actual results with the test list, and generate a test report.

[0032] Compared with the prior art, the present invention has the following advantages through the above solution: (1) Significantly improved inspection efficiency: Automatic analysis of the principle block diagram reduces the traditional manual inspection time of several minutes to seconds, supports rapid switching between multiple models of mixed production, and reduces the impact on production cycle.

[0033] (2) High accuracy and consistency: CBAM-YOLO is used to enhance the detection capability of small devices. Combined with PaddleOCR and LSD algorithms, text and line segments are accurately extracted. The connection relationship is automatically verified by geometric matching and BFS to eliminate human error and ensure consistent and reliable test results.

[0034] (3) Full-process automated closed loop: Seamless connection from block diagram analysis, list generation, MES task scheduling to terminal testing and result feedback, eliminating data silos and realizing traceability and auditability of the inspection process.

[0035] (4) Intelligent processing of complex topology: Automatically identify the multiplexing mode of USB Switch (multi-in-one or one-output), assign channel identifiers, and generate an accurate detection list containing signal paths to facilitate fault location.

[0036] (5) Deep integration with MES: Supports automatic acquisition of work order diagrams, automatic task distribution, automatic report return, and structured storage of detection data to support quality traceability and digital factory construction.

[0037] The following lists the relevant content of the algorithm implementation of this invention: I. YOLOv11n Object Detection Model with Added CBAM Attention Mechanism (a) Establishing a principle block diagram dataset Collect various USB system schematic diagram files (including PDF, PNG, JPG, etc.) and segment them into individual images. Use the VoTT annotation tool to manually annotate the functional modules in each image, select five categories of targets: SOC chip, USB Switch, USB Hub, USB-A port, and Type-C port, and assign corresponding category labels to each annotation box to form an initial annotation dataset.

[0038] For YOLO's DETECT format annotation rules, the annotation results exported from VoTT are converted into YOLO format txt files. Each image corresponds to a txt file with the same name. Each line in the file represents a target and contains the following information: (I) USB Device Category ID: According to the predefined category label, such as SOC=0, Switch=1, Hub=2, USB-A=3, Type-C=4; (II) The normalized bounding box coordinates are in the following format: <Category ID><Center point x normalized coordinate><Center point y normalized coordinate><Width normalized value><Height normalized value>. All coordinate values ​​are normalized by dividing by the image width and height, and the values ​​range from 0 to 1.

[0039] To improve the model's generalization ability and robustness, multi-level data augmentation techniques are applied during training: a) Geometric enhancement: Random horizontal flipping, random vertical flipping, random translation, random scaling, and perspective transformation are used to simulate changes in the drawing from different perspectives; b) Color space enhancement: Randomly adjust the hue, saturation, and brightness of the image to enhance the model's adaptability to differences in lighting and print quality; c) Advanced composite enhancement: Introducing Mosaic enhancement and MixUp enhancement, multiple images are randomly stitched or mixed to enrich the contextual information of the training samples.

[0040] (II) Improved YOLOv11n-CBAM model USB device identification is performed through training and inference using an improved YOLOv11n-CBAM model and the aforementioned annotation files. Specifically, a CBAM attention mechanism is embedded at the end of the YOLOv11n backbone network, after the SPPF layer. CBAM consists of two sub-modules, channel attention and spatial attention, arranged sequentially to adaptively refine the feature maps. Channel attention module: Compresses spatial dimensions through average pooling and max pooling, generates channel weights through a shared multilayer perceptron, and focuses on channel features with important semantics in the feature map; Spatial Attention Module: Performs average pooling and max pooling on the channel dimension, concatenates the data, and generates spatial weights through convolutional layers to highlight the responses of key regions in the feature map.

[0041] Embedding CBAM after the SPPF layer, the feature map at that location already contains rich semantic information, which can further suppress background noise response, enhance attention to small-sized devices (such as USB ports and Switch chips), and amplify the most distinguishing features between different types of devices, effectively reducing the probability of false detection.

[0042] The trained model can automatically detect each functional module in the block diagram and output the position coordinates, confidence score and category attribute of each detected USB device, providing accurate module position information for subsequent topology analysis.

[0043] II. Optimized OCR Engine Based on System Block Diagram The PP-OCRv4 model from the PaddleOCR framework is used for text tag detection and recognition. As a lightweight OCR tool, PP-OCRv4 maintains high accuracy while possessing excellent inference speed, supports multilingual and complex scene recognition, and is particularly suitable for the real-time deployment needs of industrial production lines.

[0044] To adapt to the textual characteristics of the system principle block diagram, a dedicated dataset was constructed to optimize the model. Spatial association was performed between the recognized textual information and the functional modules detected in step one, ultimately establishing complete model and port labeling information for each USB device.

[0045] III. LSD Line Segment Detection Algorithm LSD (Line Segment Detector) is a sub-pixel precision detection algorithm that can quickly extract line segments from images. This algorithm requires no manual parameter tuning and uses a built-in error control mechanism to ensure that only one false alarm is allowed per image on average, making it particularly suitable for scenarios with dense lines and complex connections, such as engineering drawings.

[0046] In this method, the LSD algorithm is used to extract all connecting line segments in the principle block diagram, providing basic topology data for subsequent SOC connection determination and multi-way switch resolution. Starting from the preprocessed binary image, the algorithm employs four core steps: gradient calculation, region growing, rectangle fitting, and NFA verification, ultimately outputting the precise start and end coordinates of each line segment. Specific parameter settings are as follows: Image preprocessing scaling factor σscale=0.6; gradient calculation using a 2×2 mask: gx = 2 / [I(x+1,y) - I(x,y) + I(x+1,y+1) - I(x,y+1)] gy = 2 / [I(x,y+1) - I(x,y) + I(x+1,y+1) - I(x+1,y)] Next, the gradient magnitude and direction are calculated. Compared to the traditional Hough transform (ρ = x cosθ + y sinθ), this scheme has significant advantages such as linear time complexity O(n), built-in NFA false positive control (average only 1 false alarm per image), and direct output of line segment width and confidence level. The measured accuracy is 96.8%, the recall is 94.2%, and the processing time is approximately 45ms.

[0047] The final output includes the coordinates of the line segment endpoints, width, confidence level, connection module information, and network topology. It is seamlessly integrated with YOLO module detection and BFS topology analysis, providing high-precision and high-efficiency line segment detection support for the automated parsing of multi-way switch principle block diagrams.

[0048] IV. OpenCV Ablation of Unconnected Lines After YOLOv11n-CBAM target detection and LSD line segment detection, a large number of functional modules and connecting lines can be obtained. However, not all detected lines and modules form a valid signal path with the main control SOC chip; retaining them would interfere with subsequent topology analysis and detection list generation.

[0049] Therefore, this invention uses OpenCV masking technology to automatically filter out modules and lines that are not connected to the SOC based on the line segment coordinate information extracted by LSD, retaining only the effective connected parts, and providing a clean topology for subsequent processing.

[0050] V. Breadth-First Search (BFS) Traversal and Path Enumeration Module Receive the set of valid modules M_valid and the set of valid line segments L_valid after connectivity filtering to construct a complete signal flow topology and enumerate all paths from the SOC chip to each USB port.

[0051] An undirected graph G=(V, E) is constructed based on the matching relationship between module pins and line segments. The node set V contains all valid functional modules and their virtual intersections, and the edge set E corresponds to the connecting line segments. Each edge includes information about the pins it connects to. Using the SOC chip node as the root node, a breadth-first search (BFS) algorithm is used to traverse all reachable nodes in the graph, recording the complete path from the SOC to each terminal node (USB-A port, Type-C port).

[0052] During the traversal, when a USB Switch node is encountered, the working mode of the Switch (such as multiple inputs combined into one output, or one input with multiple outputs) is automatically determined by combining the model information and pin connection relationship identified by OCR, and channel numbers (such as "SW1" and "SW2") are assigned according to the pin labels or geometric positions to ensure that each path accurately marks the channels it passes through.

[0053] After traversal, a structured list of paths is output. Each path contains: source SOC pins, sequence of intermediate devices passed through (including module ID, model, and channel number), target USB port pins, and a readable complete path description (e.g., "SOC_USB2 → Switch_U5(CH1) → Type-C1_CC").

[0054] The above embodiments and figures are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. A functional testing system for USB devices in intelligent terminal equipment, characterized in that, include: The image acquisition module is used to acquire the schematic diagram image of the device to be tested. The image preprocessing module is used to preprocess the principle block diagram image; The visual recognition and topology analysis module is used to identify functional modules in the pre-processed image using a YOLO object detection model with added CBAM attention mechanism, extract module model and port identifier using the PaddleOCR framework, extract connecting line segments using the LSD line segment detection algorithm, and construct the signal flow topology from the main control chip to each USB port through geometric matching and breadth-first search. A structured list generation module is used to generate a structured detection list based on the topology. The MES integration and task scheduling module is used to associate the inspection list with the production work order and distribute it to the equipment to be inspected; Test execution and feedback module: Used to automatically perform USB tests on the device to be tested, compare the actual results with the test checklist and generate a test report.

2. The functional testing system for USB devices in intelligent terminal equipment as described in claim 1, characterized in that: The visual recognition and topology analysis module is also used to eliminate isolated modules and / or line segments that are not connected to the main control chip, and to assign channel identifiers to the USB switch chip according to the pin connection relationship.

3. The functional testing system for USB devices in intelligent terminal equipment as described in claim 1, characterized in that: The MES integration and task scheduling module is also used to automatically obtain the corresponding principle block diagram image from the manufacturing execution system according to the production work order, and automatically send the test report back to the manufacturing execution system.

4. A method for functional testing of USB devices in intelligent terminal equipment, characterized in that, Includes the following steps: Step S1: Obtain the schematic diagram image of the device to be tested; Step S2: Preprocess the principle block diagram image; Step S3: For the preprocessed image, the YOLO object detection model with CBAM attention mechanism is used to identify the functional modules in the image, the PaddleOCR framework is used to extract the module model and port identifier, the LSD line segment detection algorithm is used to extract the connecting line segments, and the signal flow topology from the main control chip to each USB port is constructed through geometric matching and breadth-first search. Step S4: Generate a structured detection list based on the topology; Step S5: Associate the inspection list with the production work order and distribute it to the equipment to be inspected; Step S6: Automatically perform USB testing on the device to be tested, compare the actual results with the test list, and generate a test report.

5. The functional testing method for USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S3, after constructing the topology, isolated modules and / or line segments that are not connected to the main control chip are removed, and channel identifiers are assigned to the USB switch chip according to the pin connection relationship.

6. The functional testing method for USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S1, the corresponding principle block diagram image is automatically obtained from the manufacturing execution system according to the production work order; in step S6, the test report is automatically sent back to the manufacturing execution system.

7. The functional testing method for USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S2, image preprocessing includes one or more of grayscale conversion, binarization, noise reduction, and tilt correction.

8. The functional testing method for USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S6, the USB test includes USB port scanning and / or USB device enumeration.

9. The functional testing method for USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S3, geometric matching is achieved by using OpenCV to spatially match line segments with module pins.

10. The method for functional testing of USB devices in intelligent terminal devices as described in claim 4, characterized in that: In step S3, a breadth-first search traverses all connected paths and records the complete path from the main control chip to each USB port.

Images