A soft and hard integrated multi-source sensor intelligent testing system and method

Abstract

This invention relates to the field of sensor testing technology, specifically to an integrated hardware and software multi-source sensor intelligent testing system and method. The method includes: acquiring the physical connection status of the target sensor; responding to a one-click detection command, establishing a physical communication connection between the target sensor and a host computer through a unified access module; the host computer executing data acquisition from LiDAR, a vision camera, and an inertial navigation IMU through a multi-threaded concurrent acquisition module; merging the acquired data into a multi-threaded secure data queue, and pushing the operating status to a monitoring UI interface in real time for display through a feedback monitoring module combined with a Qt signal slot mechanism; inputting abnormal data streams into a diagnostic evaluation module, performing anomaly identification and classification through secondary semantic parsing, and outputting a quality inspection report. This invention achieves automated quality inspection of multi-source sensors, optimizes the quality inspection process, shortens assembly and adjustment time, and improves troubleshooting efficiency.

Description

Technical Field

[0001] This invention relates to the field of sensor testing technology, and in particular to an integrated hardware and software multi-source sensor intelligent testing system and method. Background Technology

[0002] With the rapid development of intelligent connected vehicles and autonomous driving technologies, the number of environmental perception sensors (such as LiDAR, cameras, and IMUs) installed on vehicles has increased dramatically. During vehicle production assembly, off-line quality inspection, and after-sales maintenance, it is necessary to quickly verify the hardware quality, data connectivity, and imaging / detection quality of these sensors.

[0003] Currently, quality inspection of automotive sensors primarily relies on traditional single-point testing solutions. The specific procedures involve testers needing to separately open the LiDAR manufacturer's dedicated client software, a serial port debugging assistant for reading the IMU, and the system camera software for testing the cameras. Physically, the sensor wiring harnesses are haphazardly plugged into different computer ports; software-wise, testers need to manually configure the LiDAR's target IP and device port, manually select the IMU's COM port and baud rate, and observe the data displayed on different software interfaces.

[0004] Existing technologies have the following drawbacks: frequent software switching leads to low efficiency; traditional methods require operating three or more isolated software programs separately, lacking linkage, resulting in cumbersome testing processes and consuming a lot of time; physical wiring is complicated; LiDAR usually uses Ethernet protocol, while cameras and IMUs usually use USB / serial protocol, and direct plugging and unplugging leads to messy test platform wiring harnesses and easy loosening of interfaces; there is a lack of intelligent fault diagnosis methods. When hardware malfunctions (such as loose cables, unstable voltage, or occupied ports), traditional software often freezes directly or throws obscure and difficult-to-understand low-level error codes, relying heavily on the personal experience of test engineers for troubleshooting, making it difficult to guarantee repair accuracy and efficiency. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention proposes an integrated hardware and software multi-source sensor intelligent testing system and method. By constructing an integrated hardware and software testing architecture and combining multi-threaded concurrent acquisition technology with an intelligent diagnostic engine, automated quality inspection of multi-source sensors is achieved.

[0006] On one hand, embodiments of the present invention provide a hardware-software integrated multi-source sensor intelligent testing method, the method comprising the following steps: The physical connection status of the target sensors is obtained, including a lidar, a visual camera, and an inertial navigation IMU. In response to a one-click detection command, a physical communication connection between the target sensor and the host computer is established through a unified access module; The host computer executes data acquisition from the lidar, the vision camera, and the inertial navigation IMU through a multi-threaded concurrent acquisition module. The lidar uses a subprocess to call the native lidar software and combines it with a robot process automation script to complete target IP positioning, device port configuration, and connection operations to obtain point cloud data. The vision camera uses an independent camera thread to call the underlying API to extract video streams to complete high-definition image decoding and imaging quality verification. The inertial navigation IMU uses an independent serial port thread to read data streams and uses a built-in protocol parser to identify frame headers and instruction types for data unpacking and status verification. The point cloud data, high-definition images, and verification data of the inertial navigation IMU are merged into a multi-threaded secure data queue. The running status is pushed to the monitoring UI interface loaded by the host computer in real time for display through the feedback monitoring module combined with the Qt signal slot mechanism. The abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends the diagnostic suggestions to the report generation module to output a quality inspection report.

[0007] Optionally, establishing a physical communication connection between the target sensor and the host computer through the unified access module includes: The unified access module receives Ethernet protocol data frames sent by the lidar through its Ethernet interface. The unified access module receives USB protocol data frames sent by the visual camera and the inertial navigation IMU through its USB interface. After the Ethernet protocol data frames and the USB protocol data frames are converted into a unified USB protocol data stream, they are transmitted to the host computer through a single high-speed data line.

[0008] Optionally, the lidar uses a subprocess to call the lidar's native software and combines it with a robotic process automation script to complete target IP location, device port configuration, and connection operations to obtain point cloud data, including: The host computer creates a sub-process through the lidar acquisition module and starts the lidar native software. The robotic process automation script simulates mouse clicks and keyboard input operations, locates the target IP address input box in the native LiDAR software interface, and fills in the target IP address; The robotic process automation script simulates the operation, locates the device port configuration item in the native software interface of the LiDAR, and sets the device port. The robotic process automation script simulates clicking the connect button to establish a communication connection with the lidar. The system receives point cloud data packets sent by the lidar in real time and parses the three-dimensional coordinate information and reflection intensity information in the point cloud data packets. The parsed point cloud data is sent to the data queue.

[0009] Optionally, the visual camera uses a separate camera thread to call the underlying API to extract the video stream in order to complete the decoding of high-definition images and verification of image quality, including: The host computer initializes the camera thread through the camera acquisition module and configures the video stream resolution to 1280×720 pixels. A video stream connection with the vision camera is established by calling the underlying API through an open-source computer vision library. The visual camera captures video frame data in MJPG compressed format in real time. The video frame data is decoded in real time to generate uncompressed high-definition image data; The high-definition image data is sent to the data queue, and the imaging screen is displayed in real time on the monitoring UI interface.

[0010] Optionally, the inertial navigation IMU reads the data stream through an independent serial port thread and performs data unpacking and status verification by identifying frame headers and command types through a built-in protocol parser, including: The host computer establishes a serial communication connection with the inertial navigation IMU through the IMU acquisition module; The hexadecimal data stream output by the inertial navigation IMU is read in real time, and specific frame header identifiers in the data frames are detected. Identify the instruction type field in the data frame to distinguish between attitude data instructions and raw data instructions; Extract the three-axis attitude angle data and acceleration data according to the protocol format corresponding to the instruction type field; The extracted three-axis attitude angle data and acceleration data are sent to the data queue.

[0011] Optionally, the abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends diagnostic suggestions to the report generation module to output a quality inspection report, including: When the diagnostic assessment module detects a file not found error or access denied exception, it determines it to be a port read failure type, retrieves the maintenance knowledge base, and outputs maintenance suggestions to check whether the sensor data cable is plugged in properly or whether the device port is occupied. When the diagnostic evaluation module detects a serial anomaly or an unexpected interruption of the serial stream, it determines it to be an unexpected disconnection during operation and suggests troubleshooting for unstable output power supply voltage or physical looseness of the interface. When the diagnostic evaluation module detects that the image array returns a null value or a read timeout error, it determines that it is a visual frame loss type and outputs maintenance suggestions to check for video cable damage or channel number selection. The maintenance recommendations are sent to the report generation module to generate a quality inspection report that includes the type of abnormality and the corresponding maintenance instructions.

[0012] Optionally, after merging the point cloud data, high-resolution images, and the verification data from the inertial navigation IMU into a multi-threaded, thread-safe data queue, the method further includes: The feedback monitoring module extracts the working status and connection progress information of each sensor from the data queue; The working status and connection progress information are converted into visual signals using the Qt signal slot mechanism; The visualization signal is sent to the monitoring UI interface to drive the progress bar control and log terminal control to update the displayed content in real time. The system traceability log is printed in chronological order in the log terminal control, recording the connection establishment time, data acquisition time, and anomaly occurrence time of each sensor.

[0013] Optionally, before acquiring the physical connection status of the target sensor, the method further includes: The host computer loads the monitoring dashboard program developed based on the PySide6 framework and initializes the monitoring UI interface. The monitoring UI interface configures a sensor parameter input area to receive user input of the target IP address and device port of the lidar, the serial port identifier and baud rate configuration parameters of the inertial navigation IMU; Configure a start detection button control in the monitoring UI interface and establish an association between the start detection button control and the instruction triggering of the main thread; Initialize the multi-threaded safe data queue, and set the queue capacity limit and thread lock mechanism; Preload the maintenance knowledge base and establish a mapping table between anomaly type identifiers and maintenance suggestion texts.

[0014] On the other hand, embodiments of the present invention provide a hardware-software integrated multi-source sensor intelligent testing system, including: The unified access module integrates Ethernet and USB interfaces; A host computer is communicatively connected to the unified access module. The host computer includes at least one processor and at least one memory. The memory stores at least one program. When the at least one program is executed by the at least one processor, the at least one processor implements the above-described method.

[0015] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the above method when executed by the processor.

[0016] The embodiments of the present invention include the following beneficial effects: Through the integrated test architecture of software and hardware integration, the present invention uses a unified access module with Ethernet and USB interfaces to converge the physical links of sensors, and is overall managed by a single software of the upper computer, solving the problem of messy wiring harnesses on the previous test bench. The tester can plug and use, improving the stability of physical connection and the standardization of operation; through the one-key automatic detection method for multi-source sensors, using multi-threading technology to simultaneously execute the concurrent quality inspection process of robot process automation scripts calling lidar, video stream decoding and imaging, and serial port hexadecimal protocol parsing, realizing one-key linkage start, completely eliminating the inefficient links of frequently switching software and manually configuring parameters in the past, and greatly shortening the average working hours per vehicle for installation and commissioning and quality inspection; through the intelligent diagnosis method based on abnormal feature matching, mapping the underlying hardware communication error reports to the intelligent maintenance knowledge base, and automatically outputting colloquial and guiding troubleshooting suggestions, making the troubleshooting process change from manually checking codes and guessing reasons in the past to the system directly informing the maintenance location, significantly improving the accuracy of fault troubleshooting. Even first-line workers with less experience can accurately troubleshoot. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 is a schematic flowchart of a software and hardware integrated multi-source sensor intelligent test method provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the standardization of the physical access path of sensors provided by an embodiment of the present invention; <000008​​​​​​​​It should be noted that although the device diagram is divided into modules and the flowchart shows the logical order, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0022] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of the invention.

[0023] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0024] The flowchart shown in the attached figures is merely an illustrative example and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described.

[0025] like Figure 1 As shown in the figure, an intelligent testing method for a hardware-software integrated multi-source sensor provided by an embodiment of the present invention includes the following steps: S100, Obtain the physical connection status of the target sensor, which includes a lidar, a visual camera, and an inertial navigation IMU; S200, in response to a one-click detection command, establishes a physical communication connection between the target sensor and the host computer through a unified access module; S300, the host computer executes data acquisition from the lidar, the vision camera, and the inertial navigation IMU through a multi-threaded concurrent acquisition module. The lidar uses a subprocess to call the native lidar software and combines it with a robot process automation script to complete target IP positioning, device port configuration, and connection operations to obtain point cloud data. The vision camera uses an independent camera thread to call the underlying API to extract the video stream to complete the decoding of high-definition images and image quality verification. The inertial navigation IMU uses an independent serial port thread to read the data stream and uses a built-in protocol parser to identify the frame header and instruction type to perform data unpacking and status verification. S400, the point cloud data, high-definition images and the verification data of the inertial navigation IMU are merged into a multi-threaded secure data queue, and the running status is pushed to the monitoring UI interface loaded by the host computer in real time for display through the feedback monitoring module combined with the Qt signal slot mechanism; S500, the abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends the diagnostic suggestions to the report generation module to output a quality inspection report.

[0026] The steps S100 to S500 provided in this embodiment of the invention organically combine the standardization of physical access of multi-source sensors, the parallelization of data acquisition, the real-time monitoring of status, and the intelligent diagnosis of anomalies through an integrated hardware and software testing architecture, thereby realizing the fully automated closed-loop control from the physical connection of sensors to the output of quality inspection reports.

[0027] like Figure 2 The illustration shows a standardized example of the physical access path for sensors provided in this embodiment of the invention. This invention employs a centralized hardware topology architecture based on a unified access module, which is clearly divided into a sensor layer, a relay hub layer, and a host computer layer at the physical level.

[0028] Specifically, the core of the relay hub layer is a unified access module 104, which integrates an Ethernet port (i.e., RJ45, registered J45 standard network interface), multiple USB interfaces (universal serial bus interface) and a USB host interface.

[0029] In terms of physical data link connection, various sensors in the sensor layer are independently connected to the unified access module 104: the LiDAR 101 is physically connected to the Ethernet port of the unified access module 104 via the network cable 111; the Vision Camera 102 is physically connected to the USB interface of the unified access module 104 via the first USB cable 112; at the same time, the Inertial Measurement Unit (IMU) 103 is physically connected to another USB interface of the unified access module 104 via the second USB cable 113.

[0030] After the aggregation of the underlying devices is completed, the unified access module 104 establishes a physical communication connection with the host computer 105 (i.e., the quality inspection computer terminal) in the upper computer layer via its USB host interface and a single high-speed data line 114 (i.e., USB cable) to the built-in USB host interface (e.g., USB 3.0 standard interface).

[0031] Through the above architectural design, this invention successfully standardizes and unifies the complex physical interfaces of sensor clusters. In actual industrial production line applications, assembly engineers only need to perform a single mainline plug-and-unplug operation on a single high-speed data cable 114 to instantly complete the physical access and communication link establishment of all underlying hardware, significantly reducing the complexity of hardware wiring and improving system deployment efficiency.

[0032] In this embodiment, the heterogeneous physical interfaces (Ethernet, USB) of the LiDAR 101, visual camera 102, and inertial navigation IMU 103 are aggregated into a single USB data stream through the unified access module 104. This stream is then connected to the host computer 105 via a single high-speed data cable 114. This solves the problem of low deployment efficiency caused by complex physical wiring and different interface types of multi-source sensors, and achieves the beneficial effects of simplifying hardware wiring, shortening installation and commissioning time, and reducing human error rate in wiring.

[0033] like Figure 3 As shown, the host computer system 201 of the present invention is developed based on PySide6 (a Python graphical user interface library based on the Qt framework). Its system architecture mainly includes three logical links: human-computer interaction monitoring, multi-threaded concurrent data acquisition, and intelligent evaluation report.

[0034] This embodiment uses the PySide6 framework to build the host computer system 201, and utilizes Qt's signal and slot mechanism to realize cross-thread data transmission and UI updates. This solves the problems of interface lag and data loss in the traditional single-thread acquisition mode, and achieves the beneficial effects of human-computer interaction monitoring and multi-threaded concurrent acquisition collaborative operation, improving system response speed and data acquisition real-time performance.

[0035] In some embodiments, before acquiring the physical connection status of the target sensor, the method further includes: S101, The host computer loads the monitoring dashboard program developed based on the PySide6 framework and initializes the monitoring UI interface. S102, Configure the sensor parameter input area in the monitoring UI interface, and receive the target IP address and device port of the lidar, the serial port identifier and baud rate configuration parameters of the inertial navigation IMU input by the user; S103, Configure a start detection button control in the monitoring UI interface, and establish an association between the start detection button control and the instruction triggering of the main thread; S104, Initialize the multi-threaded safe data queue, set the queue capacity limit and thread lock mechanism; S105, preload the maintenance knowledge base and establish a mapping table between abnormality type identifiers and maintenance suggestion texts.

[0036] This embodiment solves the problem of detection interruption caused by parameter configuration errors or uninitialized resources by preloading the monitoring dashboard program, configuring sensor parameters, and initializing a multi-threaded secure data queue and maintenance knowledge base before detection. It achieves the beneficial effects of standardized detection process, traceable parameter configuration, and evidence-based anomaly diagnosis.

[0037] In some embodiments, establishing a physical communication connection between the target sensor and the host computer through a unified access module includes: S210, receive Ethernet protocol data frames sent by the lidar through the Ethernet interface of the unified access module; receive USB protocol data frames sent by the vision camera and the inertial navigation IMU through the USB interface of the unified access module; S220: After converting the Ethernet protocol data frame and the USB protocol data frame into a unified USB protocol data stream, it is transmitted to the host computer through a single high-speed data line.

[0038] This embodiment executes S210 and S220 through the unified access module to convert heterogeneous protocol data frames (Ethernet protocol, USB protocol) into a unified USB protocol data stream. This solves the problems of incompatibility between multi-source sensor protocols and the need for the host computer to configure multiple interface drivers, achieving the beneficial effects of transparent protocol transmission, reduced host computer interface dependence, and improved data transmission stability.

[0039] In some embodiments, the host computer executes data acquisition from the lidar, the vision camera, and the inertial navigation IMU through a multi-threaded concurrent acquisition module. The lidar uses a subprocess to call the native lidar software and combines it with a robot process automation script to complete target IP positioning, device port configuration, and connection operations to obtain point cloud data. The vision camera uses an independent camera thread to call the underlying API to extract video streams to complete high-definition image decoding and imaging quality verification. The inertial navigation IMU uses an independent serial port thread to read data streams and uses a built-in protocol parser to identify frame headers and instruction types for data unpacking and status verification.

[0040] This embodiment solves the problems of low efficiency and asynchronous data acquisition timing of traditional serial acquisition methods by using a multi-threaded concurrent acquisition module to execute the LiDAR subprocess, the visual camera thread, and the IMU serial port thread in parallel. This achieves the beneficial effects of improving overall detection efficiency, ensuring the consistency of timestamps of multi-source data, and shortening the single detection cycle.

[0041] In some embodiments, the lidar uses a subprocess to call the lidar's native software and combines it with a robotic process automation script to complete target IP location, device port configuration, and connection operations to obtain point cloud data, including: S311, the host computer creates a sub-process through the lidar acquisition module and starts the lidar native software; S312, the robot process automation script simulates mouse clicks and keyboard input operations, locates the target IP address input box in the native LiDAR software interface, and fills in the target IP address; S313, the robot process automation script simulates the operation by locating the device port configuration item in the native software interface of the LiDAR and setting the device port; S314, the robot process automation script simulates clicking the connection button to establish a communication connection with the lidar; S315, receive point cloud data packets sent by the lidar in real time, and parse the three-dimensional coordinate information and reflection intensity information in the point cloud data packets; S316, the parsed point cloud data is sent to the data queue.

[0042] This embodiment utilizes robotic process automation scripts (S311 to S316) to automatically complete the IP location, port configuration, and connection operations of the native LiDAR software. This solves the problems of low efficiency and error-proneness in manually configuring LiDAR parameters one by one, achieving the beneficial effects of automating LiDAR data acquisition, eliminating human configuration errors, and improving the reliability of point cloud data acquisition.

[0043] In some embodiments, the visual camera uses a separate camera thread to call the underlying API to extract the video stream in order to complete the decoding of high-definition images and verification of image quality, including: S321, the host computer initializes the camera thread through the camera acquisition module and configures the video stream resolution to 1280×720 pixels; S322, establish a video stream connection with the vision camera by calling the underlying API through an open-source computer vision library; S323, capture video frame data in MJPG compressed format from the visual camera in real time; S324, The video frame data is decoded in real time to generate uncompressed high-definition image data; S325, the high-definition image data is sent to the data queue, and the imaging screen is displayed in real time on the monitoring UI interface.

[0044] This embodiment, through steps S321 to S325, employs an independent camera thread combined with an open-source computer vision library to capture and decode MJPG video streams in real time. This solves the problems of data acquisition conflicts between the visual camera and the main thread, as well as image decoding delays, achieving the beneficial effects of real-time acquisition of high-definition images and simultaneous verification of imaging quality, ensuring the integrity and timeliness of visual data.

[0045] In some embodiments, the inertial navigation IMU reads the data stream through an independent serial port thread and performs data unpacking and status verification by identifying the frame header and instruction type through a built-in protocol parser, including: S331, the host computer establishes a serial communication connection with the inertial navigation IMU through the IMU acquisition module; S332, read the hexadecimal data stream output by the inertial navigation IMU in real time and detect specific frame header identifiers in the data frames; S333, identify the instruction type field in the data frame and distinguish between attitude data instructions and raw data instructions; S334, extract the three-axis attitude angle data and acceleration data according to the protocol format corresponding to the instruction type field; S335, the extracted three-axis attitude angle data and acceleration data are sent to the data queue.

[0046] This embodiment, through steps S331 to S335, utilizes an independent serial port thread combined with a built-in protocol parser to identify frame headers, instruction types, and extract three-axis attitude angles and acceleration data. This solves the problems of data extraction errors caused by reliance on manual interpretation of inertial navigation IMU data stream parsing and complex protocol formats. It achieves automatic unpacking and status verification of inertial navigation data, ensuring the accuracy and real-time performance of attitude data parsing.

[0047] This embodiment aggregates multi-source sensor data through a multi-threaded, secure data queue and uses the Qt signal slot mechanism to push the operating status to the monitoring UI interface in real time. This solves the problems of data contention and status display delay in a multi-threaded environment, achieving simultaneous data acquisition and status monitoring, improving system operation transparency and data throughput stability.

[0048] In some embodiments, after merging the point cloud data, high-resolution images, and the verification data of the inertial navigation IMU into a multi-threaded, secure data queue, the method further includes: S410, the feedback monitoring module extracts the working status and connection progress information of each sensor from the data queue; S420, using the Qt signal slot mechanism, converts the working status and the connection progress information into visual signals; S430, the visualization signal is sent to the monitoring UI interface to drive the progress bar control and log terminal control to update the displayed content in real time; S440, the system traceability log is printed in chronological order in the log terminal control, recording the connection establishment time, data acquisition time and anomaly occurrence time of each sensor.

[0049] This embodiment extracts status information from the data queue and drives the UI control to update in real time through steps S410 to S440, while recording system traceability logs. This solves the problems of invisible status and difficulty in tracing fault time points during the detection process, achieving the beneficial effects of fully transparent monitoring of the detection process, accurate location of abnormal time, and improved fault troubleshooting efficiency.

[0050] In some embodiments, the abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends the diagnostic suggestions to the report generation module to output a quality inspection report.

[0051] This embodiment uses a diagnostic evaluation module to perform secondary semantic analysis and classification of abnormal data streams, which solves the problems of traditional detection systems that only report errors without locating the root cause and lack intelligent fault diagnosis methods. It achieves the beneficial effects of automatic identification of abnormal types, timely push of maintenance suggestions, and improved troubleshooting efficiency.

[0052] In some embodiments, the abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends diagnostic suggestions to the report generation module to output a quality inspection report, including: S510, when the diagnostic evaluation module captures a file not found error or access denied exception, it determines that the port read failure type, retrieves the maintenance knowledge base, and outputs maintenance suggestions to check whether the sensor data cable is plugged in tightly or whether the device port is occupied. S520, when the diagnostic evaluation module detects a serial abnormality or unexpected interruption of the serial stream, it determines it to be an unexpected disconnection during operation and provides troubleshooting suggestions for unstable power supply voltage or physical looseness of the interface. S530, when the diagnostic evaluation module detects that the image array returns a null value or a read timeout error, it determines that it is a visual frame loss type and outputs maintenance suggestions to check for video cable damage or channel number selection. S540, the maintenance suggestion is sent to the report generation module to generate a quality inspection report containing the abnormality type and corresponding maintenance instructions.

[0053] This embodiment, through steps S510 to S540, matches maintenance suggestions from the maintenance knowledge base with three types of anomalies: port read failure, unexpected disconnection during operation, and visual frame loss, and generates quality inspection reports. This solves the problems of relying on human experience for anomaly handling and non-standardized maintenance guidance, and achieves the beneficial effects of accurate anomaly root cause location, structured output of maintenance suggestions, and traceable quality inspection.

[0054] This invention also provides a hardware-software integrated multi-source sensor intelligent testing system, comprising: The unified access module integrates Ethernet and USB interfaces; A host computer is communicatively connected to the unified access module. The host computer includes at least one processor and at least one memory. The memory stores at least one program. When the at least one program is executed by the at least one processor, the at least one processor implements the above-described method.

[0055] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0056] This embodiment integrates the unified access module and the host computer into a hardware and software system, enabling the program in the memory to perform all the above-mentioned steps when executed by the processor. This solves the problems of poor compatibility and complex deployment caused by the separation of hardware and software in traditional testing systems, achieving the beneficial effects of system integration, plug-and-play functionality, and reduced operation and maintenance costs.

[0057] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0058] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0059] This invention also provides a computer program product, including a computer program or computer instructions, which are stored in a memory. A processor of a computer device reads the computer program or computer instructions from the memory and executes the computer program or computer instructions, causing the computer device to perform the above-described method.

[0060] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0061] In summary, this invention solves the technical problems of complex physical wiring, low testing efficiency, and lack of intelligent fault diagnosis methods in the prior art by using an integrated hardware and software testing architecture, a one-click automated detection method for multi-source sensors, and an intelligent diagnostic method based on abnormal feature matching. It achieves the beneficial effects of optimizing the quality inspection process, shortening the assembly and adjustment time, and improving troubleshooting efficiency.

[0062] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0063] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A hardware-software integrated multi-source sensor intelligent testing method, characterized in that, The method includes the following steps: The physical connection status of the target sensors is obtained, including a lidar, a visual camera, and an inertial navigation IMU. In response to a one-click detection command, a physical communication connection between the target sensor and the host computer is established through a unified access module; The host computer executes data acquisition from the lidar, the vision camera, and the inertial navigation IMU through a multi-threaded concurrent acquisition module. The lidar uses a subprocess to call the native lidar software and combines it with a robot process automation script to complete target IP positioning, device port configuration, and connection operations to obtain point cloud data. The vision camera uses an independent camera thread to call the underlying API to extract video streams to complete high-definition image decoding and imaging quality verification. The inertial navigation IMU uses an independent serial port thread to read data streams and uses a built-in protocol parser to identify frame headers and instruction types for data unpacking and status verification. The point cloud data, high-definition images, and verification data of the inertial navigation IMU are merged into a multi-threaded secure data queue. The running status is pushed to the monitoring UI interface loaded by the host computer in real time for display through the feedback monitoring module combined with the Qt signal slot mechanism. The abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends the diagnostic suggestions to the report generation module to output a quality inspection report.

2. The method according to claim 1, characterized in that, The step of establishing a physical communication connection between the target sensor and the host computer through a unified access module includes: The Ethernet protocol data frames sent by the lidar are received through the Ethernet interface of the unified access module. The unified access module receives USB protocol data frames sent by the visual camera and the inertial navigation IMU through its USB interface. After the Ethernet protocol data frames and the USB protocol data frames are converted into a unified USB protocol data stream, they are transmitted to the host computer through a single high-speed data line.

3. The method according to claim 1, characterized in that, The lidar uses a subprocess to call the lidar's native software and combines it with a robotic process automation script to complete target IP location, device port configuration, and connection operations to obtain point cloud data, including: The host computer creates a subprocess through the lidar acquisition module and starts the lidar's native software. The robotic process automation script simulates mouse clicks and keyboard input operations, locates the target IP address input box in the native LiDAR software interface, and fills in the target IP address; The robotic process automation script simulates the operation, locates the device port configuration item in the native software interface of the LiDAR, and sets the device port. The robotic process automation script simulates clicking the connect button to establish a communication connection with the lidar. The system receives point cloud data packets sent by the lidar in real time and parses the three-dimensional coordinate information and reflection intensity information in the point cloud data packets. The parsed point cloud data is sent to the data queue.

4. The method according to claim 1, characterized in that, The visual camera uses a separate camera thread to call the underlying API to extract the video stream in order to complete the decoding of high-definition images and the verification of image quality, including: The host computer initializes the camera thread through the camera acquisition module and configures the video stream resolution to the target pixels. A video stream connection with the vision camera is established by calling the underlying API through an open-source computer vision library. The visual camera captures video frame data in MJPG compressed format in real time. The video frame data is decoded in real time to generate uncompressed high-definition image data; The high-definition image data is sent to the data queue, and the imaging screen is displayed in real time on the monitoring UI interface.

5. The method according to claim 1, characterized in that, The inertial navigation IMU reads the data stream through an independent serial port thread and uses a built-in protocol parser to identify the frame header and command type for data unpacking and status verification, including: The host computer establishes a serial communication connection with the inertial navigation IMU through the IMU acquisition module; The hexadecimal data stream output by the inertial navigation IMU is read in real time, and specific frame header identifiers in the data frames are detected. Identify the instruction type field in the data frame to distinguish between attitude data instructions and raw data instructions; Extract the three-axis attitude angle data and acceleration data according to the protocol format corresponding to the instruction type field; The extracted three-axis attitude angle data and acceleration data are sent to the data queue.

6. The method according to claim 1, characterized in that, The abnormal data streams thrown by the multi-threaded concurrent acquisition module, the camera thread, and the serial port thread are input into the diagnostic evaluation module. The diagnostic evaluation module performs abnormal identification and classification through secondary semantic parsing, and sends diagnostic suggestions to the report generation module to output a quality inspection report, including: When the diagnostic assessment module detects a file not found error or access denied exception, it determines it to be a port read failure type, retrieves the maintenance knowledge base, and outputs maintenance suggestions to check whether the sensor data cable is plugged in properly or whether the device port is occupied. When the diagnostic evaluation module detects a serial anomaly or an unexpected interruption of the serial stream, it determines it to be an unexpected disconnection during operation and suggests troubleshooting for unstable output power supply voltage or physical looseness of the interface. When the diagnostic evaluation module detects that the image array returns a null value or a read timeout error, it determines that it is a visual frame loss type and outputs maintenance suggestions to check for video cable damage or channel number selection. The maintenance recommendations are sent to the report generation module to generate a quality inspection report that includes the type of abnormality and the corresponding maintenance instructions.

7. The method according to claim 1, characterized in that, After merging the point cloud data, high-resolution images, and the verification data from the inertial navigation IMU into a multi-threaded, secure data queue, the method further includes: The feedback monitoring module extracts the working status and connection progress information of each sensor from the data queue; The working status and connection progress information are converted into visual signals using the Qt signal slot mechanism; The visualization signal is sent to the monitoring UI interface to drive the progress bar control and log terminal control to update the displayed content in real time. The system traceability log is printed in chronological order in the log terminal control, recording the connection establishment time, data acquisition time, and anomaly occurrence time of each sensor.

8. The method according to claim 1, characterized in that, Before acquiring the physical connection status of the target sensor, the method further includes: The host computer loads the monitoring dashboard program developed based on the PySide6 framework and initializes the monitoring UI interface. The monitoring UI interface configures a sensor parameter input area to receive user input of the target IP address and device port of the lidar, the serial port identifier and baud rate configuration parameters of the inertial navigation IMU; Configure a start detection button control in the monitoring UI interface and establish an association between the start detection button control and the instruction triggering of the main thread; Initialize the multi-threaded safe data queue, and set the queue capacity limit and thread lock mechanism; Preload the maintenance knowledge base and establish a mapping table between anomaly type identifiers and maintenance suggestion texts.

9. A hardware-software integrated multi-source sensor intelligent testing system, characterized in that, include: The unified access module integrates Ethernet and USB interfaces; A host computer is communicatively connected to the unified access module. The host computer includes at least one processor and at least one memory. The memory stores at least one program. When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to perform the method as described in any one of claims 1 to 8.

Images