Automated test performance benchmark construction method, equipment and storage medium

Abstract

The present application relates to the field of industrial visual inspection and provides a method for constructing an automated test performance benchmark, the method comprising: dynamically generating region of interest (ROI) parameters; performing pixel cropping on a target area of ​​a production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and aligning the image acquisition moment with the phase of the mechanical movement of the production line; selecting a matching visual inspection model based on target features in the high-speed image data stream, and outputting defect detection results by the matching visual inspection model; embedding simulated defect features into the high-speed image data stream according to defect distribution parameters of a preset test scenario to generate a mixed test data stream; performing multi-device timing alignment on the mixed test data stream and defect detection results, and encapsulating them into a standardized data set containing a unified time reference; parsing the standardized data set, and calculating the multi-dimensional performance indicators of the visual inspection system by comparing the simulated defect true value labels with the defect detection results, and generating a test report containing parameter optimization suggestions.

Description

Technical Field

[0001] The present invention relates to the field of industrial visual inspection, and in particular to a method, device and storage medium for constructing an automated test performance benchmark. Background Art

[0002] With the advancement of intelligent industrial manufacturing, visual inspection systems have become a core component of production line quality control. This is particularly true in precision manufacturing (such as semiconductors and automotive parts), where high-speed production lines place higher demands on the real-time performance, stability, and accuracy of visual inspection systems. Traditional industrial vision systems typically operate at a fixed frame rate, making them difficult to adapt to the precise capture and detection of dynamic objects required in high-speed assembly line scenarios.

[0003] To improve detection efficiency, existing technologies often employ methods such as local image acquisition optimization or model switching detection. However, these technologies suffer from shortcomings such as insufficient adaptability to dynamic scenarios, poor controllability of test data, and a lack of system synergy. Summary of the Invention

[0004] The present application provides a method, device and storage medium for constructing an automated test performance benchmark, which can provide an efficient, reliable and reproducible automated test benchmark for high-speed industrial visual inspection systems.

[0005] In one aspect, the present application provides a method for constructing an automated test performance benchmark, the method comprising:

[0006] Dynamically generate region of interest (ROI) parameters;

[0007] Pixel cropping of the target area of ​​the production line is performed based on dynamically generated ROI parameters, generating a high-speed image data stream and aligning the image acquisition moment with the mechanical motion phase of the production line;

[0008] Selecting a matching visual inspection model based on target features in the high-speed image data stream, and outputting a defect detection result by the matching visual inspection model;

[0009] According to the defect distribution parameters of the preset test scenario, the simulated defect features are embedded in the high-speed image data stream to generate a hybrid test data stream;

[0010] Performing multi-device timing alignment on the mixed test data stream and the defect detection results, and encapsulating them into a standardized data set containing a unified time reference;

[0011] The standardized data set is parsed, and by comparing the simulated defect true value labels with the defect detection results, the multi-dimensional performance indicators of the visual inspection system are calculated and a test report including parameter optimization suggestions is generated.

[0012] On the other hand, the present application provides an automated test performance benchmark construction device, the device comprising:

[0013] The first generation module is used to dynamically generate region of interest (ROI) parameters;

[0014] The first alignment module is used to perform pixel cropping of the target area of ​​the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition moment with the mechanical motion phase of the production line;

[0015] an output module, configured to select a matching visual inspection model based on target features in the high-speed image data stream, and output defect detection results from the matching visual inspection model;

[0016] A second generating module is configured to embed simulated defect features into the high-speed image data stream according to defect distribution parameters of a preset test scenario to generate a mixed test data stream;

[0017] a second alignment module, configured to perform multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulate them into a standardized data set containing a unified time reference;

[0018] The third generation module is used to parse the standardized data set, calculate the multi-dimensional performance indicators of the visual inspection system by comparing the true value labels of the simulated defects with the defect detection results, and generate a test report containing parameter optimization suggestions.

[0019] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the technical solution of the above-mentioned method for constructing an automated test performance benchmark are implemented.

[0020] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the above-mentioned method for constructing an automated test performance benchmark.

[0021] From the technical solutions provided by the present application, it can be seen that, on the one hand, by dynamically generating ROI parameters to adaptively crop the target area, combining the image acquisition time with the phase alignment of the mechanical movement of the production line, while improving the image acquisition frame rate, it ensures the precise alignment of the acquisition time with the mechanical movement of the production line, solves the problem of mismatch between image data and physical movement phase in high-speed scenes, and provides high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate hybrid test data, it not only retains the environmental characteristics of the real production line, but also realizes the controllable adjustment of defect types and distribution, providing a data foundation with both authenticity and flexibility for the test process; thirdly, by parsing the standardized data set to generate multi-dimensional indicators covering detection accuracy, processing efficiency and resource consumption, and combining historical data to provide parameter optimization suggestions, it realizes the upgrade from single result judgment to systematic performance analysis, and provides data support for the continuous optimization of the visual inspection system. In summary, the technical solution of the present application can effectively overcome the core problems of poor adaptability to dynamic scenes, insufficient authenticity of test data, and lack of multi-device collaboration in the existing technology, and provides an efficient, reliable and reproducible automated test benchmark for high-speed industrial visual inspection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 This is a flowchart of the method for constructing an automated test performance benchmark provided by an embodiment of the present application;

[0024] Figure 2 This is a schematic diagram of the structure of the automated test performance benchmark construction device provided in an embodiment of the present application;

[0025] Figure 3 It is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] In this specification, adjectives such as first and second may be used only to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0028] In this specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn according to the actual proportions.

[0029] With the intelligent transformation of industrial manufacturing, visual inspection systems have become a core component of production line quality control. In particular, in precision manufacturing (such as semiconductors and automotive parts), high-speed production lines place higher demands on the real-time performance, stability, and accuracy of visual inspection systems. Traditional industrial vision systems typically operate at a fixed frame rate, making them difficult to adapt to the precise capture and detection of dynamic targets in high-speed assembly line scenarios. Existing technologies typically employ the following two approaches to improve inspection efficiency:

[0030] 1. Local image acquisition optimization: Fixed region ROI (region of interest) cropping is used to reduce image processing workload. However, static ROI settings cannot adapt to the trajectory changes of moving targets, which can easily lead to target loss or data redundancy.

[0031] 2. Multi-model switching detection: Multiple visual models are preloaded for different detection tasks. However, model switching can lead to issues such as excessive video memory usage and significant switching delays, impacting detection continuity. Furthermore, existing test benchmarks often rely on manually constructed test data, resulting in insufficient defect simulation realism and a lack of control over the timing consistency of multi-device collaboration, making test results difficult to reproduce and compare horizontally.

[0032] The core defects of the above-mentioned prior art are:

[0033] 1) Insufficient adaptability to dynamic scenes, specifically in that static ROI and fixed model loading strategies are difficult to match the detection requirements of high-speed moving targets;

[0034] 2) Test data has poor controllability, specifically due to deviations between artificially constructed defects and the actual production line environment, which affects the credibility of test results.

[0035] 3) Lack of system coordination, specifically manifested in the fact that the timing asynchrony between multiple devices leads to data misalignment and the inability to establish a unified performance evaluation benchmark.

[0036] In response to the above problems of the prior art, this application proposes a method for constructing an automated test performance benchmark, the flow chart of which is shown in the attached figure. Figure 1As shown, it mainly includes steps S101 to S106, which are detailed as follows:

[0037] Step S101: Dynamically generate region of interest (ROI) parameters.

[0038] In the field of image processing, if the region of interest (ROI) is fixed, on the one hand, it will inevitably contain a large number of irrelevant background pixels, which will increase the processing burden and prevent the frame rate from breaking through the frame rate bottleneck. For example, 150FPS cannot reach a high frame rate of 200FPS+. On the other hand, when high-speed moving targets easily move out of the fixed ROI range, subsequent detection fails and the system error rate increases. To solve the problems caused by the fixed ROI mentioned above, this application adopts a solution for dynamically generating ROI parameters. Specifically, as an embodiment of this application, the dynamic generation of ROI parameters can be achieved through steps S1011 to S1013, as detailed below:

[0039] Step S1011: Call the camera SDK instruction to limit the pixel area read by the sensor and generate an initial cropping range.

[0040] Step S1012: performing displacement compensation on the initial cropping range based on the target motion speed prediction model, wherein the displacement compensation amount is calculated according to the pixel displacement speed of the target between consecutive frames.

[0041] Specifically, the displacement compensation of the initial cropping range based on the target motion speed prediction model can be achieved through the following steps Sa1 to Sa5:

[0042] Step Sa1: Track the target through consecutive frames.

[0043] Specifically, the original image data of two adjacent frames (Frame N and Frame N+1) can be obtained, and then the position difference of the same target in the two frames can be identified through a feature matching algorithm (such as ORB feature point matching).

[0044] Step Sa2: Calculate pixel displacement speed.

[0045] Specifically, the pixel displacement of the measurement target in the X / Y axis direction Then, according to the frame interval time ( , determined by the inverse of the camera frame rate), calculate the target motion speed: horizontal speed , vertical speed .

[0046] Step Sa3: Predict the target's movement speed based on the target's current movement speed.

[0047] Specifically, a speed prediction model is established and the exponential smoothing method (EMA) is used for short-term prediction. The formula is: , ,in, is the smoothing coefficient, ranging from 0.6 to 0.9; the input of the speed prediction model is the historical speed series , the output is the predicted speed of the next acquisition cycle .

[0048] Step Sa4: Calculate the ROI displacement compensation amount based on the predicted target motion speed.

[0049] Specifically, the compensation amount can be calculated according to the predicted speed: , vertical compensation , then, generate the new ROI center coordinates: , .

[0050] Step Sa5: Dynamically adjust ROI.

[0051] Specifically, instructions can be sent through the camera SDK to update the ROI position to the compensated coordinates. Then, it is verified in real time whether the target is completely within the new ROI area. If there is a boundary overflow, the ROI range is expanded.

[0052] Step S1013: iteratively reducing the cropping range of the ROI to the target motion trajectory coverage area, so as to increase the image acquisition frame rate to above the preset frame rate.

[0053] Specifically, iteratively reducing the cropping range of the ROI to the target motion trajectory coverage area to increase the image acquisition frame rate to above the preset frame rate can be achieved through the following steps Sb1 to Sb5:

[0054] Step Sb1: Model the target's motion trajectory.

[0055] Specifically, the motion trajectory coordinates of the target in N consecutive frames (such as 50 frames) can be recorded; and the motion trajectory equation of the target, such as a quadratic curve equation, can be fitted using the least squares method.

[0056] Step Sb2: Determine the initial range of ROI.

[0057] Specifically, the maximum / minimum value of the trajectory in the X / Y axis direction can be calculated based on the fitted target motion trajectory equation; the initial ROI is set as a rectangular area covering the extreme points of the trajectory, and expanded outward by a preset ratio, such as 5% as a safety margin.

[0058] Step Sb3: Iteratively optimize ROI.

[0059] Specifically, iterative ROI optimization mainly includes reducing the ROI range, coverage verification of the ROI range, and dynamic adjustment of the ROI range. Among them, reducing the ROI range can be specifically reducing the ROI size along the X / Y axis direction at a fixed step size (for example, reducing it by 2% per step), keeping the ROI center coincident with the center of the trajectory fitting curve; covering the ROI range can be specifically detecting whether M consecutive frames (for example, 20 frames) completely contain the target within the reduced ROI area, and using the background difference method to determine whether the target boundary contacts the ROI edge; dynamically adjusting the ROI range can be specifically: if there is no contact with the ROI edge for several consecutive frames, for example, more than 10 frames, then continue to reduce the ROI range. If ROI edge contact is detected, then return to the last valid ROI range and stop reducing.

[0060] Step Sb4: Improve the image acquisition frame rate.

[0061] Specifically, the theoretical frame rate improvement factor can be calculated based on the ratio of the final optimized ROI area (S_optimized) to the maximum supported area of ​​the sensor (S_max), that is, the frame rate improvement factor K = S_max / S_optimized. Then, the ROI mode interface of the camera SDK is called to set the acquisition frame rate to the base frame rate × K (the base frame rate is usually 50 FPS, and can reach 200 FPS+ when K ≥ 4).

[0062] Step Sb5: Process the exception.

[0063] That is, when the target motion trajectory changes suddenly (for example, the speed changes by more than 15%), the ROI reset mechanism is triggered and the above steps Sb1 to Sb3 are executed again.

[0064] Through the above dynamic adjustment and iterative optimization of ROI, the core contradiction between ROI position drift and limited frame rate in high-speed scenes is resolved.

[0065] Step S102: Pixel cropping is performed on the target area of ​​the production line according to the dynamically generated ROI parameters, a high-speed image data stream is generated, and the image acquisition time is aligned with the mechanical motion phase of the production line.

[0066] Experiments have shown that if the deviation between the captured image and the actual motion of the robotic arm exceeds a preset value, such as 1ms, the positioning error of defect detection in high-speed conveyor belt scenarios can reach millimeter levels. This also causes multiple devices (such as cameras, PLC devices, and mechanical controllers) to be unable to coordinate due to timing deviations, making the test benchmark unreliable. Therefore, this method can be used to crop the target area of ​​the production line based on dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition time with the phase of the production line's mechanical motion.

[0067] Specifically, considering that the Programmable Logic Controller (PLC) signal is the core control source of the production line beat, direct analysis can avoid the additional delay introduced by the external trigger signal. The PTP protocol can control the clock deviation within 1μs, while the NTP protocol error reaches the millisecond level, which cannot meet the requirements of high-speed scenarios. In an embodiment of the present application, aligning the image acquisition time with the mechanical motion phase of the production line includes: parsing the device status code sent by the PLC based on the unified clock reference calibrated by PTP, and generating a camera trigger signal synchronized with the production line beat; receiving the real-time motion phase feedback signal of the mechanical controller, and dynamically adjusting the trigger signal generation time so that the deviation between the image acquisition and the mechanical motion phase is less than a preset threshold, for example, 50μs. The receiving the real-time motion phase feedback signal of the mechanical controller and dynamically adjusting the trigger signal generation time so that the deviation between the image acquisition and the mechanical motion phase is less than the preset threshold can be specifically: calculating the clock deviation value of the image acquisition device by comparing the unified timestamp of the device with the clock of the mechanical controller; dynamically adjusting the depth of the image data buffer queue according to the clock deviation value of the acquisition device to compensate for the timing jitter caused by network delay; calculating the timing deviation between the trigger signal and the target motion phase according to the encoder pulse signal sent by the mechanical controller, and correcting the next trigger time.

[0068] From step S101 to step S102 of the above embodiment, it can be seen that by dynamically generating ROI parameters to adaptively crop the target area, combined with timestamp synchronization technology, while improving the image acquisition frame rate, it ensures that the acquisition moment is accurately aligned with the mechanical movement of the production line, solves the problem of phase mismatch between image data and physical movement in high-speed scenarios, and provides highly timely and high-integrity input data for subsequent detection.

[0069] Step S103: Based on the target features in the high-speed image data stream, a matching visual inspection model is selected, and the matching visual inspection model outputs a defect detection result.

[0070] Considering that, on the one hand, a single model cannot cover multiple defect types (for example, different models are required to detect scratches and foreign objects simultaneously), which increases the missed detection rate; on the other hand, preloading all models without selection and matching can lead to excessive video memory usage. As for achieving real-time model loading through memory resource optimization strategies, this is because memory overflow can interrupt inspections and prevent continuous test execution. Increased model switching time can also easily lead to a loss of real-time performance. Precisely because dynamic model loading and memory optimization are core mechanisms for ensuring inspection accuracy, system stability, and real-time performance, the aforementioned method selects a matching visual inspection model based on target features in a high-speed image data stream, and outputs defect detection results from the matching visual inspection model. Specifically, a matching visual inspection model is selected from a pre-stored model library based on target features in the high-speed image data stream, and a memory resource optimization strategy is used to achieve real-time model loading, with the matching visual inspection model outputting defect detection results. Dynamically selecting and loading visual inspection models based on real-time image features, combined with memory resource optimization strategies, enables on-demand allocation of computing resources during model switching, avoiding the high video memory usage and switching delays caused by model preloading in traditional solutions, and ensuring the continuity and stability of inspection tasks.

[0071] The memory resource optimization strategy of the above embodiment may be: real-time monitoring of GPU memory occupancy and model inference time; when the memory occupancy exceeds a threshold, prioritizing unloading models that are not called by subsequent detection tasks, wherein prioritizing unloading models that are not called by subsequent detection tasks may specifically be establishing a loading priority based on the calling frequency of the model in the test scenario; retaining memory cache for models that are called frequently, and performing complete unloading for models that are called infrequently.

[0072] Step S104: according to the defect distribution parameters of the preset test scenario, the simulated defect features are embedded into the high-speed image data stream to generate a mixed test data stream.

[0073] On the one hand, relying on real production line defects not only fails to control the defect distribution (for example, a scenario with a defect rate of 0.1% requires collecting 100,000 images), but also leads to low test efficiency. In addition, rare defects with a probability of no more than 0.01% are difficult to obtain through natural data, and the test benchmark is incomplete. On the other hand, without true value labels, it is impossible to quantify the detection accuracy (such as F1-Score), and one can only rely on manual visual inspection. The credibility of the results is low, and the lack of benchmark data support makes it impossible to establish a quantitative relationship between model parameters and performance. Therefore, in an embodiment of the present application, it is possible to simulate defect feature embedding (i.e., implanting artificial synthetic defects into high-speed image data streams) and adding true value labels (providing benchmark information such as location and type for synthetic defects) to solve the above dilemma. Specifically, it can be based on the defect distribution parameters of the preset test scenario, and embed the simulated defect features into the high-speed image data stream to generate a mixed test data stream.

[0074] As an embodiment of the present application, according to the defect distribution parameters of the preset test scenario, the simulated defect features are embedded in the high-speed image data stream, and the generation of the hybrid test data stream can be: using the trained generative adversarial network (GAN) to generate a defect feature map that matches the material of the current production line; replacing the original image block in the high-speed image data stream with the defect feature map at a preset ratio, wherein the replacement of the original image block in the high-speed image data stream with the defect feature map at a preset ratio can specifically be: locating the original image block to be replaced in the high-speed image data stream according to the target position information in the defect detection result; dynamically adjusting the transparency parameter of the defect feature layer generated by GAN according to the surface reflection characteristics of the material, so that the simulated defect matches the lighting conditions of the original image. By embedding the simulated defect features into the original image data stream to generate hybrid test data, the environmental characteristics of the real production line are retained, and the defect type and distribution are controllable and adjustable, providing a data foundation for the testing process that is both authentic and flexible.

[0075] Step S105: perform multi-device timing alignment on the mixed test data stream and defect detection results, and encapsulate them into a standardized data set containing a unified time reference.

[0076] Since the image and test results at the same moment may correspond to different physical locations (for example, when the timing deviation is 1ms, the target displacement of the high-speed conveyor belt is 5cm), the detection delay caused by the timing deviation is mistakenly attributed to the model performance problem. If the image, test results and equipment status data are stored in a scattered manner, not only can they not be correlated and analyzed, but the test process cannot be repeated in the absence of unified metadata (such as timestamps, equipment parameters). Therefore, in order to ensure data consistency, reproducibility and cross-system comparability, multi-device timing alignment can be achieved by unifying the time base of image streams, test results and mechanical motion data. At the same time, the standardized data set can be encapsulated by integrating the data and appending unified metadata. Specifically, in an embodiment of the present application, multi-device timing alignment can be performed on the mixed test data stream and defect detection results and encapsulated into a standardized data set containing a unified time base.

[0077] As one embodiment of the present application, multi-device timing alignment of a mixed test data stream and defect detection results may include: attaching a device-unified timestamp generated based on the Precision Time Protocol (PTP) to each image frame in the mixed test data stream and the corresponding defect detection result; detecting image frames or detection result packets whose timing deviation exceeds a threshold by comparing the device-unified timestamps of adjacent frames; interpolating and repairing abnormal frames based on pixel displacement vectors calculated using the optical flow method; and synchronously correcting the associated defect detection results. The abnormal frames are image frames whose timing deviation exceeds the threshold, and interpolating and repairing the abnormal frames based on the pixel displacement vectors calculated using the optical flow method may specifically include: constructing a target's motion trajectory equation between consecutive frames based on the pixel displacement vectors calculated using the optical flow method; inversely compensating the pixel coordinates of the ROI in the abnormal frame using the target's motion trajectory equation between consecutive frames; and updating the corresponding defect detection result position information based on the compensated image coordinates. As for synchronously correcting the associated defect detection results, the timestamps and coordinates of the abnormal detection result packets (i.e., the detection result packets whose timing deviation exceeds the threshold) may be linearly interpolated and compensated based on the repaired image frame timestamps.

[0078] As can be seen from the above embodiments, timing alignment and packaging of test data based on a unified time reference eliminates the impact of clock deviation in the collaborative work of multiple devices, ensures the global consistency and reproducibility of test data, and provides a reliable basis for cross-system performance comparison.

[0079] Step S106: parse the standardized data set, calculate the multi-dimensional performance indicators of the visual inspection system by comparing the simulated defect true value labels with the defect detection results, and generate a test report including parameter optimization suggestions.

[0080] To distinguish between model omissions and defect-free data, and to identify performance bottlenecks (e.g., whether frame rate increases lead to insufficient video memory or model computation timeouts), objective evaluation, efficient tuning, and continuous system improvement can be achieved by parsing standardized datasets. By comparing simulated defect ground truth labels with defect detection results, multidimensional performance indicators of the visual inspection system can be calculated, and a test report containing parameter optimization recommendations can be generated. The multidimensional performance indicators of the visual inspection system can be generated by continuously adjusting the acquisition frame rate within a preset frame rate gradient range and recording the defect detection results at the corresponding time. Based on the simulated defect ground truth labels embedded in the mixed test data stream, a quantitative relationship curve reflecting the change in F1-Score with frame rate is generated. During the model switching process, real-time video memory utilization data is collected to extract the time series characteristics of video memory peak value and the time required to restore stability. Parameter optimization recommendations can be generated by establishing a quantitative relationship matrix between model hyperparameters and detection accuracy and frame rate stability, where detection accuracy is calculated by comparing defect detection results with ground truth labels. A genetic algorithm is then used to search for Pareto-optimal parameter combinations within this relationship matrix. By analyzing standardized data sets to generate multi-dimensional indicators covering detection accuracy, processing efficiency and resource consumption, and combining historical data to provide parameter optimization suggestions, it has achieved an upgrade from single result judgment to systematic performance analysis, providing data support for the continuous optimization of visual inspection systems.

[0081] From the above attached Figure 1 The example of the method for constructing an automated test performance benchmark shows that, on the one hand, by dynamically generating ROI parameters to adaptively crop the target area, combining the image acquisition time with the phase alignment of the mechanical movement of the production line, while improving the image acquisition frame rate, it ensures the precise alignment of the acquisition time with the mechanical movement of the production line, solves the problem of mismatch between image data and physical movement phase in high-speed scenes, and provides high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate hybrid test data, it not only retains the environmental characteristics of the real production line, but also realizes the controllable adjustment of defect types and distribution, providing a data foundation with both authenticity and flexibility for the test process; thirdly, by parsing the standardized data set to generate multi-dimensional indicators covering detection accuracy, processing efficiency and resource consumption, and combining historical data to provide parameter optimization suggestions, it realizes the upgrade from single result judgment to systematic performance analysis, and provides data support for the continuous optimization of the visual inspection system. In summary, the technical solution of the present application can effectively overcome the core problems of poor adaptability to dynamic scenes, insufficient authenticity of test data, and lack of multi-device collaboration in the prior art, and provides an efficient, reliable and reproducible automated test benchmark for high-speed industrial visual inspection systems.

[0082] Please see the attached Figure 2, is an automated test performance benchmark construction device provided in an embodiment of the present application, the device may include a first generation module 201, a first alignment module 202, an output module 203, a second generation module 204, a second alignment module 205, and a third generation module 206, as detailed below:

[0083] The first generating module 201 is used to dynamically generate the parameters of the region of interest (ROI);

[0084] The first alignment module 202 is used to perform pixel cropping on the target area of ​​the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition time with the mechanical motion phase of the production line;

[0085] An output module 203 is configured to select a matching visual inspection model based on target features in the high-speed image data stream, and output defect detection results from the matching visual inspection model;

[0086] The second generating module 204 is configured to embed simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream;

[0087] The second alignment module 205 is used to perform multi-device timing alignment on the mixed test data stream and defect detection results, and encapsulate them into a standardized data set containing a unified time reference;

[0088] The third generation module 206 is used to parse the standardized data set, calculate the multi-dimensional performance indicators of the visual inspection system by comparing the true value labels of the simulated defects with the defect detection results, and generate a test report including parameter optimization suggestions.

[0089] From the above attached Figure 2The example of the automated test performance benchmark construction device shows that, on the one hand, by dynamically generating ROI parameters to adaptively crop the target area, combining the image acquisition time with the phase alignment of the mechanical movement of the production line, while improving the image acquisition frame rate, it ensures the precise alignment of the acquisition time with the mechanical movement of the production line, solves the problem of mismatch between image data and physical movement phase in high-speed scenes, and provides high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate hybrid test data, it not only retains the environmental characteristics of the real production line, but also realizes the controllable adjustment of defect types and distribution, providing a data foundation with both authenticity and flexibility for the test process; thirdly, by parsing the standardized data set to generate multi-dimensional indicators covering detection accuracy, processing efficiency and resource consumption, and combining historical data to provide parameter optimization suggestions, it realizes the upgrade from single result judgment to systematic performance analysis, and provides data support for the continuous optimization of the visual inspection system. In summary, the technical solution of the present application can effectively overcome the core problems of poor adaptability to dynamic scenes, insufficient authenticity of test data, and lack of multi-device collaboration in the prior art, and provides an efficient, reliable and reproducible automated test benchmark for high-speed industrial visual inspection systems.

[0090] Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for the method for constructing an automated test performance benchmark. When the processor 30 executes the computer program 32, the steps in the above-mentioned method for constructing an automated test performance benchmark are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of the modules / units in the above-mentioned device embodiments are realized, for example Figure 2 The functions of the first generating module 201 , the first aligning module 202 , the output module 203 , the second generating module 204 , the second aligning module 205 and the third generating module 206 are shown.

[0091] Exemplarily, the computer program 32 of the automated test performance benchmark construction method mainly includes: dynamically generating region of interest (ROI) parameters; performing pixel cropping on the target area of ​​the production line according to the dynamically generated ROI parameters, generating a high-speed image data stream, and aligning the image acquisition time with the phase of the mechanical movement of the production line; selecting a matching visual inspection model based on the target features in the high-speed image data stream, and having the matching visual inspection model output defect detection results; embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a hybrid test data stream; performing multi-device timing alignment on the hybrid test data stream and the defect detection results, and encapsulating them into a standardized data set containing a unified time base; parsing the standardized data set, and calculating the multi-dimensional performance indicators of the visual inspection system by comparing the simulated defect true value labels with the defect detection results, and generating a test report containing parameter optimization suggestions. The computer program 32 can be divided into one or more modules / units, one or more of which are stored in the memory 31 and executed by the processor 30 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 can be divided into the functions of a first generation module 201, a first alignment module 202, an output module 203, a second generation module 204, a second alignment module 205 and a third generation module 206 (modules in the virtual device), and the specific functions of each module are as follows: the first generation module 201 is used to dynamically generate ROI parameters of the region of interest; the first alignment module 202 is used to perform pixel cropping on the target area of ​​the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition time with the mechanical motion phase of the production line; the output module 203 is used to perform pixel cropping on the target area of ​​the production line based on the target characteristics in the high-speed image data stream Features, select a matching visual inspection model, and the matching visual inspection model outputs the defect detection results; the second generation module 204 is used to embed the simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scene to generate a mixed test data stream; the second alignment module 205 is used to perform multi-device timing alignment on the mixed test data stream and the defect detection results, and encapsulate them into a standardized data set containing a unified time reference; the third generation module 206 is used to parse the standardized data set, and by comparing the simulated defect true value labels with the defect detection results, calculate the multi-dimensional performance indicators of the visual inspection system and generate a test report containing parameter optimization suggestions. The electronic device 3 may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will understand that, Figure 3 It is only an example of electronic device 3 and does not constitute a limitation of electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0092] The processor 30 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0093] The memory 31 can be an internal storage unit of the electronic device 3, such as the hard drive or memory of the electronic device 3. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 31 can include both the internal storage unit of the electronic device 3 and an external storage device. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or is about to be output.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0095] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0096] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0097] In the embodiments provided in this application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0098] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0099] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0100] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program of the automated test performance benchmark construction method can be stored in a storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments, namely, dynamically generating region of interest ROI parameters; performing pixel cropping on the target area of ​​the production line according to the dynamically generated ROI parameters, generating a high-speed image data stream, and aligning the image acquisition time with the mechanical motion phase of the production line; selecting a matching visual inspection model based on the target features in the high-speed image data stream, and outputting the defect detection results by the matching visual inspection model; embedding the simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream; performing multi-device timing alignment on the mixed test data stream and the defect detection results, and encapsulating them into a standardized data set containing a unified time base; parsing the standardized data set, and calculating the multi-dimensional performance indicators of the visual inspection system by comparing the simulated defect true value labels with the defect detection results, and generating a test report containing parameter optimization suggestions. Computer programs include computer program code, which may be in source code, object code, executable files, or some intermediate form. Storage media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content of storage media may be appropriately expanded or reduced based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, storage media do not include electric carrier signals or telecommunications signals.

[0101] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application. The specific implementation methods described above further explain the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present invention.

Claims

1. A method for constructing an automated test performance benchmark, characterized in that: The method comprises: Dynamically generate region of interest (ROI) parameters; Pixel cropping of the target area of ​​the production line is performed based on dynamically generated ROI parameters, generating a high-speed image data stream and aligning the image acquisition moment with the mechanical motion phase of the production line; Selecting a matching visual inspection model based on target features in the high-speed image data stream, and outputting a defect detection result by the matching visual inspection model; According to the defect distribution parameters of the preset test scenario, the simulated defect features are embedded in the high-speed image data stream to generate a hybrid test data stream; Performing multi-device timing alignment on the mixed test data stream and the defect detection results, and encapsulating them into a standardized data set containing a unified time reference; Parsing the standardized data set, calculating multi-dimensional performance indicators of the visual inspection system by comparing the true value labels of the simulated defects with the defect detection results, and generating a test report including parameter optimization suggestions; The dynamic generation of ROI parameters includes: calling a camera SDK instruction to limit the pixel area read by the sensor to generate an initial cropping range; performing displacement compensation on the initial cropping range based on a target motion speed prediction model, where the displacement compensation amount is calculated based on the pixel displacement speed of the target between consecutive frames; and iteratively reducing the ROI cropping range to the area covered by the target motion trajectory to increase the image acquisition frame rate to above a preset frame rate. The multi-device timing alignment of the mixed test data stream and the defect detection results includes: adding a device unified timestamp generated based on the Precision Time Protocol (PTP) to each frame image in the mixed test data stream and the corresponding defect detection result; detecting timing deviation by comparing the device unified timestamps of adjacent frames; treating the image frame whose timing deviation exceeds a threshold as an abnormal frame; interpolating and repairing the abnormal frame based on the pixel displacement vector calculated by the optical flow method, and synchronously correcting the associated defect detection result.

2. The method for constructing an automated test performance benchmark according to claim 1, wherein: The method of embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream includes: Use the trained generative adversarial network (GAN) to generate defect feature maps that match the material of the current production line; The defect feature map replaces the original image block in the high-speed image data stream according to a preset ratio.

3. The method for constructing an automated test performance benchmark according to claim 1, wherein: The aligning of the image acquisition moment with the mechanical motion phase of the production line comprises: Based on a unified clock reference calibrated by the Precision Time Protocol (PTP), it parses the device status code sent by the Programmable Logic Controller (PLC) and generates a camera trigger signal synchronized with the production line rhythm. Receive the real-time motion phase feedback signal from the mechanical controller and dynamically adjust the trigger signal generation time so that the deviation between image acquisition and mechanical motion phase is less than a preset threshold.

4. The method for constructing an automated test performance benchmark according to claim 3, wherein: The receiving of the real-time motion phase feedback signal from the mechanical controller and the dynamic adjustment of the trigger signal generation time so that the deviation between the image acquisition and the mechanical motion phase is less than a preset threshold comprises: By comparing the device's unified timestamp with the mechanical controller's clock, the clock deviation value of the image acquisition device is calculated; Dynamically adjust the depth of the image data buffer queue according to the clock deviation value to compensate for timing jitter caused by network delay; According to the encoder pulse signal sent by the mechanical controller, the timing deviation between the trigger signal and the target motion phase is calculated, and the next triggering moment is corrected.

5. The method for constructing an automated test performance benchmark according to claim 1, wherein: The interpolation and repair of the abnormal frame based on the pixel displacement vector calculated by the optical flow method includes: Constructing a motion trajectory equation of the target between consecutive frames based on the pixel displacement vector calculated by the optical flow method; Performing reverse compensation on the pixel coordinates of the ROI in the abnormal frame using the motion trajectory equation; Update the corresponding defect detection result position information according to the compensated image coordinates.

6. An automated test performance benchmark construction device, characterized in that: The device comprises: A first generation module is configured to dynamically generate ROI parameters, wherein the dynamic generation of ROI parameters includes: calling a camera SDK instruction to limit the pixel area read by the sensor to generate an initial cropping range; performing displacement compensation on the initial cropping range based on a target motion speed prediction model, wherein the displacement compensation amount is calculated based on the pixel displacement speed of the target between consecutive frames; and iteratively reducing the ROI cropping range to an area covered by the target motion trajectory to increase the image acquisition frame rate to above a preset frame rate. The first alignment module is used to perform pixel cropping of the target area of ​​the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition moment with the mechanical motion phase of the production line; an output module, configured to select a matching visual inspection model based on target features in the high-speed image data stream, and output defect detection results from the matching visual inspection model; A second generating module is configured to embed simulated defect features into the high-speed image data stream according to defect distribution parameters of a preset test scenario to generate a mixed test data stream; A second alignment module is configured to perform multi-device timing alignment on the mixed test data stream and the defect detection results, encapsulating them into a standardized data set containing a unified time reference. The multi-device timing alignment on the mixed test data stream and the defect detection results includes: appending a device unified timestamp generated based on the Precision Time Protocol (PTP) to each frame of the mixed test data stream and the corresponding defect detection result; detecting timing deviations by comparing the device unified timestamps of adjacent frames; treating image frames whose timing deviations exceed a threshold as abnormal frames; interpolating and repairing the abnormal frames based on pixel displacement vectors calculated using an optical flow method, and synchronously correcting the associated defect detection results; The third generation module is used to parse the standardized data set, calculate the multi-dimensional performance indicators of the visual inspection system by comparing the true value labels of the simulated defects with the defect detection results, and generate a test report containing parameter optimization suggestions.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images