Automatic test performance benchmark construction method and device and storage medium

Abstract

The invention relates to the field of industrial visual inspection, and provides an automatic test performance benchmark construction method which comprises the following steps: dynamically generating ROI (Region of Interest) parameters; performing pixel cutting on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and aligning the image acquisition moment with the mechanical motion phase of the production line; selecting a matched visual detection model based on target features in the high-speed image data stream, and outputting a defect detection result by the matched visual detection model; according to defect distribution parameters of a preset test scene, the simulation defect features are embedded into the high-speed image data stream, and a mixed test data stream is generated; performing multi-device time sequence alignment on the mixed test data stream and the defect detection result, and packaging into a standardized data set containing a unified time reference; and analyzing the standardized data set, comparing the simulated defect truth value label with a defect detection result, calculating a multi-dimensional performance index of the visual inspection system, and generating a test report containing parameter optimization suggestions.

Description

Technical Field

[0001] The present invention relates to the field of industrial vision inspection, and particularly to a method, device, and storage medium for constructing an automated test performance benchmark. Background Art

[0002] With the intelligent upgrade of industrial manufacturing, the vision inspection system has become a core link in the quality control of production lines. Especially in the field of precision manufacturing (such as semiconductors, auto parts, etc.), high-speed production lines have put forward higher requirements for the real-time performance, stability, and detection accuracy of vision inspection systems. Traditional industrial vision systems usually operate at a fixed frame rate and are difficult to meet the precise capture and detection requirements of dynamic targets in high-speed assembly line scenarios.

[0003] In the prior art, in order to improve the detection efficiency, solutions such as local image acquisition optimization or model switching detection are usually adopted. However, the above-mentioned prior art has defects such as insufficient adaptability to dynamic scenarios, poor controllability of test data, and lack of system coordination. Summary of the Invention

[0004] The present application provides a method, device, and storage medium for constructing an automated test performance benchmark, which can provide an efficient, reliable, and reproducible automated test benchmark for high-speed industrial vision inspection systems.

[0005] On the one hand, the present application provides a method for constructing an automated test performance benchmark, the method comprising: Dynamically generating Region of Interest (ROI) parameters; Performing pixel cropping on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and aligning the image acquisition time with the mechanical movement phase of the production line; Selecting a matching vision detection model based on the target features in the high-speed image data stream, and outputting a defect detection result by the matching vision detection model; Embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream; Performing multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulating them into a standardized data set containing a unified time reference; Parsing the standardized data set, calculating multi-dimensional performance indicators of the vision inspection system by comparing the true labels of simulated defects with the defect detection results, and generating a test report containing parameter optimization suggestions.

[0006] On the other hand, the present application provides an apparatus for constructing an automated test performance benchmark, the apparatus comprising: A first generation module, configured to dynamically generate Region of Interest (ROI) parameters; The first alignment module is used to perform pixel cropping on the target area of the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition time with the mechanical movement phase of the production line; The output module is used to select a matching visual detection model based on the target features in the high-speed image data stream, and output a defect detection result by the matching visual detection model; The second generation module is used to embed simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario, and generate a mixed test data stream; The second alignment module is used to perform multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulate them into a standardized data set containing a unified time reference; The third generation module is used to parse the standardized data set, calculate the multi-dimensional performance indicators of the visual detection system by comparing the simulated defect true value labels with the defect detection results, and generate a test report containing parameter optimization suggestions.

[0007] In a third aspect, the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the technical solution of the above-mentioned automated test performance benchmark construction method are implemented.

[0008] In a fourth aspect, the present application provides a storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the technical solution of the above-mentioned automated test performance benchmark construction method are implemented.

[0009] As can be seen from the technical solutions provided by the present application above, on the one hand, by dynamically generating ROI parameters to perform adaptive cropping on the target area and aligning the image acquisition time with the mechanical movement phase of the production line, while improving the image acquisition frame rate, it ensures the precise alignment of the acquisition time with the mechanical movement of the production line, solving the problem of the mismatch between image data and physical movement phase in high-speed scenarios, and providing high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate mixed test data, it not only retains the environmental characteristics of the real production line but also realizes the controllable adjustment of the defect type and distribution, providing a data basis with both authenticity and flexibility for the test process; thirdly, by analyzing the standardized data set to generate multi-dimensional indicators covering detection accuracy, processing efficiency, and resource consumption, and combining historical data to provide parameter optimization suggestions, it realizes the upgrade from single-result determination to systematic performance analysis, providing data support for the continuous optimization of the vision detection system. In summary, the technical solutions of the present application can effectively overcome the core problems in the prior art, such as poor adaptability to dynamic scenarios, insufficient authenticity of test data, and lack of multi-device collaboration, providing an efficient, reliable, and reproducible automated test benchmark for high-speed industrial vision detection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0011] Figure 1 is a flowchart of a method for constructing an automated test performance benchmark provided by an embodiment of the present application; Figure 2 is a schematic structural diagram of an apparatus for constructing an automated test performance benchmark provided by an embodiment of the present application; Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0013] In this specification, adjectives such as first and second are only used to distinguish one element or action from another element or action, and do not necessarily require or imply any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0014] In this specification, for ease of description, the dimensions of the various parts shown in the drawings are not drawn in accordance with the actual proportional relationship.

[0015] With the intelligent upgrade of industrial manufacturing, the vision inspection system has become the core link of production line quality control. Especially in the field of precision manufacturing (such as semiconductors, auto parts, etc.), high-speed production lines have put forward higher requirements for the real-time performance, stability, and detection accuracy of the vision inspection system. Traditional industrial vision systems usually operate at a fixed frame rate and are difficult to meet the precise capture and detection requirements of dynamic targets in high-speed assembly line scenarios. In the prior art, to improve the detection efficiency, the following two solutions are usually adopted: 1. Optimization of local image acquisition: By cropping a fixed region of interest (ROI) to reduce the amount of image processing, but the static ROI setting cannot adapt to the trajectory changes of moving targets, easily resulting in target loss or data redundancy; 2. Multi-model switching detection: Pre-load multiple vision models for different detection tasks, but there are problems such as excessive video memory occupation and significant switching delay during model switching, affecting the continuity of detection. In addition, existing test benchmark solutions mostly rely on artificially constructed test data, with insufficient authenticity of defect simulation, and lack of control over the timing consistency of multi-device collaborative work, resulting in test results being difficult to reproduce or compare horizontally.

[0016] The core defects of the above prior art are as follows: 1) Insufficient adaptability to dynamic scenarios, specifically manifested in that the static ROI and fixed model loading strategy are difficult to match the detection requirements of high-speed moving targets; 2) Poor controllability of test data, specifically manifested in that there are deviations between artificially constructed defects and the real production line environment, affecting the credibility of test results; 3) Lack of system coordination, specifically manifested in data misalignment caused by out-of-sync timing between multiple devices, and it is impossible to construct a unified performance evaluation benchmark.

[0017] In view of the above problems of the prior art, the present application proposes an automated test performance benchmark construction method, the flowchart of which is as shown in the appendix Figure 1 and mainly includes steps S101 to S106, which are described in detail as follows: Step S101: Dynamically generate parameters of the region of interest (ROI).

[0018] In the field of image processing, if the Region Of Interest (ROI) is fixed, on the one hand, it will inevitably contain a large number of irrelevant background pixels, which will increase the processing burden and cause the frame rate to be unable to break through the frame rate bottleneck, such as 150 FPS, and it cannot reach a high frame rate of 200 FPS+; on the other hand, when a fast-moving target is likely to move out of the fixed ROI range, subsequent detection fails and the system misjudgment rate increases. To solve the problems caused by the above fixed ROI, this application adopts a scheme of dynamically generating ROI parameters of the region of interest. Specifically, as an embodiment of this application, the dynamic generation of ROI parameters of the region of interest can be realized through steps S1011 to step S1013, and the details are as follows: Step S1011: Call the camera SDK instruction to limit the pixel area read by the sensor and generate an initial cropping range.

[0019] Step S1012: Perform displacement compensation on the initial cropping range based on the target motion speed prediction model, where the displacement compensation amount is calculated according to the pixel displacement speed of the target between consecutive frames.

[0020] Specifically, the displacement compensation of the initial cropping range based on the target motion speed prediction model can be realized through the following steps Sa1 to step Sa5: Step Sa1: Track the target through consecutive frames.

[0021] Specifically, the original image data of two adjacent frames (Frame N and Frame N+1) can be obtained, and then the position difference of the same target in the two frames can be identified through a feature matching algorithm (such as ORB feature point matching).

[0022] Step Sa2: Calculate the pixel displacement speed.

[0023] Specifically, after measuring the pixel displacement of the target in the X / Y axis direction and according to the frame interval time ( , determined by the reciprocal of the camera frame rate), calculate the target motion speed: horizontal speed , vertical speed .

[0024] Step Sa3: Predict the target motion speed based on the current motion speed of the target.

[0025] Specifically, establish a speed prediction model and use the Exponential Moving Average (EMA) method for short-term prediction. The formula is: , , where is the smoothing coefficient, with a value of 0.6 - 0.9; the input of the speed prediction model is the historical speed sequence , and the output is the predicted speed of the next acquisition period 。

[0026] Step Sa4: Calculate the ROI displacement compensation amount based on the predicted target motion speed.

[0027] Specifically, the compensation amount can be calculated according to the predicted speed: the horizontal compensation amount , the vertical compensation amount , and then, generate the new ROI center coordinates: , 。

[0028] Step Sa5: Dynamically adjust the ROI.

[0029] Specifically, it can be achieved by sending an instruction through the camera SDK to update the ROI position to the compensated coordinates, and then, verifying in real time whether the target is completely within the new ROI area. If there is a boundary overflow, expand the ROI range.

[0030] Step S1013: Iteratively narrow the cropping range of the ROI to the area covered by the target motion trajectory to increase the image acquisition frame rate to above the preset frame rate.

[0031] Specifically, iteratively narrowing the cropping range of the ROI to the area covered by the target motion trajectory to increase the image acquisition frame rate to above the preset frame rate can be achieved through the following steps Sb1 to Sb5:

[0032] Step Sb1: Model the motion trajectory of the target.

[0033] Specifically, it can be to record the motion trajectory coordinates of the target in consecutive N frames (such as 50 frames); use the least squares method to fit the motion trajectory equation of the target, such as a quadratic curve equation, etc.

[0034] Step Sb2: Determine the initial range of the ROI.

[0035] Specifically, it can be to calculate the maximum / minimum values of the trajectory in the X / Y axis directions according to the fitted motion trajectory equation of the target; set the initial ROI as a rectangular area covering the trajectory extreme points and expand it outward by a preset ratio, such as 5% as a safety margin.

[0036] Step Sb3: Iteratively optimize the ROI.

[0037] Specifically, the iterative optimization of the ROI mainly includes steps such as narrowing the ROI range, covering and verifying the ROI range, and dynamically adjusting the ROI range. Among them, narrowing the ROI range can specifically be reducing the ROI size along the X / Y axis direction at a fixed step (for example, reducing by 2% per step), while keeping the center of the ROI coincident with the center of the trajectory fitting curve; covering and verifying the ROI range can specifically be detecting whether the target is completely included in the ROI area for M consecutive frames (for example, 20 frames) within the narrowed ROI area, and using the background difference method to determine whether the target boundary touches the ROI edge; dynamically adjusting the ROI range can specifically be: if there is no contact at the ROI edge for a certain number of consecutive frames, such as more than 10 frames, continue to narrow the ROI range, and if it is detected that there is contact at the ROI edge, roll back to the previous valid ROI range and stop narrowing.

[0038] Step Sb4: Increase the image acquisition frame rate.

[0039] Specifically, the theoretical frame rate increase multiple can be calculated according to the ratio of the finally optimized ROI area (S_optimized) to the maximum supported area of the sensor (S_max), that is, the frame rate increase multiple K = S_max / S_optimized. Then, call the ROI mode interface of the camera SDK to set the acquisition frame rate to the reference frame rate × K (the reference frame rate is usually 50 FPS, and when K≥4, it can reach 200 FPS+).

[0040] Step Sb5: Handle exceptions.

[0041] That is, when the target motion trajectory mutates (for example, the speed change exceeds 15%), trigger the ROI reset mechanism and re-execute the above steps Sb1 to Sb3.

[0042] Through the above dynamic adjustment and iterative optimization of the ROI, the core contradiction of ROI position drift and frame rate limitation in high-speed scenarios is solved.

[0043] Step S102: Pixel-crop the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and align the image acquisition time with the mechanical motion phase of the production line.

[0044] Experiments show that if the deviation between the captured image and the actual movement of the robotic arm exceeds the preset value, such as 1 ms, in the high-speed conveyor belt scenario, the positioning error of defect detection can reach the millimeter level, and multiple devices (such as cameras, PLC devices, and mechanical controllers) cannot cooperate due to timing deviation, and the test benchmark is unreliable. Therefore, pixel cropping of the target area of the production line according to the dynamically generated ROI parameters, generating a high-speed image data stream, and aligning the image acquisition time with the mechanical movement phase of the production line can be pixel cropping of the target area of the production line according to the dynamically generated ROI parameters, generating a high-speed image data stream, and at the same time aligning the image acquisition time with the mechanical movement phase of the production line through timestamp synchronization technology.

[0045] Specifically, considering that the signal of the Programmable Logic Controller (PLC) is the core control source of the production line rhythm, direct parsing can avoid the additional delay introduced by external trigger signals. The Precision Time Protocol (PTP) can control the clock deviation within 1 μs, while the error of the NTP protocol reaches the millisecond level and cannot meet the requirements of high-speed scenarios. In the embodiments of this application, aligning the image acquisition time with the mechanical movement phase of the production line includes: based on the unified clock reference calibrated by PTP, parsing the device status code sent by the PLC to generate a camera trigger signal synchronized with the production line rhythm; receiving the real-time movement phase feedback signal of the mechanical controller, dynamically adjusting the generation time of the trigger signal to make the deviation between the image acquisition and the mechanical movement phase less than the preset threshold, for example, 50 μs. Among them, receiving the real-time movement phase feedback signal of the mechanical controller and dynamically adjusting the generation time of the trigger signal to make the deviation between the image acquisition and the mechanical movement phase less than the preset threshold can specifically be: calculating the clock deviation value of the image acquisition device by comparing the unified timestamp of the device with the clock of the mechanical controller; dynamically adjusting the depth of the image data buffer queue according to the clock deviation value of the acquisition device to compensate for the timing jitter caused by network delay; calculating the timing deviation between the trigger signal and the target movement phase according to the encoder pulse signal sent by the mechanical controller, and correcting the next trigger time.

[0046] As can be seen from step S101 to step S102 of the above embodiments, by adaptively cropping the target area according to the dynamically generated ROI parameters and combining the timestamp synchronization technology, while improving the image acquisition frame rate, ensuring the accurate alignment of the acquisition time with the mechanical movement of the production line, solving the problem of mismatch between image data and physical movement phase in high-speed scenarios, and providing high-timeliness and high-integrity input data for subsequent detection.

[0047] Step S103: Based on the target features in the high-speed image data stream, select a matching visual detection model, and output the defect detection result by the matching visual detection model.

[0048] On the one hand, it is difficult for a single model to cover multiple types of defects (for example, different models are required to detect scratches and foreign objects simultaneously), resulting in an increased missed detection rate. On the other hand, if all models are pre-loaded without selection and matching, the video memory occupancy will exceed the limit. As for completing the real-time loading of the model through the memory resource optimization strategy, it is because the detection is interrupted due to video memory overflow, and the test process cannot be continuously executed. Moreover, the increased time consumption of model switching is also likely to lead to the loss of real-time performance. Based on the fact that dynamic model loading and memory optimization are the core mechanisms to ensure detection accuracy, system stability, and real-time performance, the above-mentioned selection of a matching visual detection model based on the target features in the high-speed image data stream and the output of the defect detection result by the matching visual detection model can specifically be to select a matching visual detection model from the pre-stored model library based on the target features in the high-speed image data stream, complete the real-time loading of the model through the memory resource optimization strategy, and output the defect detection result by the matching visual detection model. Dynamically selecting and loading the visual detection model based on the real-time image features, combined with the memory resource optimization strategy, realizes the on-demand allocation of computing resources during the model switching process, avoids the problems of excessive video memory occupancy or switching delay caused by model pre-loading in the traditional solution, and ensures the continuity and stability of the detection task.

[0049] The memory resource optimization strategy in the above embodiment can be: real-time monitoring of the GPU video memory occupancy rate and the model inference time consumption; when the video memory occupancy rate exceeds the threshold, preferentially unload the models that are not called by subsequent detection tasks. Specifically, preferentially unloading the models that are not called by subsequent detection tasks can be to establish a loading priority according to the call frequency of the models in the test scenario; retain the memory cache for the models with high-frequency calls and perform a complete unloading for the models with low-frequency calls.

[0050] Step S104: Embed the simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream.

[0051] On the one hand, relying on real production line defects not only cannot control the defect distribution (for example, 100,000 images need to be collected in a scenario with a defect rate of 0.1%), resulting in low test efficiency, but also rare defects with a probability of occurrence not greater than 0.01% are difficult to obtain through natural data, and the test benchmark is incomplete. On the other hand, without true value labels, the detection accuracy (such as F1-Score) cannot be quantified, and only manual visual inspection can be relied on, resulting in low result credibility. Moreover, without the support of benchmark data, the quantitative relationship between model parameters and performance cannot be established. Therefore, in the embodiments of the present application, the above dilemmas can be solved by simulating defect feature embedding (that is, implanting artificially synthesized defects into the high-speed image data stream) and adding true value labels (providing benchmark information such as the position and type for the synthesized defects). Specifically, the simulated defect features can be embedded into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream.

[0052] As an embodiment of the present application, according to the defect distribution parameters of the preset test scenario, embedding simulated defect features into the high-speed image data stream to generate a mixed test data stream may be: using a trained generative adversarial network (GAN) to generate a defect feature map that matches the material of the current production line; replacing the original image blocks in the high-speed image data stream with the defect feature map according to a preset ratio. Specifically, replacing the original image blocks in the high-speed image data stream with the defect feature map according to a preset ratio may be: locating the original image blocks to be replaced in the high-speed image data stream according to the target position information in the defect detection result; based on the defect feature layer generated by the GAN, dynamically adjusting its transparency parameter according to the surface reflection characteristics of the material to make the simulated defect match the illumination condition of the original image. By embedding the simulated defect features into the original image data stream to generate mixed test data, both the environmental characteristics of the real production line are retained, and the controllable adjustment of the defect type and distribution is realized, providing a data basis with both authenticity and flexibility for the test process.

[0053] Step S105: Perform multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulate them into a standardized data set containing a unified time reference.

[0054] Since the images and detection results at the same moment may correspond to different physical positions (for example, when the timing deviation is 1 ms, the target displacement of the high-speed conveyor belt is 5 cm), thus mistakenly attributing the detection delay caused by the timing deviation to the model performance problem. Moreover, if the data such as images, detection results, and device states are stored separately, not only can they not be analyzed in association, but also the test process cannot be repeated without a unified meta-information (such as time stamps, device parameters). Therefore, in order to ensure data consistency, reproducibility, and cross-system comparability, multi-device timing alignment can be achieved by unifying the time references of the image stream, detection results, and mechanical motion data. At the same time, the standardized data set is encapsulated by integrating the data and attaching unified meta-information. Specifically, in the embodiment of the present application, multi-device timing alignment can be performed on the mixed test data stream and the defect detection result, and encapsulated into a standardized data set containing a unified time reference.

[0055] As an embodiment of the present application, multi-device timing alignment of the mixed test data stream and the defect detection results can be as follows: Attach device unified timestamps generated based on the Precision Time Protocol (PTP) to each frame of the image in the mixed test data stream and the corresponding defect detection results; Detect image frames or detection result data packets with timing deviations exceeding the threshold by comparing the device unified timestamps of adjacent frames, interpolate and repair the abnormal frames based on the pixel displacement vectors calculated by the optical flow method, and synchronously correct the associated defect detection results. Among them, the abnormal frames are the image frames with timing deviations exceeding the threshold, and the specific method of interpolating and repairing the abnormal frames based on the pixel displacement vectors calculated by the optical flow method can be: Construct the motion trajectory equation of the target between consecutive frames according to the pixel displacement vectors calculated by the optical flow method; Use the motion trajectory equation of the target between consecutive frames to perform reverse compensation on the pixel coordinates of the Region of Interest (ROI) in the abnormal frame; Update the position information of the corresponding defect detection result according to the compensated image coordinates. As for synchronously correcting the associated defect detection results, it can be to perform linear interpolation compensation on the timestamps and coordinates of the abnormal detection result data packets (i.e., the detection result data packets with timing deviations exceeding the threshold) according to the timestamps of the repaired image frames.

[0056] As can be seen from the above embodiments, timing alignment and encapsulation of the test data based on a unified time reference eliminate the influence of clock deviations in multi-device collaborative work, ensure the global consistency and reproducibility of the test data, and provide a reliable basis for cross-system performance comparison.

[0057] Step S106: Parse the standardized data set, calculate the multi-dimensional performance indicators of the visual detection system by comparing the simulated defect ground truth labels with the defect detection results, and generate a test report containing parameter optimization suggestions.

[0058] To distinguish between the cases where the model misses detections and the data itself has no defects, and to locate the performance bottlenecks (such as insufficient video memory due to increased frame rate or model calculation timeout), for objective evaluation, efficient tuning, and continuous system improvement, a standardized dataset can be parsed. By comparing the simulated defect ground truth labels with the defect detection results, multi-dimensional performance metrics of the visual detection system can be calculated and a test report containing parameter optimization suggestions can be generated. Among them, the multi-dimensional performance metrics of the visual detection system can be generated in the following ways: within a preset frame rate gradient range, by continuously adjusting the acquisition frame rate and recording the defect detection results at corresponding moments, based on the simulated defect ground truth labels embedded in the mixed test data stream, a quantization relationship curve reflecting the change of F1-Score with the frame rate can be generated; during the model switching process, the video memory occupancy data is collected in real time, and the timing characteristics of the peak video memory and the time required to restore stability are extracted. As for the parameter optimization suggestions, they can be specifically generated in the following way: a quantization relationship matrix between the model hyperparameters, detection accuracy, and frame rate stability is established, where the detection accuracy is calculated by comparing the defect detection results with the ground truth labels; then, a genetic algorithm is used to search for the Pareto optimal parameter combination in the relationship matrix. By parsing the standardized dataset to generate multi-dimensional metrics covering detection accuracy, processing efficiency, and resource consumption, and providing parameter optimization suggestions in combination with historical data, the upgrade from single result determination to systematic performance analysis is realized, providing data support for the continuous optimization of the visual detection system.

[0059] From the above-mentioned method for constructing an automated test performance benchmark in the attached Figure 1 example, on the one hand, by dynamically generating ROI parameters to adaptively crop the target area, combined with aligning the image acquisition time with the mechanical movement phase of the production line, while increasing the image acquisition frame rate, ensuring the precise alignment of the acquisition time with the mechanical movement of the production line, the problem of image data and physical movement phase mismatch in high-speed scenarios is solved, providing high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate mixed test data, both the environmental characteristics of the real production line are retained, and the controllable adjustment of defect types and distributions is achieved, providing a data basis with both authenticity and flexibility for the test process; thirdly, by parsing the standardized dataset to generate multi-dimensional metrics covering detection accuracy, processing efficiency, and resource consumption, and providing parameter optimization suggestions in combination with historical data, the upgrade from single result determination to systematic performance analysis is realized, providing data support for the continuous optimization of the visual detection system. In summary, the technical solution of this application can effectively overcome the core problems in the prior art such as poor adaptability to dynamic scenarios, insufficient authenticity of test data, and lack of multi-device collaboration, providing an efficient, reliable, and reproducible automated test benchmark for the high-speed industrial visual detection system.

[0060] Please refer to the attached Figure 2, is an automated test performance benchmark construction device provided in an embodiment of the present application, the device may include a first generation module 201, a first alignment module 202, an output module 203, a second generation module 204, a second alignment module 205, and a third generation module 206, as detailed below: The first generating module 201 is used to dynamically generate the parameters of the region of interest (ROI); The first alignment module 202 is used to perform pixel cropping on the target area of the production line according to the dynamically generated ROI parameters, generate a high-speed image data stream, and align the image acquisition time with the mechanical motion phase of the production line; An output module 203 is configured to select a matching visual inspection model based on target features in the high-speed image data stream, and output defect detection results from the matching visual inspection model; The second generating module 204 is configured to embed simulated defect features into the high-speed image data stream according to the defect distribution parameters of the preset test scenario to generate a mixed test data stream; The second alignment module 205 is used to perform multi-device timing alignment on the mixed test data stream and defect detection results, and encapsulate them into a standardized data set containing a unified time reference; The third generation module 206 is used to parse the standardized data set, calculate the multi-dimensional performance indicators of the visual inspection system by comparing the true value labels of the simulated defects with the defect detection results, and generate a test report including parameter optimization suggestions.

[0061] From the above attached Figure 2 The example of the automated test performance benchmark construction device shows that, on the one hand, by dynamically generating ROI parameters to adaptively crop the target area, combining the image acquisition time with the phase alignment of the mechanical movement of the production line, while improving the image acquisition frame rate, it ensures the precise alignment of the acquisition time with the mechanical movement of the production line, solves the problem of mismatch between image data and physical movement phase in high-speed scenes, and provides high-timeliness and high-integrity input data for subsequent detection; on the other hand, by embedding simulated defect features into the original high-speed image data stream to generate hybrid test data, it not only retains the environmental characteristics of the real production line, but also realizes the controllable adjustment of defect types and distribution, providing a data foundation with both authenticity and flexibility for the test process; thirdly, by parsing the standardized data set to generate multi-dimensional indicators covering detection accuracy, processing efficiency and resource consumption, and combining historical data to provide parameter optimization suggestions, it realizes the upgrade from single result judgment to systematic performance analysis, and provides data support for the continuous optimization of the visual inspection system. In summary, the technical solution of the present application can effectively overcome the core problems of poor adaptability to dynamic scenes, insufficient authenticity of test data, and lack of multi-device collaboration in the prior art, and provides an efficient, reliable and reproducible automated test benchmark for high-speed industrial visual inspection systems.

[0062] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 3 shown, the electronic device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for an automated test performance benchmark construction method. When the processor 30 executes the computer program 32, it implements the steps in the above-mentioned embodiment of the automated test performance benchmark construction method, such as Figure 1 the steps S101 to S106 shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the above-mentioned device embodiments, such as Figure 2 the functions of the first generation module 201, the first alignment module 202, the output module 203, the second generation module 204, the second alignment module 205, and the third generation module 206 shown.

[0063] Exemplarily, the computer program 32 of the automated test performance benchmark construction method mainly includes: dynamically generating region of interest (ROI) parameters; performing pixel cropping on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and aligning the image acquisition time with the mechanical motion phase of the production line; selecting a matching visual detection model based on the target features in the high-speed image data stream, and outputting a defect detection result by the matching visual detection model; embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream; performing multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulating them into a standardized data set including a unified time reference; parsing the standardized data set, calculating multi-dimensional performance indicators of the visual detection system by comparing the simulated defect true value label with the defect detection result, and generating a test report including parameter optimization suggestions. The computer program 32 can be divided into one or more modules / units, and one or more modules / units are stored in the memory 31 and executed by the processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 can be divided into the functions of a first generation module 201, a first alignment module 202, an output module 203, a second generation module 204, a second alignment module 205, and a third generation module 206 (modules in the virtual device). The specific functions of each module are as follows: The first generation module 201 is used to dynamically generate region of interest (ROI) parameters; the first alignment module 202 is used to perform pixel cropping on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and align the image acquisition time with the mechanical motion phase of the production line; the output module 203 is used to select a matching visual detection model based on the target features in the high-speed image data stream, and output a defect detection result by the matching visual detection model; the second generation module 204 is used to embed simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream; the second alignment module 205 is used to perform multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulate them into a standardized data set including a unified time reference; the third generation module 206 is used to parse the standardized data set, calculate multi-dimensional performance indicators of the visual detection system by comparing the simulated defect true value label with the defect detection result, and generate a test report including parameter optimization suggestions. The electronic device 3 may include but is not limited to the processor 30 and the memory 31. Those skilled in the art can understand that Figure 3 merely examples of the electronic device 3, which do not constitute a limitation on the electronic device 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0064] The so-called processor 30 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0065] The memory 31 may be an internal storage unit of the electronic device 3, such as the hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device 3. Further, the memory 31 may also include both the internal storage unit of the electronic device 3 and the external storage device. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0066] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of this application. The specific working processes of the units and modules in the above-mentioned device can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0067] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0068] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0069] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical or other forms.

[0070] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0072] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present application, it can also be completed by instructing relevant hardware through a computer program. The computer program of the automated test performance benchmark construction method can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned various method embodiments, that is, dynamically generate Region of Interest (ROI) parameters; perform pixel cropping on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and align the image acquisition time with the mechanical movement phase of the production line; based on the target features in the high-speed image data stream, select a matching visual detection model, and output a defect detection result by the matching visual detection model; according to the defect distribution parameters of the preset test scenario, embed simulated defect features into the high-speed image data stream to generate a mixed test data stream; perform multi-device timing alignment on the mixed test data stream and the defect detection result, and package them into a standardized data set containing a unified time reference; parse the standardized data set, calculate the multi-dimensional performance indicators of the visual detection system and generate a test report containing parameter optimization suggestions by comparing the simulated defect true value label with the defect detection result. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The storage medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electrical carrier signals and telecommunication signals.

[0073] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application. The specific implementation manners described above have further elaborated on the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above is only the specific implementation manner of the present application and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should all be included in the protection scope of the present invention.

Claims

1. An automated test performance benchmark construction method, characterized in that, The method includes: Dynamically generating Region of Interest (ROI) parameters; Performing pixel cropping on the target area of the production line according to the dynamically generated ROI parameters to generate a high-speed image data stream, and aligning the image acquisition time with the mechanical motion phase of the production line; Selecting a matching visual detection model based on the target features in the high-speed image data stream, and outputting a defect detection result by the matching visual detection model; Embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream; Performing multi-device timing alignment on the mixed test data stream and the defect detection result, and encapsulating them into a standardized data set including a unified time reference; Parsing the standardized data set, calculating multi-dimensional performance indicators of the visual detection system by comparing the simulated defect ground truth labels with the defect detection results, and generating a test report including parameter optimization suggestions.

2. The automated test performance benchmark construction method according to claim 1, wherein The dynamically generating Region of Interest (ROI) parameters includes: Invoking the camera SDK instruction to limit the pixel area read by the sensor to generate an initial cropping range; Performing displacement compensation on the initial cropping range based on a target motion speed prediction model, and calculating the displacement compensation amount according to the pixel displacement speed of the target between consecutive frames; Iteratively reducing the cropping range of the ROI to the area covered by the target motion trajectory to increase the image acquisition frame rate to above a preset frame rate.

3. The automated test performance benchmark construction method according to claim 1, characterized in that The embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream includes: Using a trained Generative Adversarial Network (GAN) to generate a defect feature map matching the current production line material; Replacing the original image blocks in the high-speed image data stream with the defect feature map at a preset ratio.

4. The automated test performance benchmark construction method according to claim 1, wherein The aligning the image acquisition time with the mechanical motion phase of the production line includes: Based on the unified clock reference calibrated by the Precision Time Protocol (PTP), parsing the device status code sent by the Programmable Logic Controller (PLC) to generate a camera trigger signal synchronized with the production line beat; Receiving the real-time motion phase feedback signal of the mechanical controller, and dynamically adjusting the generation time of the trigger signal to make the deviation between the image acquisition and the mechanical motion phase less than a preset threshold.

5. The automated test performance benchmark construction method according to claim 4, wherein The receiving the real-time motion phase feedback signal of the mechanical controller, and dynamically adjusting the generation time of the trigger signal to make the deviation between the image acquisition and the mechanical motion phase less than a preset threshold includes: Calculating the clock deviation value of the image acquisition device by comparing the device unified timestamp with the mechanical controller clock; Dynamically adjusting the depth of the image data buffer queue according to the clock deviation value to compensate for the timing jitter caused by network latency; Calculating the timing deviation between the trigger signal and the target motion phase according to the encoder pulse signal sent by the mechanical controller, and correcting the next trigger time.

6. The method for constructing an automated test performance benchmark according to claim 1, wherein The performing multi-device timing alignment on the mixed test data stream and the defect detection result includes: Attaching a device unified timestamp generated based on the Precision Time Protocol (PTP) to each frame of the image in the mixed test data stream and the corresponding defect detection result; By comparing the device unified timestamps of adjacent frames, detect image frames or detection result data packets whose timing deviation exceeds the threshold, interpolate and repair abnormal frames based on the pixel displacement vector calculated by the optical flow method, and synchronously correct the associated defect detection results.

7. The automated test performance benchmark construction method according to claim 6, characterized in that, The interpolating and repairing of the abnormal frames includes: Construct a motion trajectory equation of the target between consecutive frames according to the pixel displacement vector calculated by the optical flow method; Use the motion trajectory equation to reversely compensate the pixel coordinates of the ROI in the abnormal frame; Update the position information of the corresponding defect detection result according to the compensated image coordinates.

8. An automated test performance benchmark construction device, characterized in that, The device includes: A first generation module for dynamically generating Region of Interest (ROI) parameters; A first alignment module for pixel cropping the target area of the production line according to the dynamically generated ROI parameters, generating a high-speed image data stream, and aligning the image acquisition time with the mechanical motion phase of the production line; An output module for selecting a matching visual detection model based on the target features in the high-speed image data stream, and outputting defect detection results by the matching visual detection model; A second generation module for embedding simulated defect features into the high-speed image data stream according to the defect distribution parameters of a preset test scenario to generate a mixed test data stream; A second alignment module for performing multi-device timing alignment on the mixed test data stream and the defect detection results, and encapsulating them into a standardized data set containing a unified time reference; A third generation module for parsing the standardized data set, calculating multi-dimensional performance indicators of the visual detection system by comparing the simulated defect ground truth labels with the defect detection results, and generating a test report containing parameter optimization suggestions.

9. An electronic device, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images