Industrial high-speed moving object blur-free imaging detection method, system, medium, and device

Abstract

The application discloses an industrial high-speed moving object defogging imaging detection method, system, medium and equipment. The method comprises the following steps: obtaining original event flow collected by a camera to obtain a standard event flow through preprocessing; extracting space-time features of the standard event flow to obtain a state vector; using a trained strategy network based on the state vector to make a decision to determine whether to output a reconstruction instruction, wherein the reconstruction instruction comprises a time window and a reconstruction mode; obtaining a window event set based on the time window, estimating the motion speed of each event based on the window event set, projecting the events to obtain a compensation event set, and generating a clear image based on the reconstruction mode and the compensation event set, and sending the clear image to a preset detection module to output a detection result. The application fundamentally eliminates motion blur, outputs a high-definition image, realizes low-delay real-time detection, adapts to complex illumination, and has a lower overall cost than a traditional high-speed imaging and multi-sensor fusion scheme.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a method, system, medium, and device for blur-free imaging detection of high-speed moving objects in industrial applications. Background Technology

[0002] In modern industrial manufacturing, such as high-speed surface mount technology (SMT), precision assembly, and online quality inspection, real-time, clear imaging and accurate detection of high-speed moving objects (such as electronic components, medicine bottles, and mechanical parts) on the production line are the core links to ensure product quality and production efficiency. Traditional machine vision systems generally use frame-based CMOS or CCD cameras. The working principle of these cameras is to perform integral imaging of the scene within a set exposure time to generate a complete frame image.

[0003] However, when there is high-speed relative motion between the object being measured and the camera, the projection of the object on the imaging plane will be displaced within a fixed exposure time, resulting in motion blur in the captured image. Motion blur will severely lose the image's detailed information, blurring key features such as edges and textures, causing a significant decrease in the accuracy and robustness of subsequent detection algorithms (such as edge detection, size measurement, and defect identification), or even complete failure.

[0004] To address this issue, the mainstream technical solutions are as follows: Solution 1 is an imaging method based on high-speed frame cameras and stroboscopic illumination. This is one of the most widely used technologies in industrial high-speed visual inspection. Its core idea is to "freeze" the instantaneous state of a moving object by shortening the exposure time or using a high-intensity, short-pulse stroboscopic light source. Solution 2 is a detection method based on traditional frame cameras and motion compensation algorithms. This solution does not directly eliminate blur during the imaging process but attempts to recover a clear image in the post-processing stage. It is usually combined with mechanical encoders or position sensors to accurately obtain the motion trajectory information of the object. Solution 3 is an imaging detection system based on multi-image sensor fusion. This is a more complex hardware system that combines different types of sensors to compensate for the shortcomings of a single sensor.

[0005] Although the aforementioned existing technologies can alleviate motion blur problems under certain conditions, they all have inherent technical defects and cannot simultaneously meet the demands of modern industry for high speed, high precision, high reliability, and low cost. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, medium, and device for blur-free imaging detection of high-speed moving objects in industrial applications, which solves the problems of motion blur, large data redundancy, poor real-time performance of traditional frame cameras, strong reliance on prior knowledge and easy failure of motion compensation algorithms, and complexity and high cost of multi-sensor fusion systems.

[0007] The first aspect of this invention provides a method for blur-free imaging detection of high-speed moving objects in industrial applications, comprising the following steps:

[0008] The raw event stream captured by the camera is preprocessed to obtain a standardized event stream;

[0009] Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate.

[0010] Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode.

[0011] Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set;

[0012] Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

[0013] In this solution, the preprocessing of the raw event stream acquired by the camera to obtain a standardized event stream specifically includes:

[0014] The raw event stream is obtained by acquiring the light intensity changes of each pixel of a high-speed moving object captured by the camera. An event is output when the absolute value of the logarithmic change in light intensity exceeds a preset threshold. ,in, It is the first The pixel coordinates of each event. It is the first The timestamp of each event It is the first The polarity of an event;

[0015] The original event stream is preprocessed to obtain the canonical event stream, wherein the preprocessing mechanism includes spatiotemporal domain inspection, denoising, timestamp alignment, and coordinate normalization.

[0016] In this scheme, the step of extracting the spatiotemporal features of the standardized event stream to obtain the state vector specifically includes:

[0017] The event density is calculated using the following formula:

[0018] ;

[0019] in, For event density, For time intervals, Time interval The total number of events within;

[0020] The spatial entropy is calculated using the following formula:

[0021] ;

[0022] in, For spatial entropy, For pixel coordinates, For events in pixels The probability of this happening;

[0023] The average timestamp transformation rate is obtained by calculating the activity level of the event stream over time. ;

[0024] The state vector is obtained based on the event density, the spatial entropy, and the average timestamp change rate. .

[0025] In this scheme, the step of using the trained policy network based on the state vector to make a decision on whether to output a reconstruction instruction specifically includes:

[0026] Use a pre-trained policy network Based on the current state vector Select Action The action space contains three key decisions. ,in,

[0027] The time window is used to determine the length of the reconstruction time window. and time offset relative to the current time ,in, ;

[0028] This is the reconstruction mode, used to determine whether to prioritize accuracy or speed.

[0029] To trigger reconstruction, it determines whether enough event information has been collected to construct a clear image in order to output a reconstruction instruction, which includes... .

[0030] In this solution, the step of reconstructing the dynamic event window based on the time window to obtain a window event set, and estimating the motion speed of each event based on the window event set, specifically includes:

[0031] Based on the time window in the reconstruction instruction, extract all events within the current window from the event stream buffer to obtain the window event set;

[0032] Project each event in the window event set onto a common reference moment. ;

[0033] The spatiotemporal distribution of events within a local neighborhood is analyzed to estimate the motion velocity corresponding to the event on the pixel. The calculation formula is as follows:

[0034] ;

[0035] in, Let be the energy function. For the speed of movement, It is the first The pixel coordinates of each event. For pixels The spatial gradient vector, It is the first The timestamp of each event It is the first The polarity of an event, It is the square norm.

[0036] In this scheme, the process of projecting events based on the motion speed to obtain a compensation event set, and generating a clear image based on the reconstruction mode and the compensation event set, and then sending the clear image into a preset detection module for detection to output the detection result, specifically includes:

[0037] After obtaining the motion speed, each event is compensated to the reference time. The pixel position is compensated for pixel coordinates. To obtain the compensation event set;

[0038] Based on the reconstruction mode and the compensation event, Gaussian kernel density estimation is used to generate the sharp image, and the calculation formula is as follows:

[0039] ;

[0040] in, For a clear image, For pixel coordinates, To compensate for pixel coordinates, It is the first The polarity of an event, It is the smoothing parameter corresponding to the reconstruction mode;

[0041] The clear image is sent to the detection module for detection to output the detection result. The detection module includes a convolutional neural network, and the detection content includes running on an embedded device to complete the classification, positioning and defect detection of the workpiece.

[0042] A second aspect of the present invention also provides a blur-free imaging detection system for high-speed moving objects in industry, comprising a memory and a processor. The memory includes a program for a blur-free imaging detection method for high-speed moving objects in industry. When the processor executes the program for a blur-free imaging detection method for high-speed moving objects in industry, it performs the following steps:

[0043] The raw event stream captured by the camera is preprocessed to obtain a standardized event stream;

[0044] Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate.

[0045] Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode.

[0046] Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set;

[0047] Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

[0048] A third aspect of the present invention provides a computer-readable storage medium comprising a machine program for a method of detecting a high-speed moving object in an industrial environment without blurring, wherein when the program is executed by a processor, it implements the steps of the method of detecting a high-speed moving object in an industrial environment without blurring as described in any of the preceding claims.

[0049] The fourth aspect of the present invention provides a computer program product, which includes computer program code, and when the computer program code is run on a computer, the computer implements the steps of the blur-free imaging detection method for high-speed moving objects in industrial applications as described in any of the preceding claims.

[0050] The fifth aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the steps of the blur-free imaging detection method for high-speed moving objects in industry as described in any of the preceding claims.

[0051] The present invention discloses a method, system, medium, and device for fuzz-free imaging detection of high-speed moving objects in industrial applications, which has the following beneficial effects:

[0052] 1. This invention fundamentally solves the problem of motion blur. Traditional methods passively "reduce" blur by shortening exposure time, while this invention actively "eliminates" blur by using agent-driven dynamic time window selection and event reconstruction based on motion compensation. Utilizing the microsecond-level temporal resolution of the event camera, the invention "reverses" events to their original positions during the reconstruction stage using a precise motion model. The resulting image is theoretically free of motion blur, making it particularly suitable for objects with high-speed movement and complex trajectories.

[0053] 2. Data efficiency and real-time performance: The inherent characteristic of event cameras, which "only record changes", results in a much smaller output data volume compared to high-speed frame cameras with the same resolution. This fundamentally solves the problem of data redundancy. Combined with a lightweight intelligent agent algorithm, this invention does not need to process every frame of the image. It only triggers reconstruction when necessary, which greatly reduces the pressure of data transmission, storage and computation, making it possible to achieve high-speed real-time detection on a low-power embedded platform.

[0054] 3. High dynamic range and adaptability: The dynamic range of event cameras typically exceeds 120dB, far higher than the 60dB of traditional CMOS cameras. This allows the present invention to adapt to both strong and weak light areas on the production line without the need for complex lighting adjustments. At the same time, the introduction of intelligent agents can adaptively adjust the reconstruction strategy (such as the time window length) according to real-time changes in object movement speed, texture characteristics, etc., and has strong environmental adaptability and robustness.

[0055] 4. Eliminating the dependence on precise external synchronization, this invention does not rely on external devices such as photoelectric sensors to provide trigger signals. Its reconfiguration timing is entirely determined by the intelligent agent based on the event flow itself. This asynchronous, self-driven mode simplifies the structure, reduces the complexity of installation and debugging and the requirement for precise synchronization, and improves reliability and ease of use.

[0056] 5. Reduced overall system cost: Compared to Solution 1, which requires expensive high-speed cameras and complex strobe lighting systems, and Solution 3, which requires multi-sensor fusion, this invention mainly relies on an event camera and an embedded computing platform, and its total cost of ownership has a significant advantage in high-speed, high-precision detection scenarios. Attached Figure Description

[0057] Figure 1 A flowchart illustrating a method for fuzz-free imaging detection of high-speed moving objects in industrial applications according to the present invention is shown.

[0058] Figure 2 This diagram illustrates motion supplementation reconstruction in traditional methods.

[0059] Figure 3 This diagram illustrates motion supplementation and reconstruction in a blur-free imaging detection method for high-speed moving objects in industrial applications according to the present invention.

[0060] Figure 4 This diagram illustrates a comparison of the effects of the blur-free imaging detection method for high-speed moving objects in industrial applications according to the present invention.

[0061] Figure 5 A block diagram of a blur-free imaging detection system for high-speed moving objects in industrial applications according to the present invention is shown. Detailed Implementation

[0062] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0063] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0064] Specifically, the implementation of Scheme 1 is as follows: a high-speed CMOS camera with an extremely short exposure time (such as microseconds) is used, along with a high-brightness LED strobe light source that is synchronized with the camera exposure. When a moving object passes through a preset detection station, a trigger signal is activated to start the camera exposure. At the same time, the light source provides brief illumination with extremely high light intensity. Because the exposure time is extremely short, the displacement of the object during the exposure period is less than one pixel, thus in principle, an image that is nearly blur-free can be obtained.

[0065] Specifically, the implementation of Scheme 2 is as follows: After the camera captures an image containing motion blur, a point spread function is constructed using known motion information (such as direction of motion, velocity, and acceleration). Then, a deconvolution algorithm (such as the Lucy-Richardson algorithm, Wiener filtering, etc.) is used to restore the blurred image in order to obtain a clear image. The technical process can be summarized as: acquiring the blurred image... ->Estimating the point spread function of motion blur -> Apply the deconvolution algorithm to obtain the original sharp image The relation satisfies , Represents noise;

[0066] Specifically, in application, Scheme 3 combines, for example, a low-resolution but high-frame-rate event camera or high-speed camera with a high-resolution but low-frame-rate traditional camera. The low-resolution, high-frame-rate sensor is used to track the motion state and general outline of objects in real time, while the high-resolution sensor triggers capture during brief intervals when the motion state is stable. Through spatial registration and time synchronization technology, the information from both is fused to obtain a clear high-resolution image. This scheme has been applied in scientific research and high-end manufacturing, but its system complexity is high, its cost is expensive, and its data fusion algorithm is complex and its real-time performance is limited.

[0067] Through in-depth analysis of the above-mentioned existing technical solutions, several key drawbacks of existing technologies in solving frame-averaged images can be clearly summarized:

[0068] 1. Regarding the disadvantages of Scheme 1 (high-speed frame camera and stroboscopic illumination): (1) Data redundancy and real-time bottleneck: The high-speed camera captures a large amount of image data at a fixed frame rate. Even in the gaps where no object passes by, a data stream will be generated, resulting in huge data redundancy. In order to process this data, the subsequent transmission, storage and computing units must have extremely high bandwidth and processing capabilities, which not only increases the system cost, but also brings a bottleneck to real-time processing; (2) Contradiction between exposure time and signal-to-noise ratio: In order to eliminate blur, the exposure time needs to be shortened as much as possible, but this will lead to a sharp reduction in the number of photons collected by each pixel, which will reduce the image signal-to-noise ratio. Especially when there is insufficient light or when shooting low-contrast objects, the image quality will be severely degraded, and a large number of noises will appear, affecting the detection accuracy; (3) High synchronization complexity. Among them, stroboscopic illumination and camera exposure need to be synchronized with high precision. Any small timing deviation will lead to illumination failure or imaging failure. In addition, the response delay of the trigger detection system (such as photoelectric sensor) of high-speed moving objects will also introduce uncertainty.

[0069] 2. Regarding the shortcomings of Scheme 2 (motion compensation algorithm): (1) Ill-conditioned problem: Blind deconvolution or non-blind deconvolution of an image is itself an ill-conditioned inverse problem. The solution process is extremely sensitive to noise and estimation errors of precise prior knowledge. Small model errors or noise amplification will cause ringing effects, artifacts or information loss in the restored image, and the restoration results are often unreliable; (2) Prior knowledge dependence: This method relies heavily on precise prior knowledge of the trajectory of the object. In actual production, the motion posture of the object may have random jitter, rotation or speed fluctuation. These non-ideal motion states are difficult to model accurately, resulting in inaccurate estimation of precise prior knowledge, thus causing the restoration algorithm to fail; (3) Large computational overhead: Deconvolution algorithms usually involve iterative calculations, which are computationally intensive and difficult to implement in real time on industrial embedded platforms. They usually require offline processing or rely on high-performance computing servers.

[0070] 3. Regarding the disadvantages of Scheme 3 (multi-sensor fusion): (1) High system complexity and cost: It requires at least two different types of imaging sensors and complex synchronization, calibration and fusion hardware / software. The system integration is difficult, the hardware cost is high, and the maintenance is complicated; (2) Spatiotemporal registration error: The spatial field of view difference and time delay between different sensors need to be accurately calibrated and compensated. Any tiny registration error will introduce new artifacts such as ghosting and misalignment in the fused image, affecting the final detection result.

[0071] To address these shortcomings, the present invention aims to provide a blur-free imaging detection method, system, medium, and device for high-speed moving objects in industrial applications. Specifically, the invention leverages the asynchronous, high temporal resolution, and low data redundancy characteristics of event cameras to fundamentally overcome the motion blur problem of traditional frame cameras. Combined with intelligent agent algorithms, it achieves an adaptive, real-time, high dynamic range, clear imaging and accurate detection method. This solves the problems of poor real-time performance, strong dependence on prior motion, system complexity, and high cost in existing technologies, providing a highly efficient, reliable, and low-cost visual inspection solution for high-speed industrial production lines. Therefore, the core innovation and key protection points are:

[0072] 1. Event-driven agent-based adaptive perception and reconstruction triggering mechanism: This is the core innovation of this invention. Specifically, it constructs an agent based on reinforcement learning, using the spatiotemporal statistical features of the event flow as the state space, and autonomously decides when and with what parameters (time window length, reconstruction mode) to reconstruct the image by learning the optimal policy. This mechanism transforms the traditional "synchronous triggering, fixed frame rate" imaging mode into a "data-driven, adaptive decision-making" intelligent imaging mode, which is the key to achieving data efficiency and real-time performance. The protection points are the agent's state space construction method, reward function design, and its closed-loop control method with the dynamic event window reconstruction module.

[0073] 2. Motion-Compensated Dynamic Event Window Blur-Free Reconstruction Algorithm: This invention proposes a method for reconstructing a clear image by performing precise motion compensation on each event within a dynamically selected time window. This is fundamentally different from simply accumulating events or using a fixed time window for reconstruction. Its key technical point lies in using local event stream data to estimate the motion velocity field in real time, and projecting events at different times onto a unified reference plane accordingly. This fundamentally eliminates image blur caused by object motion. The key protection lies in the specific steps of this motion compensation reconstruction algorithm, particularly the method of estimating the motion velocity field using event stream data, and the method of generating a grayscale image from the compensated events through kernel density estimation.

[0074] 3. Intelligent mapping relationship between event flow spatiotemporal features and reconstruction parameters: This invention establishes a nonlinear mapping relationship between event flow dynamic features (such as event density and spatial entropy) and optimal reconstruction parameters (window length and offset) through an intelligent agent. This mapping relationship is not fixed, but is continuously optimized through training, enabling the invention to automatically adapt to objects with different speeds and textures. The key point is this learning-based dynamic mapping mechanism from event flow features to reconstruction parameters.

[0075] 4. Integrated closed-loop system architecture of "perception-decision-reconstruction-detection": This invention integrates event perception, intelligent decision-making, image reconstruction and target detection into a complete, end-to-end closed-loop process. The event flow is not only used for imaging, but also to guide the imaging process itself, realizing an efficient closed loop of information flow. This overall architecture is significantly different from the traditional "image first, process later" separate scheme. The protection point lies in the overall system architecture that includes all the above modules and forms a closed-loop feedback.

[0076] Specifically, Figure 1 A flowchart illustrating a method for fuzz-free imaging detection of high-speed moving objects in industrial applications according to the present invention is shown.

[0077] like Figure 1 As shown, this invention discloses a blur-free imaging detection method for high-speed moving objects in industrial applications, comprising the following steps:

[0078] The raw event stream captured by the camera is preprocessed to obtain a standardized event stream;

[0079] Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate.

[0080] Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode.

[0081] Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set;

[0082] Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

[0083] It should be noted that, in this embodiment, the present invention first asynchronously acquires the original event stream triggered by high-speed moving objects using an event camera. After preprocessing, denoising, and formatting, the cleaned event stream is simultaneously sent to the intelligent agent. The intelligent agent extracts spatiotemporal features such as event density and spatial entropy in real time to obtain a state vector. The intelligent agent is one of the core innovations of this invention. It is an intelligent decision-making unit based on reinforcement learning or adaptive control theory, capable of analyzing the spatiotemporal distribution of the event stream in real time and dynamically determining when and how to trigger image reconstruction. It does not rely on external synchronization signals but is data-driven. The reward function of the intelligent agent is designed as follows:

[0084] ;

[0085] Among them, the reward function Guide the agent toward the goal optimization. To assess the quality of reconstructed images, for example, using the Laplacian variance of the image as a sharpness metric, for a reconstructed image... Its clarity can be defined as: ,in, This is the Laplacian operator for images; a larger value generally means a sharper image with more defined edges. These are weighting coefficients. It is the total delay from the appearance of the object to the output of the detection result. It refers to the consumption of computing resources.

[0086] Furthermore, in this embodiment, the reinforcement learning policy network then adaptively decides whether to trigger reconstruction and determines the optimal time window and reconstruction mode; subsequently, events within the corresponding time window are extracted according to the corresponding reconstruction instructions, the local motion velocity is estimated by minimizing the energy function, and motion compensation projection is performed on each event. Then, a clear grayscale image without motion blur is generated using Gaussian kernel density estimation as the clear image; finally, lightweight deep learning or traditional visual detection is performed on the clear image to output results such as defects, categories, or sizes.

[0087] According to an embodiment of the present invention, the preprocessing of the raw event stream acquired by the camera to obtain a standardized event stream specifically includes:

[0088] The raw event stream is obtained by acquiring the light intensity changes of each pixel of a high-speed moving object captured by the camera. An event is output when the absolute value of the logarithmic change in light intensity exceeds a preset threshold. ,in, It is the first The pixel coordinates of each event. It is the first The timestamp of each event It is the first The polarity of an event;

[0089] The original event stream is preprocessed to obtain the canonical event stream, wherein the preprocessing mechanism includes spatiotemporal domain inspection, denoising, timestamp alignment, and coordinate normalization.

[0090] It should be noted that, in this embodiment, the camera is an event camera, which is used to continuously capture the dynamics of high-speed moving objects. When a moving object enters the camera's field of view, the movement of its surface texture or edges triggers a dense event stream. Therefore, the original event stream is obtained by acquiring the light intensity changes on each pixel of the high-speed moving object captured by the camera. When the absolute value of the logarithmic change in light intensity exceeds a preset threshold, an event is output. ,in, It is the first The pixel coordinates of each event. It is the first The timestamp of each event (in microseconds). It is the first The polarity of each event indicates whether the light intensity of the current pixel increases (ON event) or decreases (OFF event). The event is triggered when the absolute value of the brightness change exceeds a preset threshold. : This is the fundamental difference between event cameras and traditional cameras: they only record changes, not absolute values.

[0091] Furthermore, in this embodiment, the original event stream is preprocessed to obtain the canonical event stream. The preprocessing mechanism includes spatiotemporal neighborhood checking, denoising, timestamp alignment, and coordinate normalization. For example, a spatiotemporal neighborhood-based denoising method can be used; if an event is in its neighborhood... If no other events are found, they are considered noise and discarded.

[0092] According to an embodiment of the present invention, the step of extracting spatiotemporal features from the canonical event stream to obtain a state vector specifically includes:

[0093] The event density is calculated using the following formula:

[0094] ;

[0095] in, For event density, For time intervals, Time interval The total number of events within;

[0096] The spatial entropy is calculated using the following formula:

[0097] ;

[0098] in, For spatial entropy, For pixel coordinates, For events in pixels The probability of this happening;

[0099] The average timestamp transformation rate is obtained by calculating the activity level of the event stream over time. ;

[0100] The state vector is obtained based on the event density, the spatial entropy, and the average timestamp change rate. .

[0101] It should be noted that, in this embodiment, an intelligent agent is used for spatiotemporal feature extraction. The agent does not process the entire image, but rather the statistical features of the event stream; correspondingly, its state space... Composed of the spatiotemporal characteristics of a series of event streams, in this embodiment, the state vector is obtained based on the event density, the spatial entropy, and the average timestamp change rate. .

[0102] Furthermore, in this embodiment, the event density , Time interval The total number of events within, of which, This reflects the dynamic activity level of an object; high density usually indicates that the object is moving rapidly or that there are areas with high texture. Spatial entropy ,in, For events in pixels The probability of occurrence, spatial entropy This reflects the breadth of event distribution and helps distinguish between overall motion and local texture changes. High spatial entropy indicates a wide spatial distribution of the event, potentially corresponding to complex object motion; average timestamp transformation rate... It reflects the activity level of the event stream over time, corresponding to the temporal evolution speed of the event stream.

[0103] It should be noted that the above embodiments describe the implementation process of the agent using reinforcement learning as an example. The core function is "adaptive decision-making based on event flow characteristics". In other embodiments of the present invention, other algorithms with online learning and adaptive capabilities can be used instead, such as: (1) Fuzzy logic control: Features such as event density and spatial entropy can be used as inputs, and a set of predefined fuzzy rules can be used to decide the reconstruction parameters. This method has a smaller computational load and is suitable for scenarios with high interpretability requirements; (2) Heuristic rules: Some simple threshold rules can be set, such as "when the total number of events in the past 10 milliseconds exceeds 5000, a reconstruction is triggered". Although this method is simple, it can work effectively under specific working conditions and is the lowest cost alternative.

[0104] According to an embodiment of the present invention, the step of using the trained policy network based on the state vector to make a decision on whether to output a reconstruction instruction specifically includes:

[0105] Use a pre-trained policy network Based on the current state vector Select Action The action space contains three key decisions. ,in,

[0106] The time window is used to determine the length of the reconstruction time window. and time offset relative to the current time ,in, ;

[0107] This is the reconstruction mode, used to determine whether to prioritize accuracy or speed.

[0108] To trigger reconstruction, it determines whether enough event information has been collected to construct a clear image in order to output a reconstruction instruction, which includes... .

[0109] It should be noted that, in this embodiment, the action space A includes three key decisions, one of which is triggering reconstruction. The first function is used to determine whether enough event information has been collected to construct a clear image, thereby outputting a reconstruction instruction to determine whether reconstruction should be performed; the second function is to select a time window. Used to determine the length of the time window for reconstruction. and time offset relative to the current time The selection of this window is dynamically adaptive. For example, for objects moving at high speeds, a shorter window is selected; for scenes with rapidly changing speeds, a window closest to the current time is selected. Thirdly, the reconstruction mode is selected. A trade-off between accuracy and speed is made, for example, choosing high-precision full-image reconstruction or low-latency local reconstruction.

[0110] According to an embodiment of the present invention, the step of reconstructing a dynamic event window based on the time window to obtain a window event set, and estimating the motion speed of each event based on the window event set, specifically includes:

[0111] Based on the time window in the reconstruction instruction, extract all events within the current window from the event stream buffer to obtain the window event set;

[0112] Project each event in the window event set onto a common reference moment. ;

[0113] The spatiotemporal distribution of events within a local neighborhood is analyzed to estimate the motion velocity corresponding to the event on the pixel. The calculation formula is as follows:

[0114] ;

[0115] in, Let be the energy function. For the speed of movement, It is the first The pixel coordinates of each event. For pixels The spatial gradient vector, It is the first The timestamp of each event It is the first The polarity of an event, It is the square norm.

[0116] It should be noted that, in this embodiment, after the agent issues a reconstruction command, it follows the time window specified in the reconstruction command. Extract all events within the window from the event stream buffer. Among them, such as Figure 2 As shown, traditional reconstruction methods (such as direct event accumulation) produce "motion blur" due to the movement of objects within a time window. Under the traditional direct event accumulation method, a time window... Inside, because the object moves from left to right, a fixed physical feature point (such as a corner of a workpiece) will appear at different times. The generated events (represented by hollow circles, light gray circles, dark gray circles, and solid circles, respectively) are distributed at different pixel positions on the image plane. Directly accumulating these events will result in a blurred band-shaped region. This invention proposes an event reconstruction method based on motion compensation. The core idea of ​​this method is that for any event within a window... Project it onto a common reference time. (Usually, the center time of the window is taken) To eliminate the effects of motion, such as Figure 3 As shown, the present invention first estimates the motion velocity vector of the physical feature point. (The horizontal arrow pointing to the right in the image) will then occur at different times. The event, according to the formula Compensation to the same reference time (The time taken at the center of the window in the figure) The compensated events (all the dots in the figure are aligned to the same column) basically overlap in spatial position. Finally, a clear and sharp imaging point (the solid circle on the right side of the figure) is generated by accumulating the compensated events. For this purpose, it is necessary to estimate the trajectory of the object point corresponding to each event in space.

[0117] Furthermore, in this embodiment, it is assumed that within an extremely short time window, the object's motion can be approximated as uniform linear motion, for pixels... For events on the surface, their corresponding velocity can be estimated by analyzing the spatiotemporal distribution of events within their local neighborhood. This can be achieved by solving an optical flow equation, where, for an event point, the intensity change follows the constraint: Since the event camera only outputs intensity changes, Can be determined by event polarity Since it approximates the time difference, event data within the window event set can be used to minimize the energy function. To estimate local motion velocity ,in, This is the energy function, corresponding to the spacetime energy function. For the speed of movement, It is the first The pixel coordinates of each event. For pixels The spatial gradient vector, It is the first The timestamp of each event It is the first The polarity of an event, It is the square norm.

[0118] Furthermore, in this embodiment, after integration, the formula for minimizing the spatiotemporal energy function is as follows: In the actual discrete event domain, the velocity and gradient of each pixel or each event are solved using all events within the window. It can be estimated by applying Gaussian smoothing to local events.

[0119] According to an embodiment of the present invention, the step of obtaining a compensation event set by projecting events based on the motion speed, and generating a clear image based on the reconstruction mode and the compensation event set, so as to send the clear image into a preset detection module for detection and outputting detection results, specifically includes:

[0120] After obtaining the motion speed, each event is compensated to the reference time. The pixel position is compensated for pixel coordinates. To obtain the compensation event set;

[0121] Based on the reconstruction mode and the compensation event, Gaussian kernel density estimation is used to generate the sharp image, and the calculation formula is as follows:

[0122] ;

[0123] in, For a clear image, For pixel coordinates, To compensate for pixel coordinates, It is the first The polarity of an event, It is the smoothing parameter corresponding to the reconstruction mode;

[0124] The clear image is sent to the detection module for detection to output the detection result. The detection module includes a convolutional neural network, and the detection content includes running on an embedded device to complete the classification, positioning and defect detection of the workpiece.

[0125] It should be noted that, in this embodiment, after obtaining the motion speed, each event is compensated to the reference time. The pixel position is compensated for pixel coordinates. ,in, , , For speed of movement The horizontal component, For speed of movement The vertical component, the reconstructed grayscale image The event intensity can be accumulated at the compensated position. One effective accumulation method is to use a Gaussian kernel for density estimation to smooth noise and generate a continuous image. The final clear image is obtained by weighted accumulation of all projected events. In this embodiment, a Gaussian kernel is used to ensure the smoothness of the image. The present invention transforms the temporal domain information of events into spatial domain information and achieves the "spatialization" of temporal resolution through motion estimation and compensation.

[0126] Furthermore, in this embodiment, the reconstructed clear image The image is fed into a high-efficiency detection module for inspection. This module uses a lightweight convolutional neural network that runs directly on embedded devices to classify, locate, and detect defects in workpieces, outputting the final detection results. In this embodiment, the convolutional neural network takes a reconstructed grayscale image as input, passes through a standard convolutional layer for initial feature extraction, and then connects to multiple depthwise separable convolutional blocks. Each depthwise separable convolutional block consists of depthwise convolution and pointwise convolution, which significantly reduces computation while maintaining accuracy. Finally, the network outputs the final detection results, such as the workpiece category probability, bounding box coordinates, or defect type, through a global average pooling layer and a fully connected layer. The entire network structure is compact and suitable for real-time inference on embedded platforms.

[0127] It should be noted that the above embodiments illustrate motion compensation and image reconstruction using Gaussian kernels. In other embodiments of the present invention, alternatives to motion estimation algorithms include: in addition to solving the optical flow equation, a feature point matching-based method can be used, for example, extracting corner points in the event stream (e.g., using an event-based FAST corner detector), and then estimating the global or local homography matrix by matching corner points within a continuous time window, thereby performing motion compensation. This method may be more advantageous when the object undergoes rotation or non-planar motion; and alternatives to image reconstruction methods include: in addition to using Gaussian kernel density estimation, more efficient methods can be used, such as directly counting the compensated events to generate a binary image; or using a Poisson-based reconstruction method to reconstruct a higher-quality grayscale image from the gradient field of the events, which can better restore the texture details of the object.

[0128] It should be noted that the above embodiments illustrate the use of convolutional neural networks as the detection module. In other embodiments of the present invention, the detection module is not limited to deep learning models. Depending on the specific task, traditional machine vision algorithms can also be used, such as: template matching based on shape matching: for workpieces with fixed shapes, their contour templates can be constructed, and then fast matching can be performed on the reconstructed clear image to achieve localization and detection; and feature-based size measurement: the edges of the workpiece are extracted using the Canny edge detection operator, and then a straight line or circle is fitted using the least squares method to achieve high-precision geometric size measurement.

[0129] It should be noted that, in this embodiment, reference is made to... Figure 4 The diagram shows a comparison between the imaging effects of a conventional camera and those of the present invention. Figure 4 In the image 'a', the image is taken by a traditional frame camera in a high-speed moving scene. The image exhibits significant motion blur, severe edge trailing of the workpiece, and loss of texture details. Figure 4In the figure, b represents the image reconstructed by the method of the present invention under the same motion conditions. The image has clear and sharp edges, complete texture details, and no motion blur artifacts, providing a high-quality image input for subsequent detection.

[0130] Figure 5 A block diagram of a blur-free imaging detection system for high-speed moving objects in industrial applications according to the present invention is shown.

[0131] like Figure 5 As shown, this invention discloses a blur-free imaging detection system for high-speed moving objects in industry, including a memory and a processor. The memory includes a program for a blur-free imaging detection method for high-speed moving objects in industry. When the processor executes the program, the blur-free imaging detection method for high-speed moving objects in industry implements the following steps:

[0132] The raw event stream captured by the camera is preprocessed to obtain a standardized event stream;

[0133] Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate.

[0134] Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode.

[0135] Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set;

[0136] Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

[0137] It should be noted that when the industrial high-speed moving object fuzz-free imaging detection system disclosed in this invention is applied, the specific process corresponds to the industrial high-speed moving object fuzz-free imaging detection method described in the above embodiments. Since the specific implementation details of the system application are consistent with the content of the above-mentioned industrial high-speed moving object fuzz-free imaging detection method, no further details will be provided in this embodiment.

[0138] A third aspect of the present invention provides a computer-readable storage medium comprising a program for a method of detecting a high-speed moving object in an industrial environment without blurring, wherein when the program is executed by a processor, it implements the steps of the method for detecting a high-speed moving object in an industrial environment without blurring as described in any of the preceding claims.

[0139] The fourth aspect of the present invention provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to execute the method of any of the embodiments described in the above-described industrial high-speed moving object blur-free imaging detection method embodiments.

[0140] The fifth aspect of the present invention provides an electronic device, the electronic device comprising: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device performs the steps of the blur-free imaging detection method for high-speed moving objects in industry as described in any of the preceding claims.

[0141] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0142] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0143] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0147] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0148] This invention discloses a blur-free imaging detection method, system, medium, and device for high-speed moving objects in industrial applications. It fundamentally eliminates motion blur and outputs high-definition images. The event-driven mechanism significantly reduces data redundancy and computational load, enabling low-latency real-time detection. It has a high dynamic range of over 120dB, adapting to complex lighting conditions. It does not rely on external synchronization signals, making the system simple and reliable. The overall cost is lower than traditional high-speed imaging and multi-sensor fusion solutions.

[0149] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for blur-free imaging detection of high-speed moving objects in industrial applications, characterized in that, Includes the following steps: The raw event stream captured by the camera is preprocessed to obtain a standardized event stream; Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate. Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode. Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set; Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

2. The method for fuzz-free imaging detection of high-speed moving objects in industrial applications according to claim 1, characterized in that, The process of preprocessing the raw event stream acquired by the camera to obtain a standardized event stream specifically includes: The raw event stream is obtained by acquiring the light intensity changes of each pixel of a high-speed moving object captured by the camera. An event is output when the absolute value of the logarithmic change in light intensity exceeds a preset threshold. ,in, It is the first The pixel coordinates of each event. It is the first The timestamp of each event It is the first The polarity of an event; The original event stream is preprocessed to obtain the canonical event stream, wherein the preprocessing mechanism includes spatiotemporal domain inspection, denoising, timestamp alignment, and coordinate normalization.

3. The method for fuzz-free imaging detection of high-speed moving objects in industrial applications according to claim 2, characterized in that, The process of extracting spatiotemporal features from the standardized event stream to obtain a state vector specifically includes: The event density is calculated using the following formula: ； in, For event density, For time intervals, Time interval The total number of events within; The spatial entropy is calculated using the following formula: ； in, For spatial entropy, For pixel coordinates, For events in pixels The probability of this happening; The average timestamp transformation rate is obtained by calculating the activity level of the event stream over time. ; The state vector is obtained based on the event density, the spatial entropy, and the average timestamp change rate. .

4. The method for fuzz-free imaging detection of high-speed moving objects in industrial applications according to claim 3, characterized in that, The step of using the trained policy network based on the state vector to make decisions on whether to output a reconstruction instruction specifically includes: Use a pre-trained policy network Based on the current state vector Select Action The action space contains three key decisions. ,in, The time window is used to determine the length of the reconstruction time window. and time offset relative to the current time ,in, ; This is the reconstruction mode, used to determine whether to prioritize accuracy or speed. To trigger reconstruction, it determines whether enough event information has been collected to construct a clear image in order to output a reconstruction instruction, which includes... .

5. The method for blur-free imaging detection of high-speed moving objects in industrial applications according to claim 4, characterized in that, The process of reconstructing a dynamic event window based on the time window to obtain a window event set, and estimating the motion speed of each event based on the window event set, specifically includes: Based on the time window in the reconstruction instruction, extract all events within the current window from the event stream buffer to obtain the window event set; Project each event in the window event set onto a common reference moment. ; The spatiotemporal distribution of events within a local neighborhood is analyzed to estimate the motion velocity corresponding to the event on the pixel. The calculation formula is as follows: ； in, Let be the energy function. For the speed of movement, It is the first The pixel coordinates of each event. For pixels The spatial gradient vector, It is the first The timestamp of each event It is the first The polarity of an event, It is the square norm.

6. The method for blur-free imaging detection of high-speed moving objects in industrial applications according to claim 5, characterized in that, The process of projecting events based on the motion speed to obtain a compensation event set, and generating a clear image based on the reconstruction mode and the compensation event set, and then sending the clear image into a preset detection module for detection to output the detection result, specifically includes: After obtaining the motion speed, each event is compensated to the reference time. The pixel position is compensated for pixel coordinates. To obtain the compensation event set; Based on the reconstruction mode and the compensation event, Gaussian kernel density estimation is used to generate the sharp image, and the calculation formula is as follows: ； in, For a clear image, For pixel coordinates, To compensate for pixel coordinates, It is the first The polarity of an event, It is the smoothing parameter corresponding to the reconstruction mode; The clear image is sent to the detection module for detection to output the detection result. The detection module includes a convolutional neural network, and the detection content includes running on an embedded device to complete the classification, positioning and defect detection of the workpiece.

7. A blur-free imaging detection system for high-speed moving objects in industrial applications, characterized in that, The system includes a memory and a processor. The memory contains a program for a method of detecting high-speed moving objects in the industrial environment without blurring. When the processor executes the program, the method performs the following steps: The raw event stream captured by the camera is preprocessed to obtain a standardized event stream; Spatiotemporal features are extracted from the normed event stream to obtain a state vector, which includes at least event density, spatial entropy, and average timestamp change rate. Based on the state vector, a trained policy network is used to make a decision to determine whether to output a reconstruction instruction. The reconstruction instruction includes a time window and a reconstruction mode. Based on the time window, the dynamic event window is reconstructed to obtain a window event set, and the motion speed of each event is estimated based on the window event set; Based on the motion speed, an event projection is performed to obtain a compensation event set. Based on the reconstruction mode, a clear image is generated by combining the compensation event set. The clear image is then sent to a preset detection module for detection to output the detection result.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a program for a method of detecting high-speed moving objects in industrial applications without blurring. When the program is executed by a processor, it implements the steps of a method for detecting high-speed moving objects in industrial applications without blurring, as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes computer program code, which, when run on a computer, causes the computer to implement the steps of the blur-free imaging detection method for high-speed moving objects in industrial applications as described in any one of claims 1 to 6.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to cause the electronic device to perform the steps of the blur-free imaging detection method for industrial high-speed moving objects as described in any one of claims 1 to 6.

Images