Infrared video dual-channel enhanced output and defect detection method and system

Abstract

The present application relates to the technical field of image processing, and discloses an infrared video dual-path enhancement output and defect detection method and system, which comprises receiving infrared detector pixel data, combining a bad point position matrix to repair and generate a pretreatment image matrix and performing gray scale mapping; extracting the gray scale gradient amplitude and local pixel variance of the basic mapping gray scale value to respectively generate local enhancement gray scale value and network enhancement gray scale value; when image frame accumulation occurs, updating the network side actual reading address across frames; respectively reading dual-path enhancement image frames to perform output, and sending the pretreatment image matrix without network side smoothing into a detection path to obtain a defect candidate connected domain area; when the connected domain area exceeds a threshold value, driving a sorting mechanism to complete a rejection action. The present application decouples heterogeneous image feature requirements of local detection and network transmission through dual-path processing, adaptively adjusts the reading address according to the buffer state to reduce system delay, and ensures the timing accuracy of the physical sorting action of the production line.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for dual-channel enhanced output and defect detection of infrared video. Background Technology

[0002] In industrial visual inspection and mass production testing of infrared equipment, the raw video stream output by the infrared detector needs to be analyzed and processed in real time. The processed image data usually needs to be simultaneously distributed to the local end for visual monitoring and defect judgment, and then transmitted to the back-end server via the network for video encoding and archiving.

[0003] Existing infrared video processing methods typically employ a single image processing link, uniformly enhancing the raw data before distributing it to local and network endpoints via a shared cache. Due to the heterogeneous requirements for image features between the local and network endpoints, this single processing architecture has limitations in practical applications. Local visual monitoring and machine vision defect recognition rely on edge details in images, requiring the preservation of high-frequency features to maintain the contrast between the target and the background. However, network-side video coding algorithms are sensitive to high-frequency noise; directly receiving and processing images containing high-frequency details would cause significant fluctuations in the encoding bitrate and consume excessive network bandwidth. If uniform smoothing and noise reduction are applied to the raw image to reduce the transmission bitrate, minute defect features in the image would be erased, leading to missed detections in subsequent inspection stages.

[0004] Furthermore, existing systems experience image frame backlogs in the buffer when encountering network bandwidth fluctuations or transmission limitations. Lacking an underlying adaptive scheduling and frame dropping mechanism to address this backlog, network congestion negatively increases the overall latency of the front-end image processing pipeline. In continuous inspection scenarios, the time it takes for an image frame to be acquired, processed, and for a defect determination to be obtained becomes uncontrollable, disrupting the synchronization between image processing timing and material transport position. This processing delay directly causes timing misalignment when the inspection system sends drive commands to the automated sorting mechanism, preventing the pusher device from acting on the corresponding target and resulting in the rejection of incorrect or missing items during the production line sorting process. Summary of the Invention

[0005] The technical problem solved by this invention is that existing infrared video detection systems use a single image processing link, which cannot simultaneously meet the requirements of smooth noise reduction at the network end to reduce the transmission bit rate and the requirements of local visual detection to preserve edge details to improve the defect recognition rate; image frame backlog caused by network fluctuations leads to system processing delays, which in turn causes misalignment of the sorting and rejection action on the production line and incorrect rejection.

[0006] To address the above problems, the present invention provides the following technical solution: The first aspect of this invention provides a method for dual-channel enhanced infrared video output and defect detection, comprising: Receive pixel data, line synchronization signal and frame synchronization signal output by infrared detector, combine with the detector bad pixel location matrix to reconstruct and repair bad pixels, and generate a preprocessed image matrix; Perform grayscale mapping on the preprocessed image matrix to generate basic mapped grayscale values; Extract the grayscale gradient magnitude and local pixel variance of the base mapping grayscale value, generate local enhanced grayscale value and network enhanced grayscale value respectively, and update the actual read address on the network side when image frame backlog occurs; Locally enhanced image frames composed of the local enhanced grayscale values ​​are read respectively, and network enhanced image frames composed of the network enhanced grayscale values ​​are read according to the actual reading address of the network side to perform enhanced output. The preprocessed image matrix that has not been smoothed by the network side is sent to the independent defect detection path for defect detection to obtain the area of ​​the defect candidate connected region. When the area of ​​the candidate defective connected region exceeds the pre-defined defect area threshold, the automated sorting mechanism of the production line is driven to complete the sorting and rejection action.

[0007] This invention calculates enhancement weights based on grayscale gradient magnitude and local pixel variance, forming a dual-path image processing branch for local visual inspection and network-encoded transmission. The local processing branch uses grayscale gradient information to preserve image edge details; the network processing branch performs smoothing and noise reduction using pixel variance to reduce high-frequency noise and decrease the network-side transmission bitrate. The defect detection path independently receives the pre-processed image matrix without smoothing and noise reduction, avoiding the loss of subtle defect features caused by the smoothing process. When image frame backlog occurs, the intermediate backlogged frames are discarded by skipping over the actual read address on the network side, reducing system processing latency, and triggering the pusher action of the automated sorting mechanism based on the defect candidate connected component area determination result.

[0008] Preferably, the pixel data, line synchronization signal, and frame synchronization signal output by the received infrared detector are combined with the detector's bad pixel location matrix for reconstruction and bad pixel repair to generate a preprocessed image matrix, including: Using the line synchronization signal and the frame synchronization signal as timing references, the pixel data output by the infrared detector is reconstructed into the original two-dimensional digital matrix; Assign frame sequence numbers and target numbers to the original two-dimensional digital matrix according to the receiving order, and write the frame sequence numbers and target numbers into the frame header of the corresponding storage space of the original two-dimensional digital matrix; When the detector bad pixel location matrix indicates that there are bad pixels in the original two-dimensional digital matrix, the bad pixel is replaced with the median of the neighboring non-bad pixels to generate the preprocessed image matrix.

[0009] This step reconstructs the pixel stream into a matrix form based on a temporal reference, and assigns frame sequence numbers and target numbers to the frame headers of the corresponding storage spaces, establishing a data correspondence between digital image features and physical targets to be inspected. By locating defective pixels and calculating the median of their neighboring non-defective pixels for overlay replacement, the influence of invalid pixel nodes from the detector on subsequent feature extraction calculations is eliminated.

[0010] Preferably, in the step of extracting the grayscale gradient magnitude and local pixel variance of the base mapped grayscale value to generate the locally enhanced grayscale value and the network enhanced grayscale value respectively, generating the locally enhanced grayscale value includes: Based on the extracted grayscale gradient magnitude, sharpening weights are generated in conjunction with noise suppression parameters; Calculate the four-neighbor Laplace detail value based on the basic mapping grayscale value; The local enhanced grayscale value is generated by superimposing the base mapping grayscale value, the sharpening weight, and the four-neighbor Laplacian detail value.

[0011] The process of generating locally enhanced grayscale values ​​combines the grayscale gradient magnitude and noise suppression parameters to calculate sharpening weights, and then uses four-neighbor Laplacian detail values ​​to superimpose and enhance the base mapped grayscale values. This process increases the grayscale variation amplitude at the image contour edges, improving the grayscale contrast between local defect areas and the background.

[0012] Preferably, in the step of extracting the grayscale gradient magnitude and local pixel variance of the base mapped grayscale value to generate the locally enhanced grayscale value and the network-enhanced grayscale value respectively, generating the network-enhanced grayscale value includes: A local neighborhood window is constructed using the base mapped grayscale value as the center pixel. Calculate the mean value of the surrounding pixels within the local neighborhood window, and calculate the local pixel variance accordingly; The smoothing weight is calculated by combining the local pixel variance and the grayscale difference between the surrounding pixels and the center pixel within the local neighborhood window. The smoothing weights are used to weight and fuse the base mapping grayscale value and the surrounding pixels within the local neighborhood window to generate the network-enhanced grayscale value.

[0013] When generating enhanced grayscale values ​​for the network, a local neighborhood window is constructed, and the smoothing weight is determined by the local pixel variance and the grayscale difference around the central pixel. The base mapped grayscale value and the surrounding pixels are weighted and fused to filter out random noise points in areas with small grayscale fluctuations, reducing spatial redundancy within the image frame and thus reducing the amount of data during network video encoding and transmission.

[0014] Preferably, updating the actual read address on the network side when an image frame backlog occurs includes: Based on the network cache write pointer position and the network cache read pointer position, monitor and determine the number of cache backlog frames in which the image frame backlog state occurs; When the number of backlogged frames in the cache reaches a preset congestion trigger condition, an adaptive frame skipping step size is calculated based on the number of backlogged frames in the cache. When the adaptive frame skipping step size is greater than 1, the reading position is updated across intermediate backlogged image frames to obtain the actual reading address on the network side.

[0015] This section obtains the number of backlogged frames in the cache based on the relative positions of the read and write pointers, and adaptively increases the frame skipping step size after the congestion trigger condition is met. When the step size is greater than 1, the read address pointer skips the corresponding backlogged image frame to update, releasing the cache space occupied by congested data and maintaining the temporal continuity of system data flow read and write operations.

[0016] Preferably, the step of feeding the preprocessed image matrix without network-side smoothing into an independent defect detection path for defect detection to obtain the area of ​​candidate connected components for defects includes: Extract the region of interest of the target to be detected from the preprocessed image matrix; Traverse the valid unmasked pixels within the region of interest of the target to be inspected, and extract the abnormal pixels of the product that meet the abnormality threshold condition. After removing isolated noise from the abnormal product pixels, connected component labeling is performed, and adjacent abnormal product pixels are divided into the same defect candidate pixel connected component. The area of ​​the defect candidate connected region is obtained by counting the total number of valid pixel nodes contained in the connected region of the defect candidate pixel.

[0017] In the defect detection path, the region of interest of the target object is extracted to exclude interfering pixels from non-target areas. Connectivity labeling is performed on the extracted abnormal pixels, and spatially adjacent abnormal pixels are integrated into defect candidate pixel connected regions. The total number of valid pixel nodes within a connected region represents the physical size of the region.

[0018] Preferably, the step of driving the automated sorting mechanism of the production line to complete the sorting and rejection action when the area of ​​the defect candidate connected region exceeds a pre-defined defect area threshold includes: The maximum candidate connected component area of ​​defects is obtained in each image frame associated with the same target number, and the maximum value in all associated image frames is taken as the maximum candidate connected component area of ​​defects of the target to be inspected. When it is determined that the area of ​​the candidate connected region of the maximum defect of the target is greater than the defect area threshold, the transport encoder count value recorded when the target to be inspected enters the image acquisition area is extracted. Write the target number, the area of ​​the candidate connected component with the largest defect of the target, and the count value of the conveyor encoder into the sorting control queue; The push rod's position is calculated based on the sum of the conveyor encoder count value and the encoder count offset recorded in the sorting control queue. When the current count value of the conveyor encoder reaches the push rod's position, the output interface is controlled to flip its level, driving the push rod device of the automated sorting mechanism on the production line to complete the sorting and rejection action.

[0019] For multiple image frames associated with the same target number, the area of ​​the candidate connected component with the largest defect is extracted as the final judgment criterion. When the area exceeds the threshold, the push rod's action position is calculated from the conveyor encoder count value and the offset. By comparing the current conveyor encoder count value with the push rod's action position, when the two count values ​​are consistent, the data output terminal is controlled to flip the level, triggering the push rod device to act.

[0020] A second aspect of the present invention provides an infrared video dual-channel enhanced output and defect detection system, comprising: The data acquisition and preprocessing module is used to receive pixel data, line synchronization signals and frame synchronization signals output by the infrared detector, and to reconstruct and repair bad pixels by combining them with the bad pixel location matrix of the detector, thereby generating a preprocessed image matrix. The dual-path differentiated image enhancement and scheduling module is used to perform grayscale mapping on the preprocessed image matrix to generate basic mapped grayscale values; extract the grayscale gradient magnitude and local pixel variance of the basic mapped grayscale values ​​to generate local enhanced grayscale values ​​and network enhanced grayscale values ​​respectively, and update the actual read address on the network side when image frame backlog occurs. The local defect detection and output execution module is used to read the local enhanced image frame composed of the local enhanced grayscale values, and to read the network enhanced image frame composed of the network enhanced grayscale values ​​according to the actual reading address on the network side to perform enhanced output. The preprocessed image matrix that has not been smoothed by the network side is sent to the independent defect detection path for defect detection to obtain the defect candidate connected component area. When the defect candidate connected component area exceeds the pre-calibrated defect area threshold, the automated sorting mechanism of the production line is driven to complete the sorting and rejection action.

[0021] The system of this invention uses independent module units to receive and process image data, dividing the image grayscale enhancement process into a two-way parallel execution architecture. Locally enhanced image frames and network-enhanced image frames are acquired separately to accommodate the data requirements of both local display and network transmission. By directly inputting the preprocessed image matrix into the defect detection channel, the interference of the smoothing and noise reduction algorithm on the defect determination result is eliminated.

[0022] Preferably, the system includes a core processing circuit and a storage cache area connected to the core processing circuit; The data acquisition and preprocessing module and the dual-channel differentiated image enhancement and scheduling module are integrated into the core processing circuit. The core processing circuit is used to process and generate the basic mapped grayscale value, the local enhanced grayscale value, and the network enhanced grayscale value. The core processing circuit writes the local enhanced grayscale value and the network enhanced grayscale value into the local display frame buffer and the network encoded image buffer, which are isolated from each other in the storage buffer, respectively. Based on the image frame backlog status of the network encoded image buffer, it determines the number of backlogged frames and updates the actual read address on the network side.

[0023] The core processing circuit simultaneously performs calculations for two enhanced grayscale values ​​and stores them separately in isolated address spaces within the storage buffer to prevent data bus read / write operations from overwriting each other. The core processing circuit reads the backlog status of the network-encoded image buffer and calculates the number of backlogged frames, directly modifying the actual read address of subsequent image frames in the buffer.

[0024] Preferably, the system further includes a transport encoder and a data output terminal connected to the core processing circuit; The local defect detection and output execution module is integrated into the core processing circuit. When the core processing circuit determines that the area of ​​the defect candidate connected region exceeds the defect area threshold, it controls the level state of the data output terminal according to the conveying position provided by the conveying encoder. The data output terminal is connected to the push rod device of the automated sorting mechanism on the production line, and the push rod device is driven by the level state to complete the sorting and rejection action.

[0025] The system monitors the material conveying position on the production line using a conveyor encoder. The core processing circuit converts the action command into a level state at the data output terminal based on the determined area. The push rod device receives the level state change signal and executes the sorting and rejection action, tracking the material movement trajectory through the encoder count value to align with the target action position.

[0026] This invention provides a method and system for dual-channel enhanced infrared video output and defect detection. It has the following beneficial effects: 1. This invention extracts the grayscale gradient magnitude and local pixel variance of the base mapping grayscale values ​​to generate locally enhanced grayscale values ​​that emphasize edge details and network-enhanced grayscale values ​​that emphasize smoothing and noise reduction, respectively. The preprocessed image matrix without network smoothing is then fed into an independent defect detection path. This method splits the original image data processing, decoupling the two outputs at the feature level. On the one hand, it uses a local smoothing algorithm to reduce the video encoding bitrate of the backend network transmission; on the other hand, it preserves the defect edge contrast of local visual detection, overcoming the limitation of traditional single image processing links that cannot simultaneously meet the heterogeneous needs of local defect recognition and network video transmission.

[0027] 2. This invention determines the number of backlogged frames by monitoring the positions of the network cache write and read pointers, and calculates an adaptive frame skipping step size to update the actual read address on the network side when congestion triggering conditions are met. When network bandwidth constraints cause image frame backlog, the system proactively abandons reading intermediate backlogged frames by skipping over the read address, releasing the space occupied by the underlying storage cache, maintaining the timing continuity of the front-end image acquisition and back-end processing pipeline, and solving the system processing delay problem caused by network fluctuations.

[0028] 3. This invention makes a comprehensive judgment by extracting the area of ​​the largest candidate connected region of defects from multiple image frames associated with the same target number, and calculates the push rod's action position by combining the sum of the encoder count value and offset when the target enters the image acquisition area. This mechanism relies on the hardware encoder to map the image frame processing sequence to the physical position of the material conveying on the production line. It only controls the flip-level state when the encoder's current count value reaches the push rod's action position, eliminating the timing misalignment caused by changes in the production line's running speed and image processing time, and reducing the error rejection rate during the production line sorting process. Attached Figure Description

[0029] Figure 1 This is a structural block diagram of the infrared video dual-channel enhanced output and defect detection system of the present invention; Figure 2 This is a flowchart illustrating the process of the infrared video dual-channel enhanced output and defect detection method of the present invention. Figure 3 This is a flowchart of the data acquisition and preprocessing stage of the present invention; Figure 4 This is a flowchart of the dual-path differential image enhancement and dual-path asynchronous scheduling stages of the present invention; Figure 5 This is a flowchart of the local defect detection and dual-channel independent output stage of the present invention; Figure 6 This is a two-dimensional distribution map of image features after the same basic grayscale image of the present invention has been processed by the local enhancement path and the network smoothing path; Figure 7 This is a diagram showing the correspondence between the number of backlogged frames in the network-coded image buffer and the adaptive frame skipping step size in this invention. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention provides a method and system for dual-channel enhanced output and defect detection of infrared video, the hardware operating environment of which includes a data access terminal, a core processing circuit, a storage buffer area and a data output terminal.

[0032] The data access terminal is equipped with a high-speed video data receiving interface and a synchronization timing extraction circuit, which is used to receive pixel data output by the infrared detector and extract the line synchronization signal and frame synchronization signal corresponding to the pixel data.

[0033] The core processing circuit employs a field-programmable gate array (FPGA) or a digital signal processor (DSP) with parallel processing capabilities. When the core processing circuit uses an FPGA, image processing is performed through parallel pipelined processing circuitry, line buffers, and independent data transfer channels. When the core processing circuit uses a DSP, separate data transfer channels are configured for acquisition, local display, and network encoding.

[0034] The storage buffer is located in physical memory and is divided into a local display frame buffer and a network-encoded image buffer, with mutually isolated address ranges. The data output is connected to the high-definition multimedia interface transmitting chip, the local monitor, the network-side hardware video encoder, and the drive relays and push rod devices of the automated sorting mechanism on the production line.

[0035] The system also includes a target positioning sensor located at the image acquisition station, a conveyor encoder for detecting the displacement of the conveying mechanism, and a pusher positioning sensor for detecting the position of the pusher. The target positioning sensor detects whether the target has entered the image acquisition area, the conveyor encoder determines the conveying displacement of the target from the image acquisition position to the pusher position, and the pusher positioning sensor provides feedback on whether the sorting and rejection action has been completed.

[0036] See attached document Figure 1 This invention provides an infrared video dual-channel enhanced output and defect detection system, which may include: The data acquisition and preprocessing module receives pixel data, line synchronization signals, and frame synchronization signals output by the infrared detector. Based on the line synchronization signals and frame synchronization signals, it reconstructs the pixel data arranged in the receiving time sequence into a two-dimensional image and combines it with the detector bad pixel location matrix stored in flash memory. Repairing detector defects in 2D images to generate a preprocessed image matrix. .in, Represents the matrix of detector dead spot locations median coordinate The element value at that location is used to indicate whether the corresponding coordinate is a bad spot on the detector; Represents the preprocessed image matrix median coordinate The number of pixels at that location.

[0037] The data acquisition and preprocessing module is also used to generate frame sequence numbers according to the receiving order of complete image frames, and when the sensor detects that the target has entered the image acquisition area, it assigns a target number to the target and timestamps the frames. The frame number and target number are stored as frame-level metadata associated with the corresponding image frame.

[0038] The dual-channel differential image enhancement and scheduling module is used to process the pre-processed image matrix according to the detector calibration digital range. The digital values ​​of each pixel in the image are mapped to grayscale to generate the basic mapped grayscale value corresponding to each pixel coordinate. Representing coordinates The base mapping grayscale value at the location; based on the base mapping grayscale value The local grayscale gradient is used to generate locally enhanced grayscale values ​​for local display. Based on the underlying mapping grayscale value The local pixel variance is used to generate network-enhanced grayscale values ​​for network coding. .

[0039] The dual-path differential image enhancement and scheduling module is also used to write pointer positions based on network cache. and network cache read pointer position Calculate the number of backlogged frames in the buffer. When the number of cached backlogged frames The network-coded image buffer is considered to have reached congestion triggering conditions when the following conditions are met: ; in, This represents the congestion trigger ratio. This indicates the total number of image frame storage slots contained in the network-coded image buffer. Upon reaching the congestion trigger condition, the core processing circuit determines the adaptive frame skipping step size according to the adaptive frame skipping step size calculation rule. ;when At that time, the network side reads the next unread image frame in the original order; when At that time, the network cache read pointer crosses The network determines the actual read address based on the target image frame storage slot after the frame is crossed, by accumulating image frames in the middle of the frame. The above cross-frame reading process only applies to the network-encoded image buffer and does not change the image frame processing order of the local display path and the local defect detection path.

[0040] The local defect detection and output execution module is used to read locally enhanced grayscale values ​​from the local display frame buffer. The local enhanced image frame is composed and sent to the display interface transmitting chip; the network-enhanced grayscale values ​​are read from the network-encoded image buffer. The network-enhanced image frames are composed of network-enhanced image frames and sent to the network-side hardware video encoder.

[0041] The local defect detection and output execution module is also used to process the preprocessed image matrix that has not undergone network-side smoothing. The signal is fed into an independent defect detection path, based on the detector's defect location matrix. The inherent bad pixel areas of the infrared detector are shielded, and the connected component statistics of abnormal pixels in the target to be inspected are performed to obtain the area of ​​candidate connected components for defects.

[0042] When the area of ​​a candidate defect connected component exceeds a pre-defined defect area threshold, the local defect detection and output execution module will assign the corresponding target number and frame-level timestamp. The defect determination results are written into the sorting control queue, and the level state of the general-purpose input / output interface is controlled according to the conveyor encoder's detection of the target to be inspected, thereby driving the automated sorting mechanism of the production line to complete the sorting and rejection action. The actual address read on the network side... It is only used to extract the image frame to be encoded from the network encoded image buffer and is not used as the basis for triggering the sorting mechanism.

[0043] See attached document Figure 2 This invention provides a method for dual-channel enhanced infrared video output and defect detection, comprising the following steps: S1, Data Acquisition and Preprocessing. The core processing circuit receives the one-dimensional pixel digital sequence arranged in time sequence from the infrared detector through the data access terminal. and its corresponding line synchronization signal and frame synchronization signal And read the detector bad spot location matrix from flash memory. .

[0044] The core processing circuit uses line synchronization signals and frame synchronization signal Using time-series references, the one-dimensional pixel digital sequence Reconstructed into a two-dimensional image matrix, and upon receiving the frame synchronization signal Effective triggering edge reads system clock count value to generate frame-level timestamps. The core processing circuit generates frame sequence numbers according to the received order of complete image frames and adds frame-level timestamps. The frame number is written into the frame header of the corresponding image frame.

[0045] When the target sensor detects that the target has entered the image acquisition area, the core processing circuit assigns a target number to the target and associates the target number with the image frames acquired within a preset acquisition time window. The preset acquisition time window is determined based on the length of the image acquisition area along the conveying direction, the operating speed of the conveying mechanism, and the acquisition frame rate of the infrared detector.

[0046] The core processing circuit uses the detector bad spot location matrix Scan the two-dimensional image matrix. When the detector's bad pixel location matrix... The value of the corresponding coordinate indicates that when that coordinate is a bad pixel on the detector, the median value of the non-bad pixel in the neighborhood of that coordinate is used to replace the current bad pixel value, thus generating a preprocessed image matrix. .

[0047] S2, dual-channel differential image enhancement and dual-channel asynchronous scheduling. The core processing circuit determines the digital threshold corresponding to the detector's calibrated lower temperature measurement limit. And the digital threshold corresponding to the calibrated upper limit of temperature measurement Determine the effective digital range of grayscale mapping based on the average global pixel digital value of the current frame. Determine the mapping gain coefficient and preprocess the image matrix The positions of each pixel's digital value relative to the effective digital value range are normalized, gain adjusted, rounded, and limited to a range of 0 to 255 to generate coordinates. The base mapping gray value at the location .

[0048] The core processing circuit calculates the basic mapped grayscale value. grayscale gradient magnitude And calculate the sharpening weights based on the noise suppression parameters. ; Sharpening weight Laplace detail values ​​of the four neighboring regions Mapping grayscale values ​​to the base Overlaying is performed to generate locally enhanced grayscale values. .

[0049] The core processing circuit uses coordinates The base mapping gray value at the location Establish a 5×5 local neighborhood window centered on the target, and calculate the local pixel variance of the local neighborhood window. And calculate the base mapping gray value of each neighboring pixel within the local neighborhood window. The base mapping grayscale value of the center pixel The grayscale difference between them. The core processing circuit calculates the grayscale difference based on the local pixel variance. Together, we determine the smoothing weights corresponding to each neighboring pixel. In this model, the smaller the grayscale difference between neighboring pixels and the center pixel, the greater the corresponding smoothing weight; conversely, the greater the grayscale difference, the smaller the corresponding smoothing weight. Then, weighted fusion is performed according to the smoothing weights of each neighboring pixel to generate coordinates. Network-enhanced grayscale values ​​at the location .

[0050] The core processing circuit will process the preprocessed image matrix that has not undergone network-side smoothing. Continuous input to an independent defect detection path avoids reducing the digital anomalous features of small-area defects through network-side smoothing. Local enhancement of grayscale values ​​is used. Composed of locally enhanced image frames, grayscale values ​​enhanced by the network The network-enhanced image frames and the preprocessed image matrix in the defect detection pathway are composed of All retain the same frame-level timestamp as the corresponding original image frame. Frame number and target number.

[0051] Independent data transfer circuit reads network cache write pointer position and network cache read pointer position Calculate the number of backlogged frames in the cache. When the number of cached backlogged frames Satisfying the congestion trigger ratio coefficient When the corresponding trigger condition is met, the core processing circuit increases the read step size of the network-encoded image buffer, causing the network buffer read pointer to span some of the intermediate backlogged image frames, thus obtaining the actual read address on the network side. Cross-frame read operations only discard expired image frames in the network encoding path, do not discard image frames to be detected in the defect detection path, and do not change the read position of the local display frame buffer.

[0052] S3, Local Defect Detection and Dual Independent Output. The core processing circuit reads the local enhanced image frame from the local display frame buffer, converts it into the data format required by the display interface, and outputs it to the local monitor through the display transmission chip; simultaneously, it reads the actual address from the network side. The network-enhanced image frames are read from the network-encoded image buffer and then sent to the network-side hardware video encoder for compression.

[0053] The core processing circuit processes the preprocessed image matrix in the independent defect detection path. The process involves extracting abnormal pixels from the target object and forming connected components of candidate defect pixels, then calculating the area of ​​these candidate connected components. When the area of ​​a candidate connected component exceeds a pre-defined defect area threshold, the core processing circuit updates the defect determination result, target number, and frame-level timestamp. The encoder count value of the conveyor when the target to be inspected enters the image acquisition area is written into the sorting control queue.

[0054] When the conveyor encoder indicates that the corresponding target to be inspected has reached the pusher's position, the core processing circuit controls the output interface to switch its level, driving the relay and pusher device to perform a sorting and rejection action, pushing the corresponding target to be inspected into the defective product area. The actual address read on the network side... It is only used to select the image frame to be encoded in the network-encoded image buffer and does not participate in the determination of the push-stick action timing.

[0055] The specific implementation mechanisms of each of the above steps will be explained in detail in the following sections.

[0056] See attached document Figure 3 This corresponds to step S1 mentioned above. In this embodiment, the specific working process of the data acquisition and preprocessing module includes the following sub-steps: S101, the core processing circuit receives pixel data and line synchronization signals output by the infrared detector through the high-speed video data receiving interface in the data acquisition and preprocessing module. and frame synchronization signal .

[0057] The pixel data output by the infrared detector is logically represented as a one-dimensional sequence of pixel numbers arranged according to the receiving time sequence. One-dimensional pixel digital sequence This data is used to characterize the transient thermal radiation intensity detected by each infrared pixel of the infrared detector. In this embodiment, its data bit width is set to a 14-bit unsigned integer. Indicates the received order index of pixel data within an image frame. Horizontal synchronization signal. Frame synchronization signal used to indicate the timing boundary of a single row of pixel data. Both are logic level signals used to indicate the timing boundaries of a single complete infrared image frame.

[0058] The high-speed video data receiving interface employs an interface receiving circuit compliant with the Camera Link protocol. When the data transmitted between the infrared detector and the core processing circuit is a differential serial signal, the Camera Link interface receiving circuit first receives and deserializes the differential serial signal, and then transmits the deserialized pixel data to the core processing circuit in the form of a parallel pixel data bus, data valid signal, line valid signal, and frame valid signal. The core processing circuit forms a one-dimensional pixel digital sequence according to the receiving order of the parallel pixel data. .

[0059] In this embodiment, the image acquisition frame rate of the infrared detector is 50Hz. In actual implementation, the data receiving parameters of the core processing circuit are matched with the actual output configuration of the infrared detector. Specifically, the image acquisition frame rate adopts the currently configured output frame rate of the infrared detector, and the pixel data bit width adopts the actual bit width of the output pixel data of the infrared detector. The Camera Link interface receiving circuit configures the receiving and deserialization parameters according to the Camera Link operating mode, data channel configuration, pixel clock frequency, and timing of the line valid signal and frame valid signal used by the infrared detector, so that the pixel data and synchronization timing received by the core processing circuit are consistent with the output of the infrared detector.

[0060] S102, After the system starts up, the core processing circuit reads the detector bad spot location matrix from the flash memory through the internal communication bus. And the detector bad spot location matrix Stored in on-chip memory or external memory accessible by the core processing circuitry.

[0061] Detector bad spot location matrix This is a two-dimensional binary mask matrix corresponding to the infrared image resolution, with its row and column numbers matching the effective pixel rows and columns of the infrared focal plane array, respectively. Detector bad pixel location matrix. Used to record the pixel coordinates of failed infrared pixels in an infrared focal plane array.

[0062] In this embodiment, the detector bad spot location matrix A value of 1 indicates a defective pixel at the corresponding coordinate, while a value of 0 indicates a normal infrared pixel at the corresponding coordinate. The core processing circuit uses this matrix as a mask for subsequent defective pixel repair processing.

[0063] When the detector bad point location matrix in flash memory When no update occurs, the core processing circuit reads the data only once during system startup; when the detector is recalibrated and the bad pixel location data is updated, the core processing circuit rereads the updated detector bad pixel location matrix. .

[0064] S103, the core processing circuit uses the line synchronization signal and frame synchronization signal Using time-series references, the one-dimensional pixel digital sequence Write the data into the frame buffer in row and column order, and reconstruct it into a frame with spatial coordinates. 14-bit original two-dimensional digital matrix .

[0065] Specifically, the core processing circuit detects the frame synchronization signal. When the frame starts and becomes valid, the row and column counters are reset, and pixel data reception for the current image frame begins. During the period when pixel data is valid, the column counter increments once for each received pixel, and the corresponding pixel data is written into the original two-dimensional digital matrix based on the current row and column count values. The corresponding coordinates.

[0066] When a line synchronization signal is detected When a newline valid edge is detected, the core processing circuit resets the column counter and increments the row counter by one. When the next frame synchronization signal is detected... When the frame starts valid, the reception of the current image frame ends and the reconstruction of the next image frame begins.

[0067] The core processing circuit counts the number of valid pixels received in each row and the number of valid rows received in each frame based on the pre-configured image resolution. When the number of valid pixels in a single row or the number of valid rows in a single frame is inconsistent with the pre-configured image resolution, the corresponding image frame is marked as an incomplete frame, and the incomplete frame is prevented from entering the defect detection and sorting control process. The circuit then detects the next frame synchronization signal. Effective time-based restart of image frame reconstruction.

[0068] S104, when the core processing circuit detects the frame synchronization signal When the frame starts on a valid edge, a hardware counter is triggered to perform a latching operation, read the current system clock count value, and use the read system clock count value as the frame-level timestamp of the current image frame. Frame-level timestamps Used to characterize the acquisition time of the current image frame relative to the system startup time or the preset timing start point.

[0069] The core processing circuit will use frame-level timestamps Write the original two-dimensional digital matrix The corresponding frame header in the storage space is generated, and an incrementing frame sequence number is generated according to the receiving order of the complete image frames. The frame sequence number is then written into the frame header.

[0070] When the system is equipped with a real-time clock or an external unified time synchronization interface, the core processing circuit can also convert the system clock count value into a timestamp under the unified time reference based on the time reference provided by the real-time clock or the external unified time synchronization interface.

[0071] When the sensor outputs a valid trigger edge when the target to be inspected is in position, the core processing circuit increments the target number counter, assigns a target number to the target to be inspected that enters the image acquisition area, and records the frame number and the encoder count value when the sensor outputs a valid trigger edge.

[0072] The core processing circuit associates the continuously acquired image frames following the effective trigger with the current target number according to the preset acquisition time window, and writes the target number into the frame header of the corresponding image frame. The preset acquisition time window covers at least the time range from when the target enters the image acquisition area to when it leaves the image acquisition area. Its duration is determined by the ratio of the length of the image acquisition area along the conveying direction to the running speed of the conveying mechanism, and is converted into the corresponding number of image frames according to the acquisition frame rate of the infrared detector.

[0073] When the same target appears consecutively in multiple image frames, each image frame uses the same target number and is distinguished by its respective frame number. Therefore, image frame data can be correlated with frame-level timestamps. A correspondence is established between frame number, target number, and conveyor encoder count value, providing a frame-level index for subsequent dual-channel image processing, defect detection, and sorting timing control.

[0074] S105, the core processing circuit is based on the detector bad spot location matrix. For the original two-dimensional digital matrix Perform pixel-by-pixel inspection. When the detector's bad pixel location matrix... When the value corresponding to the current pixel coordinate is 1, the current pixel is identified as a defective pixel to be repaired; when the corresponding value is 0, the original two-dimensional digital matrix is ​​retained. The current pixel value in the image.

[0075] For the defective pixel to be repaired, the core processing circuit first obtains a 3×3 neighborhood centered on the defective pixel and excludes the defective pixel location matrix from the detector. The median value of the neighboring pixels is 1, and the remaining non-bad pixels are sorted. The median value of the non-bad pixels is used to replace the current bad pixel value.

[0076] When the defective pixel to be repaired is located at the image boundary, the edge pixel symmetric mapping method is used to obtain the neighborhood coordinates beyond the image boundary. When the number of non-defective pixels in the 3×3 neighborhood is less than the minimum effective neighborhood number, the repair neighborhood is expanded to 5×5, and the median value of the non-defective pixels in the 5×5 neighborhood is used to replace the current defective pixel value.

[0077] In this embodiment, the minimum effective neighboring pixel count is set to 3. In actual implementation, after the infrared focal plane array defect calibration is completed, the core processing circuit or calibration device counts the number of non-defective pixels in the 3×3 neighborhood corresponding to each detector defect based on the detector defect location matrix Md. When all detector defective pixels have at least 5 non-defective pixels in their corresponding 3×3 neighborhoods, the minimum effective neighboring pixel count is set to 5; when the aforementioned condition is not met, but all detector defective pixels have at least 4 non-defective pixels in their corresponding 3×3 neighborhoods, the minimum effective neighboring pixel count is set to 4; otherwise, the minimum effective neighboring pixel count is set to 3. For defective pixels to be repaired whose actual number of non-defective pixels in the 3×3 neighborhood is lower than the set minimum effective neighboring pixel count, the repair neighborhood is expanded to 5×5 in the aforementioned manner.

[0078] When the number of non-dead pixels in the 5×5 neighborhood is still lower than the number of minimum effective neighbor pixels, the repaired pixel value at the same coordinates in the previous effective image frame is used for replacement; when the previous effective image frame does not exist or the pixel value at the same coordinates in the previous effective image frame is unavailable, the non-dead pixel value in the current image frame that is closest to the bad pixel to be repaired is used for replacement.

[0079] After the core processing circuit repairs all detector defects in the current image frame, it generates a preprocessed image matrix while maintaining the 14-bit pixel data width and the two-dimensional image size. Preprocessed image matrix The corresponding frame header continues to retain the frame-level timestamp. The frame number, target number, and corresponding transport encoder count value.

[0080] After preprocessing, the system will preprocess the image matrix. As input data for dual-path differential image enhancement and local defect detection, and using the detector bad spot location matrix It continues to be passed to the local defect detection path to exclude inherent bad spot areas of the detector during the defect detection process.

[0081] See attached document Figure 4 This corresponds to step S2 mentioned above. In this embodiment, the specific working process of the dual-path differential image enhancement and scheduling module includes the following sub-steps: S201, the dual-channel differential image enhancement and scheduling module processes the pre-processed image matrix according to the detector calibration digital range. Perform grayscale mapping to generate the base mapping grayscale values. .

[0082] The pixel count output by an infrared detector may experience baseline drift due to variations in detector operating temperature and ambient temperature. If each image is independently linearly stretched according to the maximum and minimum pixel counts of the current image frame, the mapping range between consecutive image frames can easily change, causing abrupt changes in the overall brightness of the locally displayed image.

[0083] To maintain the stability of grayscale mapping between consecutive image frames, this embodiment uses digital thresholds corresponding to the upper and lower limits of detector calibration temperature measurement to limit the mapping range, and adjusts the grayscale mapping gain based on the average global pixel digital value of the current frame. The digital thresholds corresponding to the upper and lower limits of calibration temperature measurement can be obtained from the calibration data stored in non-volatile memory after the infrared detector is calibrated at the factory. When this system performs calibration, under the condition that the integration time, gain parameters, and operating temperature of the infrared detector remain at the set state, a standard radiation source with a known temperature is brought to the preset lower and upper temperature limits, respectively. After the temperature and infrared detector output stabilize, multiple images are acquired, and after excluding detector dead pixels, the pixel digital value of the stable imaging area is statistically analyzed. The digital thresholds corresponding to the lower limit of calibration temperature measurement are determined based on the statistical results. And the digital threshold corresponding to the calibrated upper limit of temperature measurement The calibration results and Write to non-volatile memory. Subsequent grayscale mapping processing includes calibration range mapping, gain adjustment based on frame average, and 8-bit grayscale range limiting.

[0084] The core processing circuit first reads the digital threshold corresponding to the calibrated upper temperature measurement limit from the detector's non-volatile memory. and the digital threshold corresponding to the calibrated lower limit of temperature measurement And determine the width of the significant digit range based on the two. and the digital quantity at the midpoint of the calibration interval .

[0085] Average global pixel digital value in the current frame Able to process preprocessed image matrices The result is obtained by summing all valid pixels and dividing by the total number of valid pixels; even with limited hardware resources, the preprocessed image matrix can be processed according to preset row and column intervals. Perform equal-interval sampling, and obtain the average global pixel digital value of the current frame based on the accumulated results of the sampled pixels. When using equal-interval sampling, each image frame uses the same sampling position and sampling interval.

[0086] When the infrared detector is equipped with a calibration coefficient that converts pixel digital values ​​to temperature values, the core processing circuit can preprocess the image matrix before grayscale mapping. Converted to temperature data; however, this embodiment generates basic mapped grayscale values. At that time, the digital threshold corresponding to the upper and lower limits of temperature measurement is directly calibrated using the detector, without the necessary step of temperature conversion.

[0087] The mapping gain coefficient is calculated using the mapping gain coefficient calculation formula. The formula for calculating the mapping gain coefficient is: ; in, This represents the mapping gain coefficient, which is dimensionless and used to adjust the grayscale mapping gain of the current image frame. This represents the slope coefficient of the gain curve, expressed as the reciprocal of the digital value, and is used to adjust the mapping gain coefficient. With the average global pixel digital value of the current frame Digital quantity at the midpoint of the calibration interval The rate of change of the deviation between them; This represents the average digital value of all pixels in the current frame, expressed in digital units. This indicates the digital value at the midpoint of the calibration interval, expressed in digital units. This indicates the digital threshold corresponding to the calibrated upper limit of temperature measurement, in digital quantity. This indicates the digital threshold corresponding to the lower limit of the calibrated temperature measurement, in digital units; the digital value at the midpoint of the calibration interval. according to calculate; Represents the natural constant.

[0088] In this embodiment, the slope coefficient of the gain curve The preferred setting is 0.001; the slope coefficient of the gain curve varies depending on the width of the detector's effective digital range. It can also be set to to between.

[0089] Subsequently, the grayscale value of the basic mapping is calculated using the basic mapping grayscale value calculation formula. The formula for calculating the basic mapping grayscale value is: ; in, Representing coordinates The base mapping grayscale value is an integer in the range of 0 to 255; Represents the coordinates in the preprocessed image matrix The pixel value at that location, expressed in digital units; This indicates the digital threshold value corresponding to the calibrated lower limit of temperature measurement, in digital quantity. Indicates the width of the significant digit range, in units of digits, and follows the... calculate; Indicates the mapping gain coefficient; This represents the rounding function; This represents a cutoff function that restricts the calculation results to a closed interval between 0 and 255.

[0090] when The digital threshold corresponding to the calibrated lower limit of temperature measurement At that time, the basic mapping grayscale value After truncation, the value is set to 0; when the calculation result is greater than 255, the base mapping grayscale value is... Take 255 after truncation.

[0091] In implementations using field-programmable gate arrays (FPGAs), the exponential function in the mapping gain coefficient calculation formula is implemented using an exponential function lookup table pre-written into on-chip read-only memory, or using a piecewise linear approximation. The exponential function lookup table is implemented using... The quantization result is used as input, and the corresponding exponential function approximation is used as output.

[0092] Basic mapping grayscale value Subtraction, multiplication, division, rounding, and truncation operations during the calculation process are performed using a pipelined fixed-point arithmetic circuit. Pipeline registers are set between each arithmetic stage, enabling the core processing circuit to receive a new preprocessed pixel digital value in each pixel clock cycle after the pipeline reaches a stable state.

[0093] S202, the dual-path differential image enhancement and scheduling module processes the base mapped grayscale values. Perform local detail enhancement to generate locally enhanced grayscale values ​​for local display. .

[0094] The locally displayed image is used by production line personnel to observe the outline and local thermal radiation changes of the target under inspection. To improve the recognizability of grayscale edges, the core processing circuit calculates the underlying mapped grayscale values. The grayscale gradient magnitude is calculated, and the sharpening weight is determined based on the grayscale gradient magnitude. When the grayscale gradient magnitude is small, the sharpening weight is limited by the noise suppression parameter to reduce the degree to which random noise in the low grayscale gradient region is synchronously amplified; when the grayscale gradient magnitude is large, the sharpening weight is increased accordingly to enhance the local details near the contour of the target to be inspected.

[0095] The gray-level gradient magnitude is calculated using the formula for calculating gray-level gradient magnitude. The formula for calculating the grayscale gradient magnitude is: ; in, Representing coordinates The grayscale gradient magnitude at a given location, expressed in grayscale levels; Representing coordinates The base mapping grayscale value at the location; Representing coordinates The base mapping grayscale value at the location; Representing coordinates The base mapping grayscale value at the location; Representing coordinates The base mapping grayscale value at the location; This represents the absolute value function.

[0096] When the coordinates of adjacent pixels exceed the image boundary, an edge pixel symmetry mapping method is used to determine the pixel value outside the boundary. Specifically, for the horizontal coordinates that exceed the left or right boundary, a horizontal coordinate symmetrical to the corresponding position inside the boundary is used; for the vertical coordinates that exceed the upper or lower boundary, a vertical coordinate symmetrical to the corresponding position inside the boundary is used.

[0097] Sharpening weights are calculated using the sharpening weight calculation formula. The formula for calculating the sharpening weight is: ; in, Representing coordinates The sharpening weight at the point is dimensionless. Representing coordinates The grayscale gradient magnitude at a given location, expressed in grayscale levels; This represents a noise suppression parameter, expressed in gray levels, used to limit the sharpening weight in low gray-level gradient regions.

[0098] In this embodiment, the noise suppression parameter The optimal setting is 4, and it can be adjusted within a range of 2 to 8 gray levels. (Noise suppression parameter) The sharpening weights for low-grayscale gradient regions are inversely adjusted, and at the same grayscale gradient magnitude... Increase the noise suppression parameter. Able to reduce sharpening weight This reduces the amplification of random noise in flat areas. Therefore, when the noise amplitude of the infrared detector increases at actual operating temperatures, increasing the noise suppression parameter is crucial. When the noise amplitude is small, the noise suppression parameter can be reduced. To preserve more local details.

[0099] Noise suppression parameters During the system calibration phase, a uniform temperature background image is used to determine the background. Specifically, after the infrared detector reaches its actual operating temperature, multiple uniform temperature background images are acquired, and the noise suppression parameters are gradually changed in a preset step size within a range of 2 to 8 gray levels. The grayscale fluctuations in flat regions after local detail enhancement are statistically analyzed. When a candidate noise suppression parameter ensures that the grayscale fluctuations in the enhanced flat regions do not exceed a preset allowable range, while maintaining discernible grayscale changes in standard edge regions, this candidate value is determined as the noise suppression parameter. When multiple candidate values ​​meet the criteria, the smaller candidate value is preferred. Noise suppression parameters. It can be calibrated by acquiring a uniform temperature background image and statistically analyzing the grayscale difference between adjacent pixels.

[0100] The four-neighbor Laplace detail value is calculated using the formula for calculating the four-neighbor Laplace detail value. The formula for calculating the Laplace detail value of the four-neighbor area is: ; in, Representing coordinates The Laplacian detail value of the four neighboring pixels at a given location, expressed in gray levels; its positive or negative sign indicates the direction of the local gray level difference between the current pixel and its four adjacent pixels (upper, lower, left, and right).

[0101] The local enhanced grayscale value is calculated using the local enhanced grayscale value calculation formula. The formula for calculating the local enhanced grayscale value is: ; in, Representing coordinates The local enhanced grayscale value at that location is an integer in the range of 0 to 255; Representing coordinates The base mapping grayscale value at the location; This represents the sharpening intensity coefficient, which is dimensionless. Representing coordinates Sharpening weight at the location; Representing coordinates Laplace detail value of the four neighboring regions at the location; This represents a cutoff function that restricts the calculation results to a closed interval between 0 and 255.

[0102] In this embodiment, the sharpening intensity coefficient The preferred setting is 1.0, and it can be adjusted within the range of 0.5 to 1.5 depending on the level of detail enhancement of the locally displayed image.

[0103] In implementations employing field-programmable gate arrays (FPGAs), the core processing circuitry uses at least two levels of row buffers and shift registers connected to the row buffers to construct adjacent pixel windows. Simultaneously, when the current pixel enters the computation pipeline, it acquires... , , and .

[0104] grayscale gradient magnitude Sharpening weights Laplace detail values ​​of the four neighboring regions and local enhanced grayscale values The calculation is performed sequentially using a fixed-point pipeline method. For pixels in the first row, last row, first column, and last column of the image, the aforementioned edge pixel symmetric mapping method is used to supplement the values ​​of adjacent pixels.

[0105] After completing the local detail enhancement of an image frame, the core processing circuit will locally enhance the grayscale values. The assembled locally enhanced image frames are written to the local display frame buffer, and the frame-level timestamps are added. The frame number and target number are written into the frame header of the corresponding image frame.

[0106] S203, the dual-path differential image enhancement and scheduling module processes the base mapping grayscale values. Perform local smoothing to generate network-enhanced grayscale values ​​for network video encoding. .

[0107] Network-side hardware video encoders are highly sensitive to high-frequency random changes in continuous images. As high-frequency random changes increase, the encoder needs to use more encoded data to represent pixel changes, leading to an increase in the amount of data after compression. To reduce random noise in flat areas and minimize contour blurring caused by averaging across areas with significant gray-level differences, this embodiment determines smoothing weights based on the gray-level difference between neighboring pixels and the center pixel, as well as the local pixel variance.

[0108] For coordinates At the center pixel, the core processing circuit constructs a 5×5 local neighborhood window centered on that center pixel. Local neighborhood window This includes 25 pixels, including the center pixel. For image boundary locations, the same edge pixel symmetric mapping method as in step S202 is used to supplement the neighboring pixels that extend beyond the image boundary.

[0109] The average value of surrounding pixels is calculated using the formula for calculating the average value of surrounding pixels. The formula for calculating the average value of surrounding pixels is: ; in, This represents the average value of surrounding pixels within the current 5×5 local neighborhood window, expressed in gray levels. This represents the total number of pixels within a local neighborhood window, and is dimensionless. In this embodiment... ; Represented by coordinates A 5×5 local neighborhood window centered on the target; and These represent the x and y coordinates of a pixel within the local neighborhood window, respectively. Representing coordinates The base mapping grayscale value at the location; Indicates a local neighborhood window Summate all pixels within the range.

[0110] The local pixel variance is calculated using the formula for calculating local pixel variance. The formula for calculating local pixel variance is: ; in, Represented by coordinates The variance of local pixels within the local neighborhood window centered on the center is expressed in squares of gray levels. Representing coordinates The grayscale difference between the base mapped grayscale value and the average grayscale value of the surrounding pixels.

[0111] Calculate the smoothing weights using the smoothing weights calculation formula. The formula for calculating the smoothing weight is: ; in, Represents the coordinates within the local neighborhood window The smoothing weights at the point are dimensionless. This represents the grayscale difference between neighboring pixels and the center pixel, expressed in grayscale levels. This represents the square of the grayscale difference, expressed in squares of grayscale levels. Represents the local pixel variance, expressed as the square of the gray levels; This represents the minimum zero-prevention constant, expressed in squares of gray levels, used to prevent the denominator from being zero when the variance of local pixels is zero. Represented by natural constant An exponential function with base 0.

[0112] in, and Both have the dimension of the square of the gray level, and the ratio of the two is dimensionless. Therefore, the exponent term of the exponential function in the smoothing weight calculation formula is dimensionless.

[0113] In this embodiment, the minimum zero constant is used. The preferred setting is the square of 1 gray level. In practical implementation, a minimal zero constant is used. It can be set to a positive number that is not less than a minimum quantization unit based on the precision of fixed-point arithmetic.

[0114] The smaller the grayscale difference between neighboring pixels and the center pixel, the lower the smoothing weight. The larger the value, the greater the grayscale difference between neighboring pixels and the center pixel, and the greater the smoothing weight. The smaller the value, the greater the impact. Therefore, pixels with similar local gray levels have a significant influence on the output, while pixels with large gray level differences have a limited impact on the output.

[0115] Calculate the network-enhanced grayscale value using the network-enhanced grayscale value calculation formula. The formula for calculating the enhanced grayscale value of the network is: ; in, Representing coordinates The network-enhanced grayscale value at the location is an integer in the range of 0 to 255; Representing coordinates The product of the base mapping gray value and the corresponding smoothing weight; This represents the sum of all weighted gray values ​​within the local neighborhood window; This represents the sum of all smoothing weights within the local neighborhood window; This represents the rounding function; This represents a cutoff function that restricts the calculation results to a closed interval between 0 and 255. It represents the multiplication operation.

[0116] Because the local neighborhood window includes the center pixel, and the smoothing weights corresponding to the center pixel... Since the sum of all smoothing weights is 1, it is not zero.

[0117] In the implementation using a field-programmable gate array (FPGA), the core processing circuit constructs a 5×5 local neighborhood window using a four-level row buffer and shift registers corresponding to each row buffer. The core processing circuit calculates the average value of surrounding pixels using a sliding window accumulation method. and local pixel variance .

[0118] When the local neighborhood window moves one pixel horizontally, the column of pixel data that has moved out of the local neighborhood window is subtracted from the current accumulated result, and the column of pixel data that has just entered the local neighborhood window is added to reduce redundant accumulation operations. The squared terms and accumulated terms in the local pixel variance calculation are stored in an extended bit-width register, which can accommodate at least the sum of squares of 25 8-bit pixels.

[0119] The exponential function in the smoothing weight calculation formula is implemented using an exponential function lookup table or a piecewise linear approximation. The input to the exponential function lookup table is the quantized value of the squared gray-level difference between neighboring pixels and the center pixel after local pixel variance normalization, and the output is the corresponding smoothing weight.

[0120] The core processing circuit accumulates the weighted grayscale values ​​and smoothing weights within the local neighborhood window. After completing the data input for the 5×5 local neighborhood window, it performs division, rounding, and truncation operations to obtain the network-enhanced grayscale values. .

[0121] After completing the local smoothing process for an image frame, the core processing circuit will use a network to enhance the grayscale values. The network-enhanced image frames are written into the network-coded image buffer, and the frame-level timestamps are added. The frame number and target number are written into the frame header of the corresponding image frame.

[0122] Meanwhile, the core processing circuit will process the pre-processed image matrix that has not undergone local smoothing. The images are continuously fed into the independent defect detection path according to their frame numbers. The independent defect detection path does not use the network buffer read pointer position of the network-coded image buffer. Cross-frame read operations in the network coding path do not change the image frame processing order of the defect detection path.

[0123] S204, the dual-path differentiated image enhancement and scheduling module performs cross-frame reading control on the network coding path according to the image frame backlog status of the network coding image buffer.

[0124] The network-encoded image buffer uses a circular buffer structure, divided into... Each image frame has a storage slot of equal length, and the capacity of each slot is not less than the data length of a single network-enhanced image frame. The sum of the frame header length and the frame value. Each image frame storage slot has a frame validity flag set, and a frame-level timestamp is stored in the frame header. Frame number and target number.

[0125] The core processing circuit sets the frame validity flag of an image frame storage slot to valid only after a complete network-enhanced image frame and its frame header have been written into the image frame storage slot. Image frame storage slots with invalid frame validity flags can be rewritten by subsequent image frames.

[0126] Network cache write pointer position This indicates the location of the image frame storage slot where a complete image frame was most recently written; the network cache read pointer location. This indicates the location of the image frame storage slot where the image frame was most recently read or discarded. Network cache write pointer location. With network cache read pointer position Each is incremented cyclically in units of image frame storage slots.

[0127] In an implementation that reserves one free image frame storage slot, when the network cache write pointer position is... With network cache read pointer position If the values ​​are the same, determine that the network-encoded image buffer is empty; if the network buffer write pointer position is... Next image frame storage slot and network cache read pointer position If the same frame is present, the network-coded image buffer is determined to be full. Therefore, the maximum number of unread image frames that the network-coded image buffer can hold is... .

[0128] When the image processing write circuit and the network-side data transport circuit operate in different clock domains, the network cache write pointer position... and network cache read pointer position Gray code cross-clock domain synchronization mechanism is used for transmission.

[0129] Specifically, the transmitting clock domain converts the binary pointer to a Gray code pointer, and the receiving clock domain receives the Gray code pointer through at least two levels of synchronization registers and converts the synchronized Gray code pointer back to a binary pointer. Since only one binary bit changes between adjacent Gray code pointers, the risk of bit inconsistencies when multiple pointers are transmitted across clock domains is reduced.

[0130] Independent data transfer circuits write pointer positions based on network cache. With network cache read pointer position The difference in cyclic positions between them is used to calculate the number of backlogged frames in the network-encoded image buffer. .

[0131] The number of backlogged frames is calculated using the formula for calculating backlogged frames. The formula for calculating the number of backlogged frames is: ; in, This indicates the number of buffered backlog frames, in image frames. Indicates the location of the network cache write pointer, in units of image frame storage slots; Indicates the network cache read pointer position, in image frame storage slots; This indicates the total number of image frame storage slots contained in the network-coded image buffer, in units of image frame storage slots. This indicates the modulo operation.

[0132] When the number of cached backlogged frames When the number of frames is small, the network-side data transport circuit reads the network-enhanced image frames sequentially according to the image frame writing order; when the number of frames in the buffer is small... When an increase occurs, the core processing circuitry adjusts the cache based on the number of backlogged frames. and congestion trigger ratio Calculate the adaptive frame skipping step size .

[0133] The adaptive frame skipping step size is calculated using the adaptive frame skipping step size calculation formula. The formula for calculating the adaptive frame skipping step size is: ; in, This represents the adaptive frame skipping step size, a dimensionless parameter used to indicate the number of image frame storage slots corresponding to one forward movement of the network cache read pointer; This indicates the number of buffered backlog frames, in image frames. This represents the congestion triggering ratio coefficient, which is dimensionless and used to determine the congestion triggering ratio of the network-coded image buffer. This indicates the total number of image frame storage slots contained in the network-coded image buffer; This represents the floor function.

[0134] In this embodiment, the congestion triggering ratio coefficient Rs is preferably set to 0.3, and can be adjusted within the range of 0.2 to 0.4. The smaller the congestion triggering ratio coefficient Rs, the earlier the system enters the adaptive frame skipping step calculation, and the earlier it performs actual cross-frame reading when the buffer continues to back up; the larger the congestion triggering ratio coefficient Rs, the more backed image frames are allowed to be retained in the network-coded image buffer before performing cross-frame reading.

[0135] During system debugging, the congestion trigger ratio coefficient Rs was initially set to 0.3, and the network output bandwidth was gradually reduced under the condition that the infrared detector continuously input images at the set frame rate. When the processing speed of the network-side hardware video encoder or the network output bandwidth is consistently lower than the processing capacity required for image writing, causing the number of backlogged frames to continuously increase and approach the maximum number of unread image frames in the network encoded image buffer, the congestion trigger ratio coefficient Rs was adjusted towards 0.2 to release expired network image frames in advance. When there are only short-term bandwidth fluctuations on the network side, and the number of backlogged frames can decrease on its own after the network recovers, but the system frequently enters the congestion scheduling state, the congestion trigger ratio coefficient Rs was adjusted towards 0.4 to reduce unnecessary cross-frame readings caused by short-term fluctuations.

[0136] When adaptive frame skipping step size When the value equals 1, the network-side data transport circuit reads the network cache read pointer position. The next unread image frame does not discard intermediate image frames; the adaptive frame skipping step size is used. When the value is greater than 1, the network-side data transport circuit crosses... The image frames that are stacked in the middle of the frame are read, and the image frames that are crossed are read.

[0137] Therefore, the number of cached backlog frames achieve At that time, the system calculates the adaptive frame skipping step size according to the above formula. Only when the calculated adaptive frame skipping step size is obtained... Only when the value is greater than 1 will the actual cross-frame read operation be performed.

[0138] The actual read address on the network side is calculated using the formula for calculating the actual read address on the network side. The formula for calculating the actual read address on the network side is: ; in, This indicates the actual address read from the network side, in bytes. Indicates the network cache read pointer position The base address of the corresponding image frame storage slot, in bytes; Indicates the adaptive frame skipping step size, a dimensionless parameter used to represent the number of image frame storage slots corresponding to the network cache read pointer moving forward from the current read position; when Read the next unread image frame after the current network cache read pointer position. Time span The image frame is compressed in the middle of the frame; This indicates the length of the image frame storage slot corresponding to a single network-enhanced image frame, in bytes. Indicates the network cache read pointer position The address offset between the target image frame storage slot and the target image frame storage slot, in bytes.

[0139] The network-encoded image buffer also has a start address and an end address. The core processing circuit obtains the actual read address from the network side. Then, the actual address read from the network side. Perform ring address wraparound processing.

[0140] When the network side actually reads the address When the address is equal to or greater than the end address of the network-encoded image buffer, the actual address is read from the network side. Subtract the total byte length of the network-encoded image buffer from the input; the actual read address on the network side after the subtraction is... If the address still falls outside the valid address range of the network-encoded image buffer, continue subtracting the total byte length of the network-encoded image buffer until the actual address read by the network side is reached. It is located between the start address and the end address of the cache.

[0141] The network-side data transfer circuit reads the pointer position from the network cache. and adaptive frame skipping step size The valid flags of the image frame storage slots to be traversed are checked one by one according to the direction of the read pointer. For the network cache read pointer position... Afterwards, for each image frame storage slot that is crossed before the target reads the image frame storage slot, the network-side data transport circuit clears its frame validity flag and marks it as rewritable.

[0142] When the network side actually reads the address When the frame validity flag of the corresponding target image frame storage slot is valid, the network-side data transfer circuit reads the complete network-enhanced image frame from the image frame storage slot and sends it to the network-side hardware video encoder. After reading, the frame validity flag of the target image frame storage slot is cleared, and the network buffer read pointer position is reset. Update to the target image frame storage slot.

[0143] When the network side actually reads the address When the frame validity flag of the corresponding target image frame storage slot is invalid, the network-side data transport circuit continues to check subsequent image frame storage slots along the read pointer's forward direction until it finds a frame validity flag that is valid and the frame sequence number is not greater than the network buffer write pointer position. The image frame storage slot corresponding to the latest complete image frame sequence number; if no matching image frame storage slot exists, the current network cache read pointer position is maintained. And wait for new, complete network-enhanced image frames to be written.

[0144] The cross-frame read operation in the network encoding path only clears the frame validity flag corresponding to the discarded image frame in the network encoding image buffer; it does not delete the image frame in the local display frame buffer, nor does it change the image frame read order in the independent defect detection path. The actual read address on the network side... It is not passed to the sorting control path.

[0145] See attached document Figure 5 This corresponds to step S3 mentioned above. In this embodiment, the specific working process of the local defect detection and output execution module includes the following sub-steps: S301, the local defect detection and output execution module reads the image frames in the local display frame buffer and the network encoded image buffer respectively, and performs local display output and network encoded output.

[0146] For the local display path, the core processing circuit reads the locally enhanced grayscale values ​​from the local display frame buffer in frame sequence. The local enhanced image frames are composed of pixel data that are converted according to the input format required by the chip and sent through the high-definition multimedia interface.

[0147] In one embodiment, when the high-definition multimedia interface transmitting chip adopts the RGB pixel data input format, the core processing circuit will locally enhance the grayscale value. The red, green, and blue channels are written separately to give each color channel the same grayscale value, thus generating a grayscale display image. When the high-definition multimedia interface transmitting chip supports the luminance component input format, the local enhanced grayscale value is used. As the luminance component input.

[0148] The core processing circuit generates pixel valid signals, line synchronization signals, and frame synchronization signals according to the preset image resolution and display refresh rate, and sends the converted pixel data and corresponding video output timing to the high-definition multimedia interface transmitter chip, which then outputs the data to the local monitor.

[0149] For the network coding path, the core processing circuit reads the address from the network side. The system reads complete network-enhanced image frames from the network-encoded image buffer and sends the network-enhanced image frames, along with their resolution, pixel format, and frame rate parameters, to the network-side hardware video encoder. The network-side hardware video encoder then compresses and encodes the network-enhanced image frames according to a preset video coding standard.

[0150] The local display frame buffer and the network-encoded image buffer are each configured with independent data transfer channels. When both data transfer channels simultaneously request access to the storage buffer, the storage control circuit reserves the minimum storage access bandwidth required for continuous output of the display interface for the local display data transfer channel and prioritizes responding to the local display data transfer request; after satisfying the local display output timing, the remaining storage access bandwidth is used for network-encoded data transfer.

[0151] Through the aforementioned storage access control, when the processing latency of the network-side hardware video encoder increases or the network encoding path performs cross-frame reading, the reading order of locally displayed image frames and the video output timing are not changed.

[0152] The independent defect detection path is configured with the network cache read pointer position. Independent detection frame read pointers are used to continuously read the preprocessed image matrix according to the frame number. Therefore, the network side actually reads the address. The changes do not alter the image frame processing order of the independent defect detection pathway.

[0153] The video output timing of the high-definition multimedia interface and the data encapsulation of the network-side hardware video encoder are performed in accordance with the corresponding interface specifications and video encoding standards.

[0154] S302, The local defect detection and output execution module executes the output based on the preprocessed image matrix. Extract abnormal pixels from the target to be inspected and calculate the area of ​​candidate connected components for defects.

[0155] The core processing circuit reads the preprocessed image matrix corresponding to the current target to be detected. And its frame header information, and call the detector bad spot location matrix. When the detector's bad spot location matrix When the value of the corresponding coordinate is 1, the coordinate is excluded from the set of valid pixels for defect detection, thus preventing the inherent bad pixels of the infrared detector from being identified as defects in the target to be inspected.

[0156] Independent defect detection pathways use preprocessed image matrices without network-side local smoothing. As a data source for detection, network-enhanced grayscale values ​​are not used. As the sole data source for detection.

[0157] The core processing circuit determines the image frame corresponding to the current target based on the target number, and then processes the image in the preprocessed image matrix. The region of interest (ROI) of the target to be inspected is extracted. The ROI of the target to be inspected is used to exclude the influence of the conveying mechanism, fixtures, and background areas on the statistics of abnormal pixels.

[0158] For production lines where the position of the target under inspection is fixed in the image or the positional offset is within a preset allowable range, the region of interest (ROI) of the target under inspection uses a pre-stored fixed coordinate range. During the system debugging phase, infrared images of multiple targets under inspection are continuously acquired under normal operating conditions of the conveyor mechanism, ensuring that the actual position of the target under inspection covers different positions within the allowable range of the conveyor mechanism's mechanical positioning. The minimum bounding rectangle corresponding to the outer contour of each target under inspection is determined, and the minimum x-coordinate, maximum x-coordinate, minimum y-coordinate, and maximum y-coordinate of the minimum bounding rectangle in all calibration images are statistically analyzed. The joint envelope region formed by these coordinates is used as the base region for the fixed coordinate range.

[0159] Further, through repeated transport tests of the target under inspection, the actual repetitive positioning deviation is statistically analyzed. Based on the actual imaging ratio, this repetitive positioning deviation is converted into a corresponding pixel offset, and the corresponding number of pixels are added outside the boundary of the base region to form a final fixed coordinate range. This ensures that the target under inspection remains within a fixed region of interest even when moving within the normal mechanical positional deviation range.

[0160] For production lines where the position of the target to be inspected is offset in the image, the core processing circuit first uses a grayscale threshold segmentation method to initially separate the target to be inspected from the background, then uses a contour extraction method to determine the outer contour of the target to be inspected, and determines the minimum bounding rectangle based on the outer contour. The image area covered by the minimum bounding rectangle is taken as the region of interest of the target to be inspected.

[0161] When there are multiple separate candidate regions in the grayscale threshold segmentation result, the candidate region corresponding to the target to be inspected can be selected based on the area, center position and degree of conformity with the preset target size range of the candidate region.

[0162] After determining the region of interest of the target to be inspected, the core processing circuit traverses the matrix of locations of undetected dead pixels within that region. The effective pixels are masked, and the average digital value and standard deviation of the effective pixel digital value are calculated.

[0163] In the infrared detector output configuration used in this embodiment, the pixel digital quantity output by the infrared detector increases with the increase of the detected thermal radiation intensity in the corresponding area. When the target under inspection undergoes the same thermal excitation and is under the same acquisition conditions, an abnormal area in which the pixel digital quantity of a local area of ​​the target under inspection deviates significantly in a positive direction relative to the overall digital quantity distribution of the normal area of ​​the target under inspection is defined as a high-temperature defect area; an abnormal area in which the pixel digital quantity of a local area of ​​the target under inspection deviates significantly in a negative direction relative to the overall digital quantity distribution of the normal area of ​​the target under inspection is defined as a low-temperature defect area.

[0164] Among them, high-temperature defects are present in the preprocessed image matrix. In the case of a medium-temperature defect, the digital value of local pixels is significantly higher than the overall digital value of the effective pixels of the target under inspection; in the case of a low-temperature defect, the digital value of local pixels is significantly lower than the overall digital value of the effective pixels of the target under inspection.

[0165] For high-temperature defects, the sum of the products of the average digital value of effective pixels, the high-temperature sensitivity coefficient, and the standard deviation of the digital value of effective pixels is used as the high-temperature anomaly threshold. Effective pixels with a digital value not less than the high-temperature anomaly threshold are marked as high-temperature anomaly pixels.

[0166] For low-temperature defects, the product of the low-temperature sensitivity coefficient and the standard deviation of the effective pixel digital value is used as the low-temperature anomaly threshold. Effective pixels with a digital value not greater than the low-temperature anomaly threshold are marked as low-temperature anomaly pixels.

[0167] For bidirectional defect detection mode that simultaneously detects high temperature and low temperature anomalies, valid pixels that meet the high temperature anomaly threshold or the low temperature anomaly threshold are marked as product abnormal pixels.

[0168] The high-temperature sensitivity coefficient and low-temperature sensitivity coefficient are determined during the system calibration phase. During calibration, images of defect-free standard samples and standard samples with known high-temperature or low-temperature defects are acquired respectively. The abnormal pixel detection results under different sensitivity coefficients are statistically analyzed, and the sensitivity coefficient that can detect known defects and whose number of false detection pixels in defect-free standard samples does not exceed a preset allowable value is selected. The calibration results are written to non-volatile memory.

[0169] The core processing circuit generates a binary anomaly image based on the abnormal pixels of the product. In the binary anomaly image, the binary state corresponding to the abnormal pixel of the product is set to 1, and the binary state corresponding to the other pixels is set to 0.

[0170] For targets whose minimum defect size is greater than or equal to the size of the structural element used, the local defect detection and output execution module uses binary morphological opening operations to remove isolated noise. The binary morphological opening operation uses a 3×3 rectangular structural element, performing erosion first, followed by dilation.

[0171] When the minimum defect size that the product can detect is less than the coverage area of ​​a 3×3 rectangular structuring element, a full 3×3 morphological opening operation is not performed. Instead, only a single isolated anomalous pixel that is not connected to the surrounding anomalous pixels is removed, or a structuring element with a size smaller than the minimum defect size to be detected is used to avoid morphological erosion operations from eliminating real small-area defects.

[0172] After noise filtering is completed, the local defect detection and output execution module uses the eight-connected-domain labeling method to divide adjacent abnormal product pixels in the horizontal, vertical or diagonal directions into the same defect candidate pixel connected domain.

[0173] The core processing circuit counts the total number of valid pixel nodes in each defect candidate pixel connected region and uses this total number of valid pixel nodes as the defect candidate connected region area. When there is no defect candidate pixel connected region in the current image frame, the maximum defect candidate connected region area of ​​the current image frame is set to 0.

[0174] For multiple consecutive image frames associated with the same target number, the core processing circuit obtains the area of ​​the largest candidate connected region of defects in each image frame, and takes the maximum value in all associated image frames as the area of ​​the largest candidate region of defects of the target to be inspected.

[0175] When the preset acquisition time window determined in step S104 ends, or when the sensor trigger signal of the next target to be inspected is detected and the previous target to be inspected has left the image acquisition area, the core processing circuit ends the area merging processing of the image frame corresponding to the current target number and generates a final defect judgment result corresponding to the current target number.

[0176] Only one final defect determination result is written for the same target number to avoid the same target appearing in multiple image frames and triggering the sorting and rejection action repeatedly.

[0177] S303, the local defect detection and output execution module generates sorting judgment results based on the area of ​​the target maximum defect candidate region, and executes sorting and rejection actions based on the counting control push rod device of the conveyor encoder.

[0178] The core processing circuit compares the area of ​​the largest candidate defect region of the target with a pre-defined defect area threshold. When the area of ​​the largest candidate defect region of the target is greater than the defect area threshold, the corresponding target to be inspected is determined to be an unqualified target; when the area of ​​the largest candidate defect region of the target is not greater than the defect area threshold, the corresponding target to be inspected is determined to be a qualified target.

[0179] The defect area threshold is determined through standard sample calibration. During calibration, under the same optical lens, imaging distance, infrared detector working status, and image processing parameters as the actual inspection station, infrared images of a defect-free standard sample, a standard sample with a known maximum permissible defect, and a standard sample with a known minimum non-conforming defect are acquired respectively, and the corresponding target maximum defect candidate area is obtained according to the defect detection process in step S302.

[0180] When there is a separable interval between the target maximum defect candidate area corresponding to the standard sample with the known maximum permissible defect and the target maximum defect candidate area corresponding to the standard sample with the known minimum non-conforming defect, the area boundary value that can make the standard sample with the maximum permissible defect qualified and the standard sample with the minimum non-conforming defect unqualified is selected within the separable interval as the defect area threshold.

[0181] The point spread function characteristics of the optical lens, the pixel size of the infrared detector, and the imaging magnification are used to determine the effective imaging scale of the actual defect in the infrared image, and to verify whether the defect area threshold is lower than the effective pixel area that the optical system can stably resolve. The maximum permissible defect size and the minimum unacceptable defect size of the product are used to determine the defect size range of the standard sample used for calibration. The finally determined defect area threshold is written to non-volatile memory.

[0182] During the defect area threshold calibration process, under the same lens focal length, target distance, infrared detector operating temperature and image processing parameters as the actual inspection station, infrared images of a standard point heat source, a defect-free standard sample, a standard sample with the known maximum allowable defect and a standard sample with the known minimum non-conforming defect were collected respectively.

[0183] First, the area of ​​the point diffusion region of the optical system is determined based on the pixel range covered by the point heat source energy in the standard point heat source image. Second, the defect detection processing step S302 is performed on standard samples with known maximum permissible defects and known minimum non-conforming defects respectively to obtain the corresponding target maximum defect candidate region area.

[0184] Based on the difference in the candidate region area of ​​the target maximum defect between a standard sample with the known maximum permissible defect and a standard sample with the known minimum non-conforming defect, an area boundary value that can distinguish between qualified and non-conforming targets is determined, and this area boundary value is written into non-volatile memory as a defect area threshold.

[0185] In one embodiment, the defect area threshold is set in the range of 5 to 15 pixel area units. The actual defect area threshold used is based on the calibration results of the corresponding product and optical system.

[0186] When the target to be inspected is determined to be unqualified, the core processing circuit will input the target number and frame-level timestamp. The maximum defect candidate area of ​​the target, the defect judgment result, and the conveyor encoder count value recorded when the target to be inspected enters the image acquisition area are written into the sorting control queue.

[0187] The conveying distance between the image acquisition position and the push rod action position is measured during the system installation and commissioning phase, and converted into encoder counting offset based on the conveying distance corresponding to each counting pulse of the conveying encoder. The encoder counting offset is written into the control parameter storage area.

[0188] The core processing circuit reads the sorting control queue in order of target number and continuously acquires the current count value of the conveyor encoder. When the current count value of the conveyor encoder reaches the sum of the conveyor encoder count value when the corresponding target enters the image acquisition area and the encoder count offset, it is determined that the target has reached the push rod action position.

[0189] The core processing circuit then controls the corresponding pins of the general-purpose input / output interface to switch their level states and maintains a preset drive pulse width, causing the drive relay to engage and driving the push rod device to perform a sorting and rejection action, pushing the target to be inspected corresponding to the target number into the defective product area.

[0190] The preset drive pulse width is set based on the pull-in time of the drive relay, the time required for the push rod device to complete the push-out action, and the running speed of the conveyor mechanism, and is stored in the control parameter storage area.

[0191] When the target to be inspected is determined to be a qualified target, no rejection instruction is written to the sorting control queue, and the general input / output interface remains in a non-rejection state, allowing the target to be inspected to pass through the sorting position normally.

[0192] After the pusher device performs the sorting and rejection action, the core processing circuit reads the feedback signal output by the pusher positioning sensor. When the pusher positioning feedback signal is detected within the preset feedback time, the sorting status of the corresponding target number in the sorting control queue is marked as completed.

[0193] Once the pusher device completes its reset and detects the reset status, the core processing circuit clears the corresponding sorting control queue record. If no pusher positioning feedback signal is detected within the preset feedback time, the core processing circuit outputs a sorting mechanism abnormality signal and retains the corresponding target number and frame-level timestamp. The target maximum defect candidate area and defect determination results are used for fault tracing.

[0194] The preset feedback time is determined based on the drive relay engagement time, push rod action time, and push rod position sensor response time, and is greater than the maximum time from the output drive level to the receipt of the push rod position feedback signal under normal circumstances.

[0195] During the sorting control process, the target number is used to distinguish different targets to be inspected, the encoder count value when the target enters the image acquisition area is used to record the target's initial conveying position, and the encoder count offset is used to represent the conveying distance between the image acquisition position and the push rod's action position. Through the correspondence between these three, the infrared image detection results are kept in sync with the sorting and rejection actions.

[0196] Actual address read on the network side It is only used to select the image frame to be encoded in the network encoded image buffer, and does not participate in the target position calculation of the sorting control queue, nor is it used as a trigger condition for the push rod device.

[0197] In the above data processing, the original two-dimensional digital matrix of the same image frame Preprocessed image matrix Based on the grayscale value The basic grayscale image is composed of locally enhanced grayscale values. Composed of locally enhanced image frames and grayscale values ​​enhanced by the network The network-enhanced image frames are all generated using the same frame-level timestamp. Establish a connection between frame number and target number.

[0198] The local display path outputs locally enhanced image frames consecutively according to frame sequence number; the network encoding path can read the actual address from the network side when there is buffer backlog. Skip expired network-enhanced image frames; independent defect detection pathways process preprocessed image matrices sequentially according to frame number. Therefore, cross-frame reading processing in the network coding path does not change the correspondence between defect detection results and the target to be inspected, nor does it change the target processing order in the sorting control queue.

[0199] Specific application examples: The pixel data bit width output by the infrared detector is set to 14 bits, and the digital threshold corresponding to the upper limit of temperature measurement is calibrated. The digital threshold corresponding to the calibrated lower limit of temperature measurement Width of significant digit range The digital quantity at the midpoint of the calibration interval .

[0200] The preprocessed image matrix is ​​acquired and global statistics are performed to obtain the average global pixel digital value of the current frame. Gain curve slope coefficient The formula for calculating the mapping gain coefficient is: ; For coordinates in the preprocessed image matrix Pixel digital quantity at the location The coordinates are calculated using the basic mapping grayscale value calculation formula. The base mapping gray value at the location The calculation results are then limited to an 8-bit grayscale range of 0 to 255. ; After calculation, the coordinates The base mapping gray value at the location The calculated coordinates The base mapping gray value at the location It serves as a common 8-bit grayscale input for both the local enhancement path and the network smoothing path.

[0201] Set the total number of image frame storage slots corresponding to the network encoding image buffer capacity. Congestion trigger ratio The corresponding congestion trigger threshold is: ; The network cache write pointer position within a certain runtime cycle Network cache read pointer position The formula for calculating the number of cached backlog frames is: ; Due to cache backlog of frames The core processing circuit enters an adaptive frame skipping step size of not less than 12. The calculation process: ; Adaptive frame skipping step size When the value equals 1, the network-side data transport circuit reads the network cache read pointer position. For the next unread image frame, no actual cross-frame reading operation is performed. According to the adaptive frame skipping step size calculation formula used in this embodiment, only the number of buffered backlog frames is considered. The calculated adaptive frame skipping step size is not less than 24. Only when the value is greater than 1 will the actual cross-frame read operation be performed.

[0202] As network-side reads slow down while writes continue to update, the write pointer position is updated to... Read pointer position Maintaining at 30, the number of cached backlog frames increased to Re-executing the adaptive frame skipping step size calculation formula yields: ; Adaptive frame skipping step size Greater than 1, the network-side data transport circuit reads the pointer position along the network cache. The forward direction crosses one intermediate backlogged image frame, reading the network cache read pointer position. The second valid image frame follows. The frame validity flag for the image frame storage slot corresponding to the crossed image frame is cleared. After the target image frame is read, the network buffer read pointer position is... Update to the target image frame storage slot.

[0203] An automated sorting and verification platform was constructed, comprising a tracked conveyor mechanism, infrared detectors, and pneumatic push rod devices. A thermal excitation device was installed upstream of the image acquisition station to apply the same thermal excitation conditions to all metal parts. Metal parts with pre-dimpled pits on their surfaces were used as inspection targets, creating a pixel difference between the pre-dimpled area and the normal surface area that could be captured by the infrared detector. The thermal excitation conditions, pre-dimple size, imaging distance, and infrared detector operating parameters remained constant throughout the same set of tests.

[0204] During testing, the infrared detector's image acquisition frame rate was set to 50Hz, and the conveyor mechanism was controlled to transport the target under inspection through the image acquisition area at a speed of 2.5m / s. A preset acquisition time window was determined based on the actual length of the image acquisition area along the conveying direction, the conveyor mechanism's speed, and the infrared detector's image acquisition frame rate, ensuring that consecutive image frames acquired from the same target within the preset acquisition time window used the same target number. Before testing, the encoder counting offset was calibrated according to the conveying distance from the image acquisition position to the push rod's action position, and the preset drive pulse width was calibrated according to the push rod's action time.

[0205] The dual-channel differentiated image enhancement and scheduling module of the core processing circuit is activated, and a hardware bandwidth controller is connected to the network-side output port. After starting the equipment and ensuring stable transmission on the production line for 10 seconds, the effective communication bandwidth of the network link is gradually reduced via the hardware bandwidth controller to decrease the number of buffered backlogged frames. The frame rate shifts from below 12 frames to the 12-23 frame range and then to above 24 frames, while controlling the number of frames backlogged in the buffer. The number of unread image frames cannot exceed the maximum number that the network-encoded image buffer can hold. After the bandwidth-limited state lasts for 5 seconds, the normal effective communication bandwidth of the network link is restored.

[0206] Record the network cache write pointer position during device operation. Network cache read pointer position This includes the corresponding frame number, frame-level timestamp, network-enhanced image frame, and the locally enhanced image frame corresponding to the same original image frame. It generates the image frame based on the base grayscale data, locally enhanced grayscale data, and network-enhanced grayscale data at the same coordinates within the same image frame. Figure 6 Write pointer position based on network cache and network cache read pointer position Recalculate the number of backlogged frames and adaptive frame skipping step size ,generate Figure 7 .

[0207] Figure 6 The horizontal axis represents the local pixel variance calculated based on the 5×5 local neighborhood of the corresponding coordinates in the base grayscale image, and the vertical axis represents the four-neighbor Laplacian detail value at the same coordinates in the locally enhanced image frame or the network enhanced image frame. Circular scatter points represent the data corresponding to the local enhancement path, and square scatter points represent the data corresponding to the network smoothing path.

[0208] Figure 7 The left vertical axis represents the write pointer position based on the network cache. and network cache read pointer position Calculated number of cached backlog frames The right vertical axis represents the adaptive frame skipping step size. The horizontal dashed line represents the congestion-triggered proportional coefficient. The total number of image frame storage slots contained in the network encoded image buffer A defined congestion trigger threshold. The gray area represents the number of frames backlogged in the buffer. The black needle-like markers only indicate the adaptive frame skipping step size. The image frame position that is greater than 1 and actually performs a cross-frame read operation.

[0209] In the image data collected this time, under the same local pixel variance values ​​of the base grayscale image, the four-neighbor Laplacian detail values ​​corresponding to the local enhancement path have a larger absolute value distribution range, while the four-neighbor Laplacian detail values ​​corresponding to the network smoothing path are mostly distributed near zero. The above distribution indicates that the response amplitude of the local enhancement path to local grayscale changes is greater than that of the network smoothing path, and the two output paths form different local detail features.

[0210] During stable operation, when the number of backlogged frames in the buffer is below the congestion trigger threshold, the read pointer retrieves processed frame data from the network coding path in sequence. When the number of backlogged frames in the buffer... When 12 frames are reached, the system begins to calculate the adaptive frame skipping step size according to the adaptive frame skipping step size calculation formula. When the number of cached backlogged frames When the frame rate is between 12 and 23 frames, the adaptive frame skipping step size When the value is 1, no actual cross-frame read operation is performed; this occurs when the number of frames in the buffer is backlogged. Achieve 24 frames per second or higher and adaptive frame skipping step size When the value is greater than 1, the network-side data transport circuit performs cross-frame reading operations.

[0211] During cross-frame reading, the network-side data transport circuit clears the frame validity flag of the image frame storage slot corresponding to the crossed image frame, restoring the corresponding image frame storage slot to a rewritable state. This operation only applies to the network-encoded image buffer and does not change the reading position of the local display frame buffer, nor does it change the order in which the independent defect detection path processes the preprocessed image matrix according to the frame sequence number.

[0212] Based on the conveyor encoder count value, encoder count offset, and the level transition time of the general input / output interface recorded in the sorting control queue when the target to be inspected enters the image acquisition area, the conveyor encoder count value at the time of the level transition of the general input / output interface is checked to see if it reaches the predetermined trigger count value of the corresponding target to be inspected. The feedback signal from the push rod positioning sensor is used to confirm whether the corresponding sorting rejection action has been completed.

Claims

1. A method for dual-channel enhanced output and defect detection of infrared video, characterized in that, include: Receive pixel data, line synchronization signal and frame synchronization signal output by infrared detector, combine with the detector bad pixel location matrix to reconstruct and repair bad pixels, and generate a preprocessed image matrix; Perform grayscale mapping on the preprocessed image matrix to generate basic mapped grayscale values; Extract the grayscale gradient magnitude and local pixel variance of the base mapping grayscale value, generate local enhanced grayscale value and network enhanced grayscale value respectively, and update the actual read address on the network side when image frame backlog occurs; Locally enhanced image frames composed of the local enhanced grayscale values ​​are read respectively, and network enhanced image frames composed of the network enhanced grayscale values ​​are read according to the actual reading address of the network side to perform enhanced output. The preprocessed image matrix that has not been smoothed by the network side is sent to the independent defect detection path for defect detection to obtain the area of ​​the defect candidate connected region. When the area of ​​the candidate defective connected region exceeds the pre-defined defect area threshold, the automated sorting mechanism of the production line is driven to complete the sorting and rejection action.

2. The infrared video dual-channel enhanced output and defect detection method according to claim 1, characterized in that, The pixel data, line synchronization signal, and frame synchronization signal output by the received infrared detector are combined with the detector's bad pixel location matrix for reconstruction and bad pixel repair, generating a preprocessed image matrix, including: Using the line synchronization signal and the frame synchronization signal as timing references, the pixel data output by the infrared detector is reconstructed into the original two-dimensional digital matrix; Assign frame sequence numbers and target numbers to the original two-dimensional digital matrix according to the receiving order, and write the frame sequence numbers and target numbers into the frame header of the corresponding storage space of the original two-dimensional digital matrix; When the detector bad pixel location matrix indicates that there are bad pixels in the original two-dimensional digital matrix, the bad pixel is replaced with the median of the neighboring non-bad pixels to generate the preprocessed image matrix.

3. The infrared video dual-channel enhanced output and defect detection method according to claim 1, characterized in that, In the step of extracting the grayscale gradient magnitude and local pixel variance of the base mapping grayscale value to generate locally enhanced grayscale values ​​and network enhanced grayscale values ​​respectively, generating the locally enhanced grayscale value includes: Based on the extracted grayscale gradient magnitude, sharpening weights are generated in conjunction with noise suppression parameters; Calculate the four-neighbor Laplace detail value based on the basic mapping grayscale value; The local enhanced grayscale value is generated by superimposing the base mapping grayscale value, the sharpening weight, and the four-neighbor Laplacian detail value.

4. The infrared video dual-channel enhanced output and defect detection method according to claim 1, characterized in that, In the step of extracting the grayscale gradient magnitude and local pixel variance of the base mapping grayscale value to generate the locally enhanced grayscale value and the network-enhanced grayscale value respectively, generating the network-enhanced grayscale value includes: A local neighborhood window is constructed using the base mapped grayscale value as the center pixel. Calculate the mean value of the surrounding pixels within the local neighborhood window, and calculate the local pixel variance accordingly; The smoothing weight is calculated by combining the local pixel variance and the grayscale difference between the surrounding pixels and the center pixel within the local neighborhood window. The smoothing weights are used to weight and fuse the base mapping grayscale value and the surrounding pixels within the local neighborhood window to generate the network-enhanced grayscale value.

5. The infrared video dual-channel enhanced output and defect detection method according to claim 1, characterized in that, The step of updating the actual read address on the network side when an image frame backlog occurs includes: Based on the network cache write pointer position and the network cache read pointer position, monitor and determine the number of cache backlog frames in which the image frame backlog state occurs; When the number of backlogged frames in the cache reaches a preset congestion trigger condition, an adaptive frame skipping step size is calculated based on the number of backlogged frames in the cache. When the adaptive frame skipping step size is greater than 1, the reading position is updated across intermediate backlogged image frames to obtain the actual reading address on the network side.

6. The infrared video dual-channel enhanced output and defect detection method according to claim 1, characterized in that, The step of feeding the preprocessed image matrix, which has not undergone network-side smoothing, into an independent defect detection path for defect detection, and obtaining the area of ​​candidate connected components for defects, includes: Extract the region of interest of the target to be detected from the preprocessed image matrix; Traverse the valid unmasked pixels within the region of interest of the target to be inspected, and extract the abnormal pixels of the product that meet the abnormality threshold condition. After removing isolated noise from the abnormal product pixels, connected component labeling is performed, and adjacent abnormal product pixels are divided into the same defect candidate pixel connected component. The area of ​​the defect candidate connected region is obtained by counting the total number of valid pixel nodes contained in the connected region of the defect candidate pixel.

7. The infrared video dual-channel enhanced output and defect detection method according to claim 2, characterized in that, When the area of ​​the candidate defective connected region exceeds a pre-defined defect area threshold, the automated sorting mechanism on the production line is driven to complete the sorting and rejection action, including: The maximum candidate connected component area of ​​defects is obtained in each image frame associated with the same target number, and the maximum value in all associated image frames is taken as the maximum candidate connected component area of ​​defects of the target to be inspected. When it is determined that the area of ​​the candidate connected region of the maximum defect of the target is greater than the defect area threshold, the transport encoder count value recorded when the target to be inspected enters the image acquisition area is extracted. Write the target number, the area of ​​the candidate connected component with the largest defect of the target, and the count value of the conveyor encoder into the sorting control queue; The push rod's position is calculated based on the sum of the conveyor encoder count value and the encoder count offset recorded in the sorting control queue. When the current count value of the conveyor encoder reaches the push rod's position, the output interface is controlled to flip its level, driving the push rod device of the automated sorting mechanism on the production line to complete the sorting and rejection action.

8. An infrared video dual-channel enhanced output and defect detection system, characterized in that, The infrared video dual-channel enhanced output and defect detection method according to any one of claims 1-7 includes: The data acquisition and preprocessing module is used to receive pixel data, line synchronization signals and frame synchronization signals output by the infrared detector, and to reconstruct and repair bad pixels by combining them with the bad pixel location matrix of the detector, thereby generating a preprocessed image matrix. The dual-path differentiated image enhancement and scheduling module is used to perform grayscale mapping on the preprocessed image matrix to generate basic mapped grayscale values; extract the grayscale gradient magnitude and local pixel variance of the basic mapped grayscale values ​​to generate local enhanced grayscale values ​​and network enhanced grayscale values ​​respectively, and update the actual read address on the network side when image frame backlog occurs. The local defect detection and output execution module is used to read the local enhanced image frame composed of the local enhanced grayscale values, and to read the network enhanced image frame composed of the network enhanced grayscale values ​​according to the actual reading address on the network side to perform enhanced output. The preprocessed image matrix that has not been smoothed by the network side is sent to the independent defect detection path for defect detection to obtain the defect candidate connected component area. When the defect candidate connected component area exceeds the pre-calibrated defect area threshold, the automated sorting mechanism of the production line is driven to complete the sorting and rejection action.

9. The infrared video dual-channel enhanced output and defect detection system according to claim 8, characterized in that, The system includes a core processing circuit and a storage cache area connected to the core processing circuit; The data acquisition and preprocessing module and the dual-channel differentiated image enhancement and scheduling module are integrated into the core processing circuit. The core processing circuit is used to process and generate the basic mapped grayscale value, the local enhanced grayscale value, and the network enhanced grayscale value. The core processing circuit writes the local enhanced grayscale value and the network enhanced grayscale value into the local display frame buffer and the network encoded image buffer, which are isolated from each other in the storage buffer, respectively. Based on the image frame backlog status of the network encoded image buffer, it determines the number of backlogged frames and updates the actual read address on the network side.

10. The infrared video dual-channel enhanced output and defect detection system according to claim 9, characterized in that, The system also includes a transport encoder and a data output terminal connected to the core processing circuit; The local defect detection and output execution module is integrated into the core processing circuit. When the core processing circuit determines that the area of ​​the defect candidate connected region exceeds the defect area threshold, it controls the level state of the data output terminal according to the conveying position provided by the conveying encoder. The data output terminal is connected to the push rod device of the automated sorting mechanism on the production line, and the push rod device is driven by the level state to complete the sorting and rejection action.

Images