Machine tool frame casting defect detection method and system based on AI vision

Abstract

The invention relates to the technical field of crossing of artificial intelligence and machine vision, discloses an AI-vision-based machine tool frame casting defect detection method and system, and aims to solve the technical problems that the image quality is reduced and the defect detection accuracy is influenced due to sudden change of instantaneous illumination in an industrial field. According to the method, illumination time sequence data are collected in real time by integrating a main imaging sensor and a high-frequency auxiliary light field sensor array, and future light field distribution is predicted based on a hardware Kalman filter; and generating a pixel-level dynamic gain matrix according to the main image, performing regional adaptive exposure compensation on the main image in a simulation domain, performing linear reconstruction and gain adaptive denoising, and outputting an illumination balanced image for an AI model to detect microcracks and pores. The system realizes microsecond-level illumination abrupt change response and local accurate compensation, and obviously improves the imaging consistency and defect detection rate.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of artificial intelligence and machine vision, specifically relating to an AI vision-based method and system for detecting defects in machine tool frame castings. Background Technology

[0002] Against the backdrop of the deep integration of intelligent manufacturing and industrial artificial intelligence, AI-based vision-based defect detection technology has become a key link in quality control for high-end equipment manufacturing. Especially in the production of high-precision, high-reliability metal components such as machine tool frame castings, automated vision systems are widely used to identify surface and near-surface defects such as microcracks, porosity, and shrinkage cavities. This technology relies on the collaborative work of high-resolution image acquisition and deep learning models, and its detection accuracy is directly constrained by the quality of the input image. However, the complex physical environment of industrial sites poses a severe challenge to image stability, with sudden changes in illumination becoming a core bottleneck affecting the continuity and accuracy of detection.

[0003] In machine tool frame casting production lines, often located in large workshops, the frequent movement of cranes causes the metal structure to periodically block the overhead lighting source, resulting in drastic fluctuations in localized illumination. While existing industrial cameras generally integrate automatic exposure or software-level histogram equalization mechanisms, their response latency typically exceeds 15ms, far exceeding the typical cycle of crane shadow switching. More critically, mainstream solutions perform compensation processing at the back end of the ISP pipeline or at the application layer, which not only interrupts the detection process but also introduces significant color cast and noise amplification due to the loss of original RAW domain information, directly leading to an increase in the false detection rate of AI models.

[0004] Existing technologies attempt to mitigate this problem through various approaches, but all fall into a triangular contradiction between "response speed, compensation accuracy, and hardware cost." For example, while multi-source array solutions can actively supplement illumination, the high reflectivity of the castings leads to uneven lighting, insufficient shadow elimination, and the inability to achieve closed-loop real-time control. Therefore, there is an urgent need for a millisecond-level embedded adaptive lighting architecture implemented at the chip firmware layer to simultaneously meet the rigid requirements of lower response latency, pixel-level compensation accuracy, and low-cost deployment in industrial environments. Summary of the Invention

[0005] The purpose of this invention is to provide an AI-based vision-based method and system for defect detection in machine tool frame castings. This aims to solve the technical problem in existing technologies where sudden changes in illumination at industrial sites lead to a decline in image acquisition quality, thus affecting the accuracy and stability of subsequent defect detection models. This invention constructs a predictive pixel-level exposure compensation mechanism embedded in the image acquisition front-end, based on high-frequency light field perception. When illumination changes occur, it dynamically and adaptively adjusts the gain of each region of the image sensor within a single frame image exposure cycle, thereby ensuring consistent illumination in the acquired images from the source and eliminating the interference of sudden illumination changes on the defect detection process.

[0006] To achieve the above objectives, the present invention provides an AI-based vision-based method for detecting defects in machine tool frame castings, the method comprising the following steps:

[0007] An integrated hybrid imaging sensing module simultaneously performs two acquisition operations: first, it uses a main imaging sensor array to acquire high-resolution image data for defect detection; second, it uses an auxiliary ambient light field sensor array, which is co-packaged or arranged adjacent to the main imaging sensor array on the same substrate, to acquire a multi-point real-time illuminance data stream on the surface of the casting under test at a sampling frequency much higher than the frame rate of the main imaging sensor array, forming a time-series light field map.

[0008] The system receives the time-series light field map output by the auxiliary ambient light field sensor array and inputs it into a hardware-implemented illuminance spatiotemporal prediction unit. The illuminance spatiotemporal prediction unit has a built-in state-space model with predetermined parameters. Based on the illuminance values ​​and their spatial distribution at multiple consecutive time points in the past contained in the time-series light field map, it predicts the future light field distribution map for each preset small time step in the next exposure cycle of the main imaging sensor.

[0009] Based on the future light field distribution map output by the illumination-spatial prediction unit, a corresponding analog gain adjustment value is generated for each independent pixel control region of the main imaging sensor array. The calculation of the analog gain adjustment value follows a preset backlight response function to ensure that the pixel control region with lower predicted illumination obtains a higher analog gain, while the pixel control region with higher predicted illumination obtains a lower analog gain, thereby generating a two-dimensional dynamic gain matrix that completely corresponds to the pixel region layout of the main imaging sensor.

[0010] At the start of a single exposure cycle of the main imaging sensor, the two-dimensional dynamic gain matrix is ​​loaded onto a pixel-level programmable gain amplifier array. Each amplifier unit in the pixel-level programmable gain amplifier array physically corresponds to a pixel control area of ​​the main imaging sensor, and during the exposure cycle, according to the gain adjustment value at the corresponding position in the two-dimensional dynamic gain matrix, the amplification factor of the analog signal in the photogenerated charge integration process of its corresponding pixel control area is adjusted in real time and independently.

[0011] After the single exposure cycle ends, the original image data after region adaptive gain modulation is read from the main imaging sensor, and the two-dimensional dynamic gain matrix actually applied to each pixel control region during the exposure cycle is obtained. A linearization reconstruction process of the original data is performed, specifically by dividing the original readout value of each pixel by the actual gain adjustment value corresponding to its pixel control region, so as to eliminate the nonlinearity introduced by gain modulation and restore the linearized image data that is proportional to the number of incident photons.

[0012] The linearized and reconstructed image data is input into a defect detection inference engine. The defect detection inference engine is loaded with a pre-trained convolutional neural network model, which is trained to identify defects with specific morphological features on the surface of machine tool frame castings, including microcracks and pores. The defect detection inference engine performs forward propagation calculations on the linearized and reconstructed image data and outputs the location coordinates, category, and confidence score of each detected defect.

[0013] As one embodiment of the present invention, the method of acquiring multi-point real-time illuminance data streams on the surface of the casting under test using an auxiliary ambient light field sensor array specifically includes: the auxiliary ambient light field sensor array is composed of a 64-row, 48-column indium gallium arsenide photodiode array, with each diode having a photosensitive area of ​​0.5 square millimeters and a response time of less than one microsecond; a dedicated high-speed sampling circuit synchronously samples all 3,072 photodiodes in the array at a frequency of 200 kHz, and the 16-bit precision digital values ​​obtained from the sampling are used to form a frame of light field diagram, which is continuously output to form the time-series light field diagram.

[0014] Furthermore, the main imaging sensor array and the auxiliary ambient light field sensor array are integrated on the same ceramic package substrate and electrically connected by flip-chip bonding. A high-precision phase-locked loop circuit ensures that the sampling clock of the auxiliary ambient light field sensor array is strictly synchronized with the pixel readout clock of the main imaging sensor array, and the timestamp error between the two is controlled within ten nanoseconds.

[0015] In one embodiment of the present invention, the illuminance spatiotemporal prediction unit incorporates a state-space model with predetermined parameters, specifically a hardware-implemented Kalman filter bank. For each sampling point in the light field map, an independent Kalman filter is configured. The state vector of each Kalman filter contains the current illuminance value and its first-order rate of change. Its state transition matrix is ​​constructed based on a constant velocity model and is used to predict the state at the next time step. Its observation matrix maps the state vector to the observed illuminance value. Through iterative prediction and update steps, the filter bank outputs a predicted light field distribution map, and its prediction time lead is set to the system response delay of the pixel-level programmable gain amplifier array, which is three microseconds.

[0016] Furthermore, the generation process of the two-dimensional dynamic gain matrix also includes: firstly, using a bilinear interpolation algorithm, upsampling the predicted light field distribution map of 64 rows and 48 columns output by the illuminance spatiotemporal prediction unit to 256 rows and 192 columns, so that its resolution matches the pixel control area division of the main imaging sensor; then, using a lookup table, mapping each interpolated predicted illuminance value to an 8-bit precision gain control code, wherein the lookup table stores the nonlinear inverse relationship curve between light intensity and the required simulated gain.

[0017] In one embodiment of the present invention, each amplifier unit in the pixel-level programmable gain amplifier array corresponds to a 16 x 16 pixel physical region of the main imaging sensor, constituting a pixel control region; the resolution of the main imaging sensor is 4096 x 3072 pixels, and therefore it is divided into 256 x 192 pixel control regions; each amplifier unit receives the corresponding eight-bit gain control code, and through an internal digital-to-analog converter, generates a precise gate bias voltage, which is applied to the analog signal amplification circuit shared by the pixel control region, thereby completing the gain setting within 0.5 microseconds.

[0018] Furthermore, after the linearization and reconstruction of the original data, a gain-adaptive denoising process is also included. The denoising process employs a modified bilateral filtering algorithm, whose spatial domain kernel function is a standard Gaussian kernel, while the strength of its domain kernel function is modulated by the two-dimensional dynamic gain matrix. Specifically, for any pixel, the standard deviation of its domain kernel function is proportional to the gain adjustment value of the pixel control region where the pixel is located, so that pixels in high-gain regions are subjected to stronger smoothing filtering to suppress amplified noise, while pixels in low-gain regions retain more detailed information.

[0019] This invention also provides an AI vision-based defect detection system for machine tool frame castings, the system comprising:

[0020] The hybrid imaging sensing module integrates a main imaging sensor array and an auxiliary ambient light field sensor array. The main imaging sensor array is used to acquire high-resolution images, while the auxiliary ambient light field sensor array is used to acquire multi-point real-time illuminance data streams at a frame rate much higher than that of the main imaging sensor array to form a time-series light field map. The two are synchronized at the sub-nanosecond level through an internal synchronization clock circuit.

[0021] The illuminance spatiotemporal prediction module is connected to the hybrid imaging sensor module via a high-speed data interface to receive the temporal light field map. The illuminance spatiotemporal prediction module has a set of parallel Kalman filter logic circuits embedded inside, which are used to predict the future light field distribution map in the next exposure cycle of the main imaging sensor based on the received historical illuminance data.

[0022] A pixel-level gain control module is established with both the illumination spatiotemporal prediction module and the hybrid imaging sensing module. This module is used to receive the future light field distribution map and calculate an independent analog gain adjustment value for each pixel control area of ​​the main imaging sensor array, generate a one-dimensional dynamic gain matrix, and apply the control signal corresponding to the matrix to the pixel-level programmable gain amplifier array inside the sensor during the exposure cycle of the main imaging sensor.

[0023] The image reconstruction and preprocessing module receives the original image data output by the hybrid imaging sensing module after region adaptive gain modulation, as well as the two-dimensional dynamic gain matrix actually used by the pixel-level gain control module during the exposure cycle. The module has a built-in hardware divider array and gain adaptive filtering circuit to perform linear reconstruction processing of the original data and gain adaptive denoising processing, and outputs a standardized image with balanced illumination and controlled noise level.

[0024] The defect detection inference module receives the standardized image output by the image reconstruction and preprocessing module. The module consists of an embedded high-performance computing platform on which an optimized convolutional neural network model runs to perform pixel-level analysis on the input standardized image and finally outputs the location, category, and confidence information of defects on the surface of the casting.

[0025] In one embodiment of the present invention, the main imaging sensor array of the hybrid imaging sensing module is a global shutter complementary metal-oxide-semiconductor sensor with an effective pixel count of 4,096 x 3,072 and a pixel size of 3.45 micrometers; the auxiliary ambient light field sensor array is a 64 x 48 indium gallium arsenide photodiode array with a sampling frequency of 200 kHz.

[0026] Furthermore, the illuminance spatiotemporal prediction module and the pixel-level gain control module are jointly implemented in a single field-programmable gate array (FPGA) chip, and exchange data through an on-chip bus to ensure that the end-to-end processing delay from receiving light field data to generating the gain control signal is less than five microseconds.

[0027] As one embodiment of the present invention, the image reconstruction and preprocessing module further includes a bad pixel correction unit and a flat field correction unit, located before the original data linearization reconstruction process, for correcting inherent pixel defects of the sensor and brightness non-uniformity introduced by the optical system.

[0028] Furthermore, the convolutional neural network model used in the defect detection inference module has an encoder-decoder structure. The encoder part is used to extract multi-scale features of the image, while the decoder part fuses the multi-scale features and upsamples them to the original resolution to achieve accurate segmentation and localization of micron-level cracks.

[0029] In summary, this application includes at least one of the following beneficial technical effects:

[0030] (1): By introducing a high-frequency auxiliary light field sensor array that works synchronously with the main imaging sensor, the present invention can capture the drastic dynamic changes in industrial lighting at a time resolution of microseconds, providing a data basis for subsequent predictive compensation.

[0031] (2): By implementing a spatiotemporal prediction algorithm based on a state-space model in hardware, the present invention can calculate the compensation scheme in advance before the actual impact of light changes on the main imaging sensor, realizing the transformation from passive response to active prediction. Its overall response delay is controlled within five microseconds, which is much faster than the millisecond-level software or camera automatic exposure response speed in the prior art.

[0032] (3): This invention creates an independent gain control mechanism at the pixel region level, which can accurately compensate for local shadows or uneven lighting in a spatially varying manner. It solves the problem of overexposure or underexposure caused by traditional global compensation methods when dealing with local lighting problems, and significantly improves the imaging contrast of small defects on the complex surface of castings.

[0033] (4): By directly adjusting the gain of photogenerated charge in the analog domain and using a precise gain map for linear reconstruction after digitization, this invention avoids the noise amplification and color distortion problems caused by post-processing compensation in the digital domain, and ensures the high signal-to-noise ratio and color fidelity of the output image.

[0034] (5): This invention deeply integrates the illumination adaptive function into the front-end hardware of the image acquisition system, forming a complete and self-consistent solution. It does not rely on external supplementary lighting equipment or cloud computing resources, reducing the complexity of the system, deployment costs and data security risks. At the same time, it provides highly stable and consistent image input for the subsequent AI defect detection model, fundamentally improving the robustness, accuracy and production line cycle of the detection system. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall technical architecture of the AI ​​vision-based machine tool frame casting defect detection method and system proposed in this invention;

[0036] Figure 2 This is a schematic diagram of the core principle framework of the predictive pixel-level exposure compensation mechanism based on high-frequency light field sensing in this invention.

[0037] Figure 3 This is a schematic diagram of the multi-level interaction relationship and data flow between image acquisition, gain modulation and defect detection inference in this invention; Detailed Implementation

[0038] This invention provides an AI-based vision-based method and system for defect detection in machine tool frame castings. The core of this system lies in constructing a predictive pixel-level exposure compensation mechanism embedded in the image acquisition front-end, based on high-frequency light field perception. When a sudden change in illumination occurs, this mechanism dynamically and adaptively adjusts the gain of each region within the single-frame image exposure cycle of the main imaging sensor, thereby ensuring consistent image illumination at the source and eliminating the interference of instantaneous illumination changes on subsequent defect detection models.

[0039] See attached document Figure 1 To be continued Figure 3 As shown, the first part describes the AI ​​vision-based defect detection method for machine tool frame castings disclosed in this application. Specifically, it includes the following steps:

[0040] Two acquisition operations are performed simultaneously through an integrated hybrid imaging sensing module: first, high-resolution image data for defect detection is acquired using a main imaging sensor array.

[0041] Secondly, an auxiliary ambient light field sensor array, co-packaged or closely arranged on the same substrate as the main imaging sensor array, is used to collect real-time illuminance data streams from multiple points on the surface of the casting under test at a sampling frequency much higher than that of the main imaging sensor array, forming a time-series light field map. Specifically, the auxiliary ambient light field sensor array consists of a 64-row, 48-column array of indium gallium arsenide photodiodes, with each diode having a photosensitive area of ​​0.5 square millimeters and a response time of less than one microsecond.

[0042] A dedicated high-speed sampling circuit synchronously samples all 3072 photodiodes in the array at a frequency of 200kHz, and the 16-bit precision digital values ​​obtained from the sampling are used to form a frame of light field pattern, which is then continuously output to form a time-series light field pattern.

[0043] This sampling frequency ensures at least 25 times the time redundancy sampling capability for sudden changes in illumination caused by the movement of the workshop crane (measured change period ≤ eight milliseconds), so that both the rising and falling edges of the illumination change can be fully captured, avoiding inaccurate predictions due to undersampling.

[0044] Furthermore, the main imaging sensor array and the auxiliary ambient light field sensor array are integrated on the same ceramic package substrate, electrically connected via flip-chip bonding. A high-precision phase-locked loop circuit ensures that the sampling clock of the auxiliary ambient light field sensor array is strictly synchronized with the pixel readout clock of the main imaging sensor array, with their timestamp error controlled within ten nanoseconds. This synchronization mechanism eliminates the time alignment error (typically greater than one millisecond) caused by asynchronous acquisition between the ambient light sensor and the image sensor in traditional solutions, ensuring a high degree of consistency between the predicted light field and the actual illumination state at the time of imaging, fundamentally solving the compensation lag problem.

[0045] The system receives the time-series light field map output by the auxiliary ambient light field sensor array and inputs it into a hardware-implemented illuminance spatiotemporal prediction unit. The illuminance spatiotemporal prediction unit has a built-in state-space model with predetermined parameters. Based on the illuminance values ​​and their spatial distribution at multiple consecutive time points in the past contained in the time-series light field map, it predicts the future light field distribution map for each preset tiny time step in the next exposure cycle of the main imaging sensor.

[0046] The state-space model is specifically a hardware-implemented Kalman filter bank. For each sampling point in the light field map, an independent Kalman filter is configured. The state vector of each Kalman filter contains the illuminance value at the current time and its first-order rate of change. Its state transition matrix is ​​constructed based on the constant velocity model and is used to predict the state at the next time step. Its observation matrix maps the state vector to the observed illuminance value. Through continuous iterative prediction and update steps, the filter bank outputs a predicted light field distribution map. Its prediction time lead is set to the system response delay of the pixel-level programmable gain amplifier array, which is three microseconds.

[0047] The mathematical expression for this Kalman filter bank is as follows:

[0048]

[0049]

[0050] in, For the first The state vector at time t, Indicates the illuminance value. Indicates the rate of change of illuminance; Here is the state transition matrix. =5 microseconds is the sampling interval; The observation matrix; and These are process noise and observation noise, respectively. Their covariance matrices Q and R are pre-calibrated based on the measured illumination fluctuation characteristics and embedded in the hardware logic. This model can output the predicted illumination value for the next three microseconds with a computational delay of less than 0.5 microseconds, relying only on three historical sampling points, thus meeting the requirement that the system's end-to-end response delay is less than five microseconds.

[0051] Based on the future light field distribution map output by the illuminance spatiotemporal prediction unit, a corresponding analog gain adjustment value is generated for each independent pixel control area of ​​the main imaging sensor array. The calculation of the analog gain adjustment value follows a preset backlight response function to ensure that the pixel control area with lower predicted illuminance obtains a higher analog gain, while the pixel control area with higher predicted illuminance obtains a lower analog gain, thereby generating a two-dimensional dynamic gain matrix that is completely corresponding to the pixel area layout of the main imaging sensor.

[0052] The generation process of the two-dimensional dynamic gain matrix also includes: firstly, using a bilinear interpolation algorithm, upsampling the predicted light field distribution map (64 rows, 48 ​​columns) output by the illuminance spatiotemporal prediction unit to 256 rows, 192 columns, so that its resolution matches the pixel control area division of the main imaging sensor; subsequently, using a lookup table, each interpolated predicted illuminance value is mapped to an 8-bit precision gain control code. The lookup table stores the nonlinear inverse relationship curve between light intensity and the required simulated gain. The mapping relationship of this lookup table is defined by the following formula:

[0053]

[0054] in, For the target gain corresponding to illuminance I, =255 is the maximum gain control code value. =800 lux is the reference illuminance. =0.0015 The sensitivity coefficient is determined through joint optimization experiments on the surface reflectivity of the casting and the ideal image signal-to-noise ratio in a standard illumination laboratory, ensuring that the image signal-to-noise ratio is maintained at no less than 40dB even when the illumination is as low as 300 lux.

[0055] At the start of a single exposure cycle of the main imaging sensor, a two-dimensional dynamic gain matrix is ​​loaded onto a pixel-level programmable gain amplifier array. Each amplifier unit in the pixel-level programmable gain amplifier array physically corresponds to a pixel control area of ​​the main imaging sensor, and during the exposure cycle, according to the gain adjustment value at the corresponding position in the two-dimensional dynamic gain matrix, the amplification factor of the analog signal in the photogenerated charge integration process of its corresponding pixel control area is adjusted in real time and independently.

[0056] Each amplifier unit in the pixel-level programmable gain amplifier array corresponds to a 16x16 pixel physical area of ​​the main imaging sensor, forming a pixel control area. The main imaging sensor has a resolution of 4096x3072 pixels, thus it is divided into 256x192 pixel control areas. Each amplifier unit receives a corresponding 8-bit gain control code and, through an internal digital-to-analog converter, generates a precise gate bias voltage, which is applied to the analog signal amplification circuit shared by that pixel control area, thereby completing the gain setting within 0.5 microseconds. This structure achieves region-level rather than global gain control, allowing for targeted compensation of locally shadowed areas (such as a crane only obscuring the edge of a casting), avoiding overexposure problems in non-shadowed areas caused by traditional global gain adjustment.

[0057] After a single exposure cycle, the raw image data modified by region adaptive gain is read from the main imaging sensor, and the two-dimensional dynamic gain matrix actually applied to each pixel control region during the exposure cycle is obtained. A linearization reconstruction process is performed on the raw data, which is to divide the raw readout value of each pixel by the actual gain adjustment value corresponding to its pixel control region, so as to eliminate the nonlinearity introduced by gain modulation and restore the linearized image data that is proportional to the number of incident photons.

[0058] After linearizing and reconstructing the original data, a gain-adaptive denoising step is included. This denoising process employs a modified bilateral filtering algorithm. The spatial domain kernel function of this algorithm is a standard Gaussian kernel, while the strength of its domain kernel function is modulated by a two-dimensional dynamic gain matrix. Specifically, for any pixel, the standard deviation of its domain kernel function is proportional to the gain adjustment value of the pixel control region, resulting in stronger smoothing filtering for pixels in high-gain regions to suppress amplified noise, while pixels in low-gain regions retain more detail. The weight calculation formula for this bilateral filter is as follows:

[0059]

[0060] Where p and q are two pixel positions in the image. =2 pixels is the standard deviation of the spatial domain. =15 is the standard deviation of the baseline range. This is a normalized coefficient (ranging from 0.2 to 1.0) for the gain adjustment value of the region containing pixel p. This design ensures that noise suppression is automatically enhanced in high-gain regions (such as areas with sudden drops in illumination), while maintaining edge sharpness in normal illumination regions, effectively balancing image quality and noise levels.

[0061] The linearized and reconstructed image data is input into a defect detection inference engine. The defect detection inference engine is loaded with a pre-trained convolutional neural network model, which is trained to identify defects with specific morphological features on the surface of the machine tool frame casting, including microcracks and pores. The defect detection inference engine performs forward propagation calculation on the linearized and reconstructed image data and outputs the location coordinates, category, and confidence score of each detected defect.

[0062] The present invention also provides an AI vision-based machine tool frame casting defect detection system. The system includes: a hybrid imaging sensing module, which integrates a main imaging sensor array and an auxiliary ambient light field sensor array. The main imaging sensor array is used to acquire high-resolution images, and the auxiliary ambient light field sensor array is used to acquire multi-point real-time illuminance data streams at a frame rate much higher than that of the main imaging sensor array to form a time-series light field map. The two are synchronized at the sub-nanosecond level through an internal synchronization clock circuit.

[0063] An illuminance spatiotemporal prediction module is connected to the hybrid imaging sensor module via a high-speed data interface to receive temporal light field maps. The illuminance spatiotemporal prediction module has a set of parallel Kalman filter logic circuits embedded inside, which are used to predict the future light field distribution map in the next exposure cycle of the main imaging sensor based on the received historical illuminance data.

[0064] A pixel-level gain control module is established with both the illumination spatiotemporal prediction module and the hybrid imaging sensing module. This module is used to receive the future light field distribution map and calculate an independent analog gain adjustment value for each pixel control area of ​​the main imaging sensor array, generating a two-dimensional dynamic gain matrix. During the exposure cycle of the main imaging sensor, the control signal corresponding to the matrix is ​​applied to the pixel-level programmable gain amplifier array inside the sensor.

[0065] An image reconstruction and preprocessing module receives raw image data with region adaptive gain modulation output from a hybrid imaging sensing module, as well as a two-dimensional dynamic gain matrix actually used by a pixel-level gain control module within the exposure cycle. The module has a built-in hardware divider array and gain adaptive filtering circuit to perform linearization reconstruction processing of the raw data and gain adaptive denoising processing, outputting a standardized image with balanced illumination and controlled noise level.

[0066] A defect detection inference module receives a standardized image output from an image reconstruction and preprocessing module. This module consists of an embedded high-performance computing platform on which an optimized convolutional neural network model runs to perform pixel-level analysis on the input standardized image and finally outputs the location, category, and confidence information of defects on the surface of the casting.

[0067] The main imaging sensor array of the hybrid imaging sensing module is a global shutter complementary metal-oxide-semiconductor sensor with an effective pixel count of 4,096 x 3,072 and a pixel size of 3.45 micrometers. The auxiliary ambient light field sensor array is a 64 x 48 indium gallium arsenide photodiode array with a sampling frequency of 200 kHz. The global shutter structure ensures that the entire image is exposed at the same time, avoiding image distortion caused by rolling shutters in high-speed motion scenes. It is suitable for the inspection of machine tool frame castings moving at a constant speed on a conveyor belt.

[0068] The illuminance spatiotemporal prediction module and the pixel-level gain control module are jointly implemented on a single field-programmable gate array (FPGA) chip, exchanging data via an on-chip bus to ensure an end-to-end processing latency of less than five microseconds from receiving light field data to generating gain control signals. This FPGA chip utilizes the Xilinx Kintex UltraScale series, with internal logic resources sufficient to deploy 3,072 Kalman filter instances in parallel. Each instance occupies approximately fifty lookup tables and twenty registers, keeping overall resource utilization below 70% and reserving upgrade space.

[0069] The image reconstruction and preprocessing module also includes a bad pixel correction unit and a flat field correction unit, located before the linearization and reconstruction processing of the original data. These units are used to correct inherent pixel defects in the sensor and brightness non-uniformity introduced by the optical system. The bad pixel correction unit uses a bad pixel coordinate table generated during the factory calibration stage to replace the abnormal pixel value with its neighborhood mean in real time during image readout. The flat field correction unit uses a pre-acquired uniform white board image to generate a correction gain map and performs pixel-by-pixel division correction on the original image to eliminate brightness gradients caused by lens vignetting and uneven light sources.

[0070] The convolutional neural network model used in the defect detection inference module has an encoder-decoder structure. The encoder extracts multi-scale features from the image, while the decoder fuses these features and upsamples them to the original resolution to achieve accurate segmentation and localization of micron-sized cracks. During training, the model used over 100,000 labeled images covering different lighting conditions, angles, casting batches, and defect types. It employed a combination of cross-entropy loss and Dice coefficient optimization, ultimately achieving a recall of 92.5% and a precision of 89.3% for microcracks less than 0.3 mm wide on the test set. This significantly outperformed the baseline model (54.7% recall) that did not utilize the lighting compensation mechanism of this invention.

[0071] In summary, this invention deeply integrates high-frequency light field sensing, hardware-level spatiotemporal prediction, pixel-region-level analog gain control, and linear reconstruction into the front end of the image acquisition chain, constructing a closed-loop, predictive, pixel-level precision adaptive illumination system. This system achieves millisecond-level response and pixel-level compensation for instantaneous illumination changes in industrial settings without interrupting the detection process, adding external light sources, or relying on cloud computing. This fundamentally ensures the stability, accuracy, and production line cycle consistency of the AI ​​visual defect detection system.

[0072] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0073] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting defects in machine tool frame castings using AI vision, characterized in that, include: By simultaneously acquiring high-resolution image data and multi-point real-time illuminance data streams through the main imaging sensor array and the auxiliary ambient light field sensor array integrated on the same substrate of the hybrid imaging sensing module, a time-series light field map is formed. The temporal light field map is input into the hardware-implemented illuminance spatiotemporal prediction unit. The illuminance spatiotemporal prediction unit has a built-in state space model with predetermined parameters, and predicts the future light field distribution map of the main imaging sensor in the next exposure cycle based on the state space model. Based on the future light field distribution map output by the illuminance spatiotemporal prediction unit, a corresponding analog gain adjustment value is generated for each independent pixel control area of ​​the main imaging sensor array. The analog gain adjustment value follows the preset backlight response function to ensure that the pixel control area with lower predicted illuminance obtains higher analog gain and the pixel control area with higher predicted illuminance obtains lower analog gain, thereby generating a two-dimensional dynamic gain matrix corresponding to the pixel area layout of the main imaging sensor. At the start of the exposure cycle of the main imaging sensor array, a two-dimensional dynamic gain matrix is ​​loaded onto a pixel-level programmable gain amplifier array, and independent analog gain adjustment is performed for each pixel control area. After the exposure cycle ends, the gain-modulated raw image data is read, and the raw data is linearized and reconstructed to restore the linearized image data. The linearized and reconstructed image data is input into the defect detection inference engine, which identifies and outputs the location, category, and confidence score of the defect through a pre-trained convolutional neural network model.

2. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, The real-time illuminance data stream at multiple points on the surface of the casting under test is acquired using an auxiliary ambient light field sensor array, including: The auxiliary ambient light field sensor array consists of a 64-row, 48-column array of indium gallium arsenide photodiodes, with each diode having a photosensitive area of ​​0.5 square millimeters and a response time of less than 1 microsecond. A dedicated high-speed sampling circuit synchronously samples all 3072 photodiodes in the array at a frequency of 200kHz, and the 16-bit precision digital values ​​obtained from the sampling are used to form a frame of light field pattern, which is then continuously output to form a time-series light field pattern.

3. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, The main imaging sensor array and the auxiliary ambient light field sensor array are integrated on the same ceramic package substrate and electrically connected through flip-chip bonding. A high-precision phase-locked loop circuit ensures that the sampling clock of the auxiliary ambient light field sensor array is strictly synchronized with the pixel readout clock of the main imaging sensor array, and the timestamp error between the two is controlled within 10 nanoseconds.

4. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, The illuminance spatiotemporal prediction unit has a built-in state-space model with predetermined parameters, specifically a hardware-implemented Kalman filter bank. For each sampling point in the light field diagram, an independent Kalman filter is configured; the state vector of each Kalman filter contains the illuminance value at the current moment and its first-order rate of change. Its state transition matrix is ​​constructed based on a constant velocity model and is used to predict the state at the next time step; Its observation matrix maps the state vector to the observed illuminance value. Through continuous iterative prediction and update steps, the Kalman filter bank outputs a predicted light field distribution map. Its prediction time lead is set to the system response delay of the pixel-level programmable gain amplifier array, which is 3 microseconds.

5. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, The process of generating the two-dimensional dynamic gain matrix also includes: First, the predicted light field distribution map of 64 rows and 48 columns output by the illumination spatiotemporal prediction unit is upsampled to 256 rows and 192 columns using a bilinear interpolation algorithm, so that its resolution matches the pixel control area division of the main imaging sensor. Then, each predicted illuminance value after interpolation is mapped to an 8-bit precision gain control code through a lookup table. The lookup table stores the nonlinear inverse relationship curve between illuminance and the required simulated gain.

6. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, Each amplifier unit in the pixel-level programmable gain amplifier array corresponds to a 16-by-16 pixel physical area of ​​the main imaging sensor, forming a pixel control area. The resolution of the main imaging sensor is 4096 by 3072 pixels, so it is divided into 256 by 192 pixel control areas. Each amplifier unit receives a corresponding 8-bit gain control code and generates a precise gate bias voltage through an internal digital-to-analog converter, which is then applied to the analog signal amplification circuit shared by the pixel control area, thereby completing the gain setting within 0.5 microseconds.

7. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 1, characterized in that, After the original data is linearized and reconstructed, the process also includes gain-adaptive denoising. The denoising process employs a modified bilateral filtering algorithm. The spatial domain kernel function of this algorithm is a standard Gaussian kernel, while the strength of its domain kernel function is modulated by a two-dimensional dynamic gain matrix. Specifically, for any pixel, the standard deviation of its domain kernel function is proportional to the gain adjustment value of the pixel control region where the pixel is located. This results in pixels in high-gain regions receiving stronger smoothing filtering to suppress amplified noise, while pixels in low-gain regions retain more detailed information.

8. The AI ​​vision-based defect detection method for machine tool frame castings according to claim 7, characterized in that, Before the linearization and reconstruction of the original data, there are also bad pixel correction and flat field correction processes. The bad pixel correction process is based on the bad pixel coordinate table generated during the factory calibration stage. When the image is read out, the abnormal pixel value is replaced with its neighborhood mean in real time. The flat field correction process uses a pre-acquired uniform white board image to generate a correction gain map and performs pixel-by-pixel division correction on the original image to eliminate the brightness gradient caused by lens vignetting and uneven light source.

9. A machine tool frame casting defect detection system based on AI vision, characterized in that, include: The hybrid imaging sensing module integrates a main imaging sensor array and an auxiliary ambient light field sensor array. The main imaging sensor array is used to acquire high-resolution images, while the auxiliary ambient light field sensor array is used to acquire multi-point real-time illuminance data streams at a frame rate much higher than that of the main imaging sensor array to form a time-series light field map. The two are synchronized at the sub-nanosecond level through an internal synchronization clock circuit. The illuminance spatiotemporal prediction module is connected to the hybrid imaging sensor module via a high-speed data interface to receive temporal light field maps. The illuminance spatiotemporal prediction module has a set of parallel Kalman filter logic circuits embedded inside, which are used to predict the future light field distribution map in the next exposure cycle of the main imaging sensor based on the received historical illuminance data. The pixel-level gain control module is connected to both the illuminance spatiotemporal prediction module and the hybrid imaging sensing module. This module is used to receive the future light field distribution map and calculate an independent analog gain adjustment value for each pixel control area of ​​the main imaging sensor array, generating a one-dimensional dynamic gain matrix. During the exposure cycle of the main imaging sensor, the control signal corresponding to the matrix is ​​applied to the pixel-level programmable gain amplifier array inside the sensor. The image reconstruction and preprocessing module receives the raw image data output by the hybrid imaging sensing module after region adaptive gain modulation, as well as the two-dimensional dynamic gain matrix actually used by the pixel-level gain control module within the exposure cycle. The module has a built-in hardware divider array and gain adaptive filtering circuit to perform linearization reconstruction processing of the raw data and gain adaptive denoising processing, and outputs a standardized image with balanced illumination and controlled noise level. The defect detection inference module receives the standardized image output by the image reconstruction and preprocessing module; The module consists of an embedded high-performance computing platform on which an optimized convolutional neural network model is run to perform pixel-level analysis on the input standardized image and finally output the location, type and confidence information of defects on the surface of the casting.

10. The AI ​​vision-based machine tool frame casting defect detection system according to claim 9, characterized in that, The illuminance spatiotemporal prediction module and the pixel-level gain control module are jointly implemented on a single field-programmable gate array chip, exchanging data via an on-chip bus to ensure that the end-to-end processing delay from receiving light field data to generating gain control signals is less than five microseconds. The convolutional neural network model used in the defect detection inference module has an encoder-decoder structure. The encoder part is used to extract multi-scale features of the image, while the decoder part fuses the multi-scale features and upsamples them to the original resolution to achieve accurate segmentation and localization of micron-level cracks.

Images