Valve casting forming defect online prediction method based on deep learning

Abstract

This invention relates to the field of intelligent manufacturing and industrial visual inspection technology, specifically to an online prediction method for valve casting forming defects based on deep learning. The method includes: acquiring online multimodal image data of valve castings and production line cycle time data; determining the forming stage status, image quality status, defect risk level, and equipment operating status based on the data; classifying the current prediction task into routine inspection tasks or key inspection tasks accordingly, and executing corresponding image acquisition control and model scheduling; inputting the multimodal image data into the model branch corresponding to the forming stage status for prediction, combining the image quality status and equipment operating status to determine the corrected risk value or result reliability, and outputting the online prediction result. This invention can improve the stability of online prediction and reduce the interference of abnormal images and equipment degradation on the final quality conclusion.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and industrial visual inspection technology, specifically to an online prediction method for valve casting forming defects based on deep learning. Background Technology

[0002] In existing valve casting production lines, image acquisition and inspection equipment are usually arranged during the casting, cooling and subsequent transfer processes. The surface condition, temperature condition and defect signs of the casting are monitored online in combination with visible light images, thermal imaging data or single sensor data, so as to detect forming defects such as shrinkage cavities, cracks, cold shuts or deformation in a timely manner.

[0003] In existing online inspection methods, some systems use fixed sampling parameters and unified recognition models to analyze casting images at different process stages. Although some systems introduce deep learning methods, they mainly rely on front-end acquired images to be directly fed into the model for judgment, without sufficient consideration of production line cycle time, position changes, image synchronization relationships, image quality fluctuations, and equipment operating status.

[0004] However, during the process of high-temperature forming and cooling stabilization of valve castings, the changes in thermal field distribution, contour shrinkage, and surface reflection exceed the preset process fluctuation threshold. Furthermore, time deviations, blurring, overexposure, smoke obscuring, frame loss, and computing power fluctuations can easily occur between different acquisition links, causing online prediction results to be prone to misjudgment, omission, or insufficient reliability. As a result, it is difficult to meet the requirements for stable online prediction of casting forming defects in complex production line environments. Summary of the Invention

[0005] The purpose of this invention is to provide an online prediction method for valve casting forming defects based on deep learning, which avoids interference with the final quality conclusion due to image quality fluctuations, time deviations, and degradation of acquisition equipment. Furthermore, it can dynamically adjust the detection task and schedule the corresponding model branches based on the specific forming stage status, thereby effectively solving the problems of misjudgment, missed judgment, or insufficient reliability that are prone to occur in complex production line environments, and improving the accuracy and stability of online defect prediction.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] The image acquisition device acquires multimodal image data of valve castings and production line cycle data online. The multimodal image data includes at least visible light images and infrared thermal images, and the production line cycle data includes at least the station trigger time and the transmission position.

[0008] Based on multimodal image data and production line cycle data, determine the molding stage status, image quality status, defect risk level, and equipment operating status;

[0009] Based on the forming stage status, image quality status, defect risk level, and equipment operating status, the current prediction task is divided into routine detection tasks or key detection tasks, and corresponding image acquisition control and model scheduling are executed.

[0010] Multimodal image data is input into a deep learning defect prediction model. Based on the model scheduling results, the model branch corresponding to the forming stage state is called to make predictions, and preliminary prediction results containing candidate defect types are obtained.

[0011] Based on the preliminary prediction results, combined with the image quality status and equipment operating status, the corrected risk value or result reliability is determined, and the online prediction result is output, which includes the defect risk type, result reliability, and equipment status identifier generated based on the equipment operating status.

[0012] Preferably, the process of acquiring multimodal image data and production line cycle data of valve castings collected online includes: establishing a unified reference time based on the system synchronization clock; and calculating the time deviation of the multimodal image data and production line cycle data relative to the unified reference time.

[0013] Based on the time deviation, determine the data synchronization index at the current moment; when the data synchronization index meets the preset conditions determined based on the calibration results, execute the steps to determine the forming stage status, image quality status, defect risk level, and equipment operating status; when the data synchronization index does not meet the preset conditions, transfer the multimodal image data and production line cycle data to the low-confidence queue.

[0014] As a preferred option, determining the state of the forming stage includes: obtaining the current position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate;

[0015] Based on the position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate, determine the indicators for the high-temperature forming stage, cooling stage, and contour stabilization stage.

[0016] The forming stage state is determined based on the maximum value among the indicators of the high-temperature forming stage, the cooling stage, and the contour stabilization stage.

[0017] Preferably, the image quality status is determined by: acquiring image blur, thermal overexposure ratio, proportion of highly reflective areas, smoke obscuration index, and inter-frame alignment error from multimodal image data; determining an image quality score based on the image blur, thermal overexposure ratio, proportion of highly reflective areas, smoke obscuration index, and inter-frame alignment error; determining the image quality status as clear, slightly polluted, or heavily polluted based on the numerical range of the image quality score; and determining the defect risk level, including:

[0018] Acquire the abnormal high temperature area change rate, boundary curvature abrupt change value, texture anomaly offset, contour deformation area, and depression growth rate of multimodal image data; determine the defect risk score based on the abnormal high temperature area change rate, boundary curvature abrupt change value, texture anomaly offset, contour deformation area, and depression growth rate.

[0019] Based on the numerical range of the defect risk score, the defect risk level is determined as normal fluctuation, suspected defect, or high-risk defect.

[0020] As a preferred method, the device operating status is determined, including: obtaining the current frame drop rate, thermal image overexposure alarm intensity, supplementary lighting anomaly value, processor frequency reduction index, and cache time difference index;

[0021] The current device health score is determined based on the frame drop rate, thermal overexposure alarm intensity, abnormal supplementary lighting value, processor frequency reduction index, and cache time difference index.

[0022] At the initial moment, the device status value is set to the preset initial value; at non-initial moments, the current device status value is determined based on the current device health score and the device status value at the previous moment; based on the numerical range of the device status value, the device operating status is determined to be normal, slightly abnormal, or degraded operation.

[0023] As a preferred option, the current prediction task is divided into a routine detection task or a key detection task, including: when the image quality is clear, the equipment is operating normally, and the defect risk level is fluctuating normally, the current prediction task is divided into a routine detection task.

[0024] If the conditions for classifying routine detection tasks are not met, but any of the following conditions are met, the current prediction task will be classified as a key detection task:

[0025] The forming stage is at the critical point of stage switching, the defect risk level is suspected defect or high-risk defect, the image quality status is slightly contaminated, and the equipment operation status is slightly abnormal; when the conditions for key detection tasks are not met, the current prediction task is classified as a routine detection task.

[0026] As a preferred option, before classifying the current prediction task as a key detection task, the following are also included: obtaining the phase switching proximity, the rate of change of abnormal high temperature area, the contour jitter value, and the normalized value of the distance from the target center point to the boundary of the detection area.

[0027] The pre-trigger value is determined based on the phase switching proximity, the abnormal high temperature area change rate, the contour jitter value, and the normalized value of the distance from the target center point to the detection area boundary.

[0028] When the pre-trigger value reaches the pre-trigger threshold set based on historical statistical data, the preparatory state is initiated; the preparatory state includes pre-allocation of edge computing resources, locking of key detection areas, and caching of images before and after triggering.

[0029] As a preferred option, the corresponding image acquisition control and model scheduling are executed, including: when the current prediction task is a regular detection task, controlling the image acquisition device to adopt the standard frame rate and standard exposure time determined based on the calibration results, and calling the model branch corresponding to the state of the forming stage to perform prediction;

[0030] When the current prediction task is the key detection task, the current sampling frame rate and current exposure time are determined according to the defect risk level, so as to control the image acquisition device to switch to a high frame rate greater than the standard frame rate and a short exposure acquisition time less than the standard exposure time.

[0031] Based on the defect risk level and / or pre-trigger value, determine the processing priority score of the current task; when the processing priority score is greater than the set priority threshold, put the current prediction task into the priority processing queue, and perform cropping and re-inspection on the images in the key detection area.

[0032] When the priority score is not greater than the set priority threshold, the current prediction task is placed in the regular queue for detection.

[0033] As a preferred method, based on the preliminary prediction results, combined with the image quality status and the equipment operating status, the corrected risk value or result reliability is determined, and the online prediction result containing the defect risk type, result reliability and the equipment status identifier generated based on the equipment operating status is output, including: obtaining the single frame image defect risk value within the current time and the set number of frames before and after;

[0034] Based on the defect risk value of a single frame image, determine the consistency risk value of multiple frames; based on the image quality status, equipment operating status, and data synchronization indicators, determine the final defect risk value.

[0035] When the final defect risk value is greater than the risk trigger threshold determined based on the process specification, and the final defect risk values ​​are consistent for at least a preset number of consecutive frames, the defect risk type is determined; based on the final defect risk value, image quality status, and equipment operating status, an online prediction result containing the defect risk type, result reliability, and equipment status identifier is generated and output.

[0036] Preferably, the method further includes: when the device is in a degraded operating state, obtaining the current image quality score and comparing it with a preset quality threshold; if the image quality score is lower than the preset quality threshold, blocking the abnormal image data from entering the deep learning defect prediction model; if the image quality score is greater than or equal to the preset quality threshold, allowing it to enter the deep learning defect prediction model.

[0037] Call upon backup image acquisition equipment or backup acquisition source, and mark the current batch of image data as abnormal restricted data;

[0038] When outputting online prediction results, attach device error event codes and store the abnormal restricted data in the error data area after marking them.

[0039] The beneficial effects of this invention are:

[0040] 1. By adopting the above technical solution, the present invention first establishes a unified reference time based on the system synchronization clock, and determines the data synchronization index by combining the time deviation between visible light image, infrared thermal image and production line cycle signal. Only when the preset conditions are met will the subsequent state judgment and defect prediction process be entered, which effectively solves the problem of misjudgment caused by multimodal data misalignment fusion when there are exposure delay, buffer delay and transmission jitter in different acquisition links.

[0041] 2. This invention obtains position progress values, average temperature gradient, contour shrinkage rate, and reflective area change rate, and determines the high-temperature forming stage index, cooling stage index, and contour stabilization stage index accordingly. It can accurately distinguish the true state of the casting under different process stages. Then, it combines the model branch corresponding to the forming stage state for prediction, effectively reducing the interference caused by the drastic changes in thermal field distribution, contour shrinkage, and surface reflection on the unified model recognition results. Attached Figure Description

[0042] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0043] Figure 1 This is a flowchart of an embodiment of an online prediction method for valve casting forming defects based on deep learning. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 A deep learning-based online prediction method for valve casting forming defects includes: acquiring multimodal image data and production line cycle data of valve castings online through image acquisition equipment. The multimodal image data includes at least visible light images and infrared thermal images, and the production line cycle data includes at least station trigger time and transmission position.

[0046] Based on multimodal image data and production line cycle data, determine the molding stage status, image quality status, defect risk level, and equipment operating status;

[0047] Based on the forming stage status, image quality status, defect risk level, and equipment operating status, the current prediction task is divided into routine detection tasks or key detection tasks, and corresponding image acquisition control and model scheduling are executed.

[0048] Multimodal image data is input into a deep learning defect prediction model. Based on the model scheduling results, the model branch corresponding to the forming stage state is called to make predictions, and preliminary prediction results containing candidate defect types are obtained.

[0049] Based on the preliminary prediction results, combined with the image quality status and equipment operating status, the corrected risk value or result reliability is determined, and the online prediction results containing the defect risk type, result reliability, and equipment status identifier generated based on the equipment operating status are output.

[0050] The system acquires multimodal image data and production line cycle time data of valve castings collected online, including: establishing a unified reference time based on the system synchronization clock; calculating the time deviation of the multimodal image data and production line cycle time data relative to the unified reference time; determining the data synchronization index at the current time based on the time deviation; when the data synchronization index meets the preset conditions determined based on the calibration results, executing the steps to determine the forming stage status, image quality status, defect risk level, and equipment operating status; when the data synchronization index does not meet the preset conditions, transferring the multimodal image data and production line cycle time data to a low-confidence queue.

[0051] Determine the state of the forming stage, including: obtaining the current position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate;

[0052] Based on the position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate, determine the indicators for the high-temperature forming stage, cooling stage, and contour stabilization stage.

[0053] The forming stage state is determined based on the maximum value among the indicators of the high-temperature forming stage, the cooling stage, and the contour stabilization stage.

[0054] Determine the image quality status, including: image blur, thermal overexposure ratio, high reflectivity area ratio, smoke obscuration index, and inter-frame alignment error of the acquired multimodal image data; determine the image quality score based on the image blur, thermal overexposure ratio, high reflectivity area ratio, smoke obscuration index, and inter-frame alignment error.

[0055] Based on the numerical range of the image quality score, determine the image quality status as clear, slightly contaminated, or heavily contaminated; determine the defect risk level, including: abnormal high temperature area change rate, boundary curvature abrupt change value, texture abnormal offset, contour deformation area, and indentation growth rate of the multimodal image data;

[0056] The defect risk score is determined based on the abnormal high temperature area change rate, boundary curvature abrupt change value, texture abnormal offset, contour deformation area, and indentation growth rate; the defect risk level is determined as normal fluctuation, suspected defect, or high-risk defect based on the numerical range of the defect risk score.

[0057] Determine the device's operating status, including: obtaining the current frame drop rate, thermal image overexposure alarm intensity, supplementary lighting anomaly value, processor frequency reduction index, and cache time difference index; and determine the current device health score based on the frame drop rate, thermal image overexposure alarm intensity, supplementary lighting anomaly value, processor frequency reduction index, and cache time difference index.

[0058] At the initial moment, the device status value is set to a preset initial value; at non-initial moments, the current device status value is determined based on the current device health score and the device status value at the previous moment.

[0059] Based on the numerical range of the equipment status value, determine whether the equipment is operating normally, slightly abnormal, or in a degraded state.

[0060] This implementation is applicable to online prediction scenarios in valve casting production lines; the system consists of a visible light acquisition unit, an infrared thermal imaging acquisition unit, a production line cycle and position sensing unit, an edge computing node, and a result output unit.

[0061] The visible light acquisition unit outputs surface texture images and reflection distribution images, the infrared thermal imaging acquisition unit outputs thermal field images of the target area, and the production line cycle time and position sensing unit outputs workstation trigger time and conveyor position.

[0062] Edge computing nodes are used to perform time alignment, state determination, model invocation, and result output.

[0063] At the current moment, the edge computing node first reads the data packet of the current moment from each acquisition unit; the data packet includes at least the visible light frame, thermal image frame, corresponding timestamp, workstation trigger time, and transmission location;

[0064] Multimodal image data refers to images of valve castings obtained by different imaging methods, including at least visible light images and infrared thermal images;

[0065] Production line cycle time data refers to the production rhythm information given by cycle time triggers, position encoders or similar position sensing components, including at least the workstation trigger time and the conveyor position.

[0066] Because different acquisition links have exposure delay, buffer delay and transmission jitter, the edge computing node first establishes a unified reference time based on the system synchronization clock, and then calculates the time deviation of each modal data relative to the reference time;

[0067] In this embodiment, the edge computing nodes process data according to the alignment unit method; for the current alignment unit, the data closest to the unified reference time from the visible light image, infrared thermal image and production line cycle signal are taken as candidate inputs, and the time deviation of each candidate input is calculated.

[0068] The time deviation can be taken as the absolute value of the difference between the candidate input timestamp and the unified reference time; the edge computing nodes normalize the time deviation of each mode according to the preset weight to obtain the data synchronization index.

[0069] The calculation of the data synchronization index is achieved through the following logic: set an upper limit for the allowable time deviation based on the high quantile value of the deviation distribution during normal equipment operation, divide the time deviations of visible light images, infrared thermal images and production line cycle signals by the upper limit for the allowable time deviation respectively and limit the maximum value to 1 to obtain three initial deviation rates between 0 and 1.

[0070] Considering that infrared thermal imaging has the lowest tolerance for the characterization delay of molding temperature changes, the highest preset weight for infrared thermal imaging is given, for example, 0.5, while the remaining preset weights for visible light images and production line cycle signals are, for example, 0.25 each.

[0071] Subtract each initial deviation rate from 1, multiply by the corresponding weight, and sum to obtain a data synchronization index with a value between 0 and 1.

[0072] The preset conditions are preferably set based on the offline calibration results. The upper limit of the allowable time deviation can be taken as the high quantile value of the deviation distribution when the equipment is running normally. The effective synchronization threshold can be selected in the range of 0.70 to 0.85. It is preferred to adjust it according to the stability of the production line cycle and the number of frames of defect precursors.

[0073] Only when the current data synchronization index meets the preset condition will it enter the subsequent state judgment and defect prediction process; when the condition is not met, the edge computing node will not send the alignment unit into the main prediction process, but will only transfer it to the low-confidence data queue to avoid incorrect fusion caused by time misalignment.

[0074] Once the data synchronization indicators meet the preset conditions, the edge computing node begins to determine the forming stage status; the forming stage status is used to reflect the stage label of the casting currently being in the high-temperature forming, cooling transition, or contour stabilization stage;

[0075] In this embodiment, the edge computing node first calculates the position progress value, average temperature gradient, contour shrinkage rate and reflective area change rate; the position progress value is obtained by normalizing the displacement given by the position sensing unit on the preset detection path, and its value is in the range of 0 to 1, which is used to characterize the relative progress of the casting along the detection path.

[0076] The average temperature gradient is obtained by averaging the temperature changes of adjacent pixels in the target area of ​​the infrared thermal image, and is used to reflect the strength of the thermal field change.

[0077] The contour shrinkage rate is obtained by the shrinkage ratio of the current contour area relative to the reference contour area, which can be taken as the reference contour area of ​​a normal product at the corresponding process stage; the reflective area change rate is obtained by normalizing the change in the area of ​​the bright reflective region in the visible light image compared to the previous moment.

[0078] After obtaining the above four types of quantities, the edge computing nodes calculate the high-temperature forming stage index, cooling stage index and contour stabilization stage index respectively according to the weight combination preset based on the process characteristics.

[0079] For each stage, the edge computing node linearly normalizes the position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate in the interval of 0 to 1, and then performs a weighted summation to obtain the corresponding stage index.

[0080] In this normalization process, for variables with physical dimensions such as the average temperature gradient, the maximum and minimum values ​​of the corresponding index distributions under historical normal process conditions are used, such as the 2% to 98th percentile, as the cutoff benchmarks for the upper and lower limits of normalization, so as to eliminate the scale bias caused by different physical units.

[0081] When calculating indicators at different stages, the weight allocation is set according to different process characteristics: when calculating indicators at the high-temperature forming stage, the sum of the weight of the average temperature gradient and the weight of the rate of change of reflective area is set to be greater than 0.6 to highlight the dominant role of thermal field and reflection.

[0082] When calculating the cooling stage indicators, the weight of the contour shrinkage rate is set to the highest within the same group; when calculating the contour stabilization stage indicators, the weight of the position progress value is set to the highest; then the stage corresponding to the maximum value among the three stage indicators is taken as the current forming stage state.

[0083] Since the thermal field gradient in the high-temperature forming stage is greater than the first set threshold and the rate of change of reflective area is greater than the second set threshold, the temperature gradient decrease rate in the cooling stage is higher than the contour shrinkage rate, and the contour change rate in the contour stabilization stage is lower than the set stability threshold, the system characterizes the features of different process stages through weighted parameters of multiple physical quantities.

[0084] While determining the state of the forming stage, the edge computing nodes also calculate the image quality state and defect risk level; the image quality state is a quality label indicating whether the current image is suitable for entering the model prediction.

[0085] Edge computing nodes extract image blur, thermal overexposure ratio, high reflectivity area ratio, smoke occlusion index, and inter-frame alignment error from the current multimodal image data; image blur can be calculated by the image edge sharpness, preferably by using Laplacian variance inverse normalization to obtain the blur;

[0086] The overexposure ratio of a thermal image can be calculated based on the percentage of pixels in the thermal image that reach the saturation temperature zone; the percentage of highly reflective areas is calculated based on the percentage of the area of ​​the bright areas in the visible light image within the target area.

[0087] The smoke obscuration index is obtained through quantitative calculation: the ratio of the global variance contrast of the current image of the target area to the contrast of the standard reference image without smoke, and the ratio of the mean Laplacian edge gradient of the current image to the standard reference image. Subtract the above two ratios by 1 respectively, and after limiting the minimum value to not be lower than 0, take the arithmetic mean of the two as the smoke obscuration index.

[0088] Inter-frame alignment error is used to reflect the alignment deviation of target regions in adjacent frames; the edge computing node uses the following linear penalty formula to weight the above five indicators to obtain the image quality score. :

[0089]

[0090] in, These represent normalized mappings to... Image blur after interval, thermal overexposure ratio, proportion of highly reflective areas, smoke obscuration index, and inter-frame alignment error, with fixed weight values ​​configured as follows. ;

[0091] Image quality scores can be represented using a range of 0 to 1, with higher values ​​indicating better quality. Based on the range of the image quality score, the image quality status is divided into clear, slightly polluted, or heavily polluted. The boundary of this range is preferably determined based on the intersection of the distribution of historical valid and invalid samples, and then adjusted jointly by the false alarm rate and the false negative rate.

[0092] At the same time, the edge computing nodes also extract the abnormal high temperature area change rate, boundary curvature mutation value, texture anomaly offset, contour deformation area and depression growth rate based on the current multimodal image data;

[0093] The specific extraction rule for the abnormal high temperature area change rate is as follows: the abnormal high temperature area change rate is the difference between the total number of consecutive connected pixels in the infrared thermal image whose temperature exceeds the preset high temperature threshold and the total number of corresponding pixels in the previous frame, divided by the total number of target pixels.

[0094] The method for extracting boundary curvature abrupt change values ​​is to calculate the second derivative curvature of each edge point along the target contour of the visible light image, and to obtain the maximum range of curvature of continuous edge points in the current frame, which is used to reflect whether the local bending change of the contour exceeds the normal fluctuation.

[0095] Texture anomaly offset is obtained by extracting the gray-level co-occurrence matrix of a specified detection area in a visible light image, and calculating four statistics of the matrix: contrast, energy, homogeneity, and correlation to construct a feature vector. The Euclidean distance between this vector and the pre-stored normal standard texture feature vector is calculated to reflect whether the surface texture distribution deviates from the normal sample.

[0096] The contour deformation area is obtained by calculating the pixel area of ​​the non-overlapping region between the currently extracted image outer contour and the standard reference contour after spatial affine alignment based on the centroid and principal axis. It is used to characterize the area of ​​the contour difference region.

[0097] The indentation growth rate is obtained by calculating the slope of the linear fitting of the area of ​​the local low temperature region in the infrared thermal image and the corresponding visible light shadow gradient in multiple consecutive frames, which is used to characterize how fast the local indentation region grows over time.

[0098] To ensure that the physical dimension consistency is not compromised when multi-source physical features are fused in the same mathematical space, edge computing nodes use a truncation mapping method to eliminate physical dimensions.

[0099] For features such as boundary curvature abrupt change values ​​and indentation growth rates with physical dimensions, edge computing nodes extract the maximum and minimum values ​​of the distribution of this physical quantity feature statistically obtained under the same molding stage in the historical normal process, such as the 1% to 99th percentile, as the boundary;

[0100] Edge computing nodes map these five types of features to a dimensionless interval of 0 to 1 based on their corresponding extreme value boundaries for normalization, and truncate values ​​that exceed the boundaries to the upper and lower limits respectively.

[0101] Edge computing nodes assign preset weights to the risk significance of the current process stage according to each feature and perform weighted summation to obtain a defect risk score in the range of 0 to 1;

[0102] As a specific example, during the high-temperature forming stage, the weights of the abnormal high-temperature area change rate and the boundary curvature mutation value are set to 0.35 and 0.25, respectively, while the texture abnormal offset, contour deformation area and depression growth rate are set to 0.15, 0.15 and 0.1, respectively.

[0103] After obtaining the defect risk score, it is then divided into normal fluctuations according to the preset range, such as a score less than 0.4, suspected defects, such as a score between 0.4 and 0.7, or high-risk defects, such as a score greater than or equal to 0.7.

[0104] Defect risk classification is based on the physical change characteristics of thermal field, boundary morphology and surface texture, and the quantitative classification result is calculated by a preset image pixel feature formula;

[0105] In terms of determining the operating status of the device, the edge computing node periodically reads data from the acquisition link and the computing link, including frame loss rate, thermal image overexposure alarm intensity, abnormal supplementary lighting value, processor frequency reduction index, and cache time difference index.

[0106] Among them, the frame drop rate represents the proportion of image frames that were not successfully received within the detection window at the current moment; the thermal overexposure alarm intensity is obtained by dividing the number of saturated pixels of the thermal imaging device by the number of effective pixels;

[0107] The abnormal value of the supplementary light is obtained by calculating the ratio of the absolute value of the deviation between the output current of the supplementary light driving unit and the rated current. The supplementary light driving unit can be configured inside or around the aforementioned visible light acquisition unit. The processor frequency reduction index is the ratio of the difference between the edge computing node's operating frequency and the maximum rated main frequency to the rated main frequency.

[0108] The cache latency metric is the ratio of the millisecond-level delay between the timestamp when an image enters the cache and the timestamp when it is read by the inference task to the preset maximum tolerable latency.

[0109] Edge computing nodes first process the feature vector The five components ( This means that these five non-negative indicators are truncated and normalized with an upper limit of 1, and then weighted according to a set weight vector. corresponding components Calculate the current time Current device health score The computational logic is embodied in the following evaluation formula:

[0110]

[0111] in, Indicator value The larger the value, the higher the abnormality and the lower the health score. At the initial moment, the device status value is set to a preset initial value, which is preferably the health status value of 1.0 corresponding to the device after factory calibration.

[0112] At non-initial moments, edge computing nodes perform smooth updates based on the current device health score and the device state value from the previous moment to reduce false handovers caused by sudden noise at a single abnormal moment; the smooth update process uses an exponentially weighted moving average formula to perform hysteretic updates on the state values:

[0113]

[0114] in, This represents the current device status value. The device state value at the previous moment, smoothing factor The value is set in the range of 0.1 to 0.3 to ensure that the current device status value is both dominated by the current health level and retains the smooth continuity of the previous time.

[0115] Based on the tolerance limit obtained from offline testing, two thresholds are set, such as 0.6 and 0.85. and According to the device status value The value range in which the device operates is defined as normal. Mild abnormality, i.e. Or it may be downgraded to a lower level, i.e. ;

[0116] After determining the forming stage status, image quality status, defect risk level, and equipment operating status, the edge computing node performs task classification. When the image quality status is clear, the equipment operating status is normal, and the defect risk level is within normal fluctuation range, the system classifies the current prediction task as a routine detection task.

[0117] Conversely, the processing mode that requires increased sampling and computing resources is switched to; the edge computing node calls the deep learning model branch corresponding to the current forming stage state; the so-called model branch refers to the defect prediction branch network trained for different forming stages, which is used to reduce the interference of visual differences at different stages on the same model.

[0118] For example, during the high-temperature forming stage, the model branch corresponding to hot spots and strong reflection features is called; during the cooling stage, the model branch corresponding to contour shrinkage and texture change features is called; and during the contour stabilization stage, the model branch that focuses more on the stable surface defect performance is called.

[0119] After the multimodal image data enters the corresponding branch, the edge computing node obtains the preliminary prediction results, which include at least the single-frame defect risk value and candidate defect type.

[0120] To ensure the transparency and feasibility of the algorithm architecture, the deep learning defect prediction model and its branches specifically adopt a dual-stream network feature extraction structure based on residual networks. One network stream processes visible light images, and the other network stream processes infrared thermal images. They are spliced ​​and fused in the channel dimension. That is, the single-channel infrared thermal image and the three-channel red, green and blue visible light image are combined along the channel axis through tensor splicing operation to obtain a three-dimensional tensor structure with 4 channels as the feature data of the network input. Its output vector dimension is generated by the activated prediction head module, which contains a one-dimensional classification array indicating the confidence of candidate defect categories and a single quantization scalar used to characterize the defect risk value of a single frame.

[0121] The activated prediction head module includes a shared global average pooling layer, and three independent multilayer perceptron branch networks corresponding to the high-temperature forming stage, the cooling stage, and the contour stabilization stage, respectively.

[0122] Based on the determined state label of the forming stage, the edge computing node adopts a hard routing mechanism to activate only a single multilayer perceptron branch network of the corresponding stage at a given time.

[0123] Each multilayer perceptron branch network consists of two fully connected layers. One layer, through a fully connected mapping, outputs lengths of [length missing]. One-dimensional classification arrays and single-dimensional risk quantification scalars, among which The total number of preset defect categories

[0124] Furthermore, the offline training samples for each model branch were obtained by collecting multimodal images from historical production batches and manually verifying and labeling defects. The training loss function... The mathematical expression is:

[0125]

[0126] in, Cross-entropy loss for classification tasks; The mean squared error loss for the risk value prediction task; This is the preset dynamic balance coefficient.

[0127] This achieves a precise mapping from image pixel input to risk quantification result output;

[0128] Edge computing nodes correct the preliminary prediction results; the correction is based on at least the image quality status and the device operating status; if the image quality status deteriorates, it indicates that the current image may be blurry, overexposed, or occluded, and the confidence level of the preliminary prediction results should be reduced accordingly.

[0129] If the equipment is operating under slightly abnormal or degraded conditions, it indicates that there is a degradation in the data acquisition or computing links, and the risk value or result reliability should be corrected accordingly.

[0130] In this embodiment, instead of using the credible reduction method to physically attenuate the absolute value of the risk in the preliminary prediction result, the true magnitude of the final defect risk value obtained from the preliminary prediction is kept unchanged, and the image quality weight and device health weight are multiplied by the data synchronization index to decouple and generate a reliable quantification value of the result independent of the risk value.

[0131] Alternatively, the risk type can remain unchanged, and only a lower level of result reliability can be output; finally, the result output unit outputs the online prediction result, which includes at least the defect risk type, result reliability, and equipment status identifier generated based on the equipment operating status;

[0132] Therefore, the production line control system can accurately distinguish between two different types of abnormal events: the high risk of the product itself and the abnormal warnings caused by the limited status of the equipment. This effectively reduces the misjudgment rate caused by fluctuations in the status of the collected equipment.

[0133] In this implementation, by first performing unified time alignment, and then performing stage determination, image quality determination, defect risk determination, and equipment operating status determination, the data input to the model can be made closer to the real image status under the same process stage.

[0134] Because the model branches are called in stages, and the final result is corrected based on image quality and equipment operating status, it can improve the stability of online prediction and reduce the interference of abnormal images and equipment degradation on the final quality conclusion in production line environments where high temperature reflection, smoke obstruction and link fluctuations coexist.

[0135] In a preferred embodiment of the present invention, classifying the current prediction task into a routine detection task or a key detection task includes: classifying the current prediction task into a routine detection task when the image quality is clear, the device is operating normally, and the defect risk level is within normal fluctuation range; and classifying the current prediction task into a key detection task when the routine detection task classification conditions are not met, but any of the following conditions are met:

[0136] The forming stage is at the critical point of stage switching, the defect risk level is suspected defect or high-risk defect, the image quality status is slightly contaminated, and the equipment operation status is slightly abnormal; when the conditions for key detection tasks are not met, the current prediction task is classified as a regular detection task.

[0137] Before classifying the current prediction task as a key detection task, the process also includes: obtaining the phase transition proximity, abnormal high temperature area change rate, contour jitter value, and normalized distance value from the target center point to the detection area boundary; and determining the pre-trigger value based on the phase transition proximity, abnormal high temperature area change rate, contour jitter value, and normalized distance value from the target center point to the detection area boundary.

[0138] When the pre-trigger value reaches the pre-trigger threshold set based on historical statistical data, the preparatory state is initiated; the preparatory state includes pre-allocation of edge computing resources, locking of key detection areas, and caching of images before and after triggering;

[0139] Execute the corresponding image acquisition control and model scheduling, including: when the current prediction task is a regular detection task, control the image acquisition device to adopt the standard frame rate and standard exposure time determined based on the calibration results, and call the model branch corresponding to the state of the forming stage to perform prediction;

[0140] When the current prediction task is the key detection task, the current sampling frame rate and current exposure time are determined according to the defect risk level, so as to control the image acquisition device to switch to a high frame rate greater than the standard frame rate and a short exposure acquisition time less than the standard exposure time.

[0141] Based on the defect risk level and / or pre-trigger value, determine the processing priority score of the current task; when the processing priority score is greater than the set priority threshold, put the current prediction task into the priority processing queue, and perform cropping and re-inspection on the images in the key detection area.

[0142] When the priority score is not greater than the set priority threshold, the current prediction task is placed in the regular queue for detection.

[0143] This embodiment is applicable to scenarios where valve castings transition from one forming stage to another and the duration of defect precursors is short. The main execution entity of this embodiment is still the edge computing node, and the image acquisition device is the controlled entity, specifically including the aforementioned visible light acquisition unit and infrared thermal imaging acquisition unit.

[0144] Based on the forming stage status, image quality status, defect risk level, and equipment operating status obtained from previous steps, the edge computing node further determines whether the current task is a routine detection task or a key detection task, and adjusts the acquisition parameters and inference order accordingly.

[0145] Routine inspection tasks refer to inspection tasks performed at the current moment when the image is clear, the equipment is operating normally, and the defect risk only shows normal fluctuations.

[0146] Under this condition, the edge computing node sends standard acquisition parameters to the image acquisition device. The standard acquisition parameters are determined by the device calibration results and include at least the standard frame rate and standard exposure time.

[0147] The standard frame rate and standard exposure time are a set of dynamically adapted parameters selected through offline calibration under the corresponding production line cycle time and imaging distance conditions. The setting criterion is to minimize the utilization of transmission bandwidth and computing resources while meeting the basic image clarity requirements.

[0148] Edge computing nodes simultaneously call the model branch corresponding to the current forming stage state for prediction; since routine detection tasks correspond to relatively stable operating conditions, the system focuses on maintaining continuous monitoring and reducing resource consumption at this time.

[0149] Key detection tasks refer to detection tasks that are executed when the conditions for classifying regular detection tasks are not met, but at least one key trigger condition is met.

[0150] The key triggering conditions here include: the forming stage is at a critical point of stage transition, the defect risk level is suspected defect or high-risk defect, the image quality status is slightly contaminated, or the equipment operation status is slightly abnormal; the critical point of stage transition is determined based on the comparison results of stage indicators. If the dominant indicator of the current stage is close to the indicator of the adjacent stage, it indicates that the casting is in a process transition state. If the rate of change of thermal field, boundary and reflection features is greater than the set transition gradient threshold, the system adjusts the sampling rate to obtain a state image with a feature duration less than the set duration.

[0151] When the image quality is clear, the device is operating normally, and the defect risk level is fluctuating normally, the system will classify the current prediction task as a routine detection task. When the device is operating slightly abnormal, it means that the link has not yet failed, but there has been some fluctuation. The system should increase the processing priority in advance to avoid the loss of key frames.

[0152] Before officially classifying the current prediction task as a key detection task, the edge computing node first calculates the pre-trigger value; the pre-trigger value is used to instruct the system to allocate computing and caching resources within the time window of the preparation state to compensate for the communication and response time span of hardware parameter instructions;

[0153] The pre-trigger value is jointly determined by the phase switching proximity, the abnormal high temperature area change rate, the contour jitter value, and the normalized value of the distance from the target center point to the detection area boundary; the phase switching proximity can be obtained by normalizing the difference between adjacent phase indicators, and the smaller the difference, the closer it is to the phase switching area;

[0154] The rate of change in area of ​​abnormally high temperatures reflects whether suspected hot spots are expanding rapidly; contour jitter value The extraction logic is as follows: calculate continuous The mean square spatial displacement of the target contour centroid coordinates relative to the average centroid in a frame of visible light image is calculated using the following formula after normalization:

[0155]

[0156] in, For the first The centroid coordinates of the frame's outline. for Average centroid coordinates within the frame, The maximum effective diagonal length of the detection area is used to eliminate the dimensional effects caused by image resolution or scale, thereby quantifying the swing intensity of the target edge relative to the detection area in consecutive frames.

[0157] The normalized distance from the target center point to the boundary of the detection area is used to determine whether the target is about to enter the critical detection area. In order to eliminate the influence of dimensions caused by image resolution or scale, this distance normalization value is explicitly calculated by dividing the pixel distance from the target center point to the boundary by the maximum effective diagonal length of the detection area.

[0158] To ensure that the directions of various physical quantities are consistent during fusion, before performing weighted summation, the edge computing nodes perform reverse mapping processing on the phase switching proximity (where a smaller difference indicates a higher risk) and the distance normalization value (where a shorter distance makes it easier to trigger key detection), for example, by subtracting the normalization value from 1. This ensures that all the processed indicators contribute positively to the increase of the pre-trigger value. The edge computing nodes then perform weighted summation on these four quantities to obtain the pre-trigger value.

[0159] The preliminary trigger threshold is preferably set based on historical statistical data. Specifically, the distribution of the pre-trigger value in confirmed precursor samples before entering the critical detection area can be compared with the distribution of normal samples. The boundary value with the lower misjudgment between the two distributions can be taken as the initial threshold, and then fine-tuned according to the review rate of the production line.

[0160] When the pre-trigger value reaches the pre-trigger threshold, the edge computing node starts the preparatory state. In the preparatory state, the edge computing node pre-allocates edge computing resources. That is, the edge computing node adjusts the thread scheduling strategy by calling the operating system application programming interface, forcibly raising the priority of the key inference process to the highest real-time level. For example, it calls the portable operating system interface library to apply the first-in-first-out scheduling strategy. At the same time, it pre-allocates and locks a fixed-size contiguous tensor memory pool in the graphics processor memory management interface, which is dedicated to the key detection model branch coverage read and write, so as to eliminate the time consumption of dynamic memory allocation at runtime.

[0161] The key inspection areas are identified, which are usually thick-walled areas, flange transition areas, gate residue areas, or areas with concentrated thermal anomalies. At the same time, the image buffer before and after the trigger is activated, and the images within a certain number of moments before and after the pre-trigger are saved in the annular buffer area so that they can be compared before and after the key inspection task is actually started.

[0162] In the ready state, the edge computing node pre-sends a parameter instruction queue to the image acquisition device through the underlying control link to start the parameter adjustment cycle of the physical hardware;

[0163] The system initiates a preparatory state and performs a preset resource allocation operation before the risk indicators reach the trigger condition threshold in order to obtain an image sequence that meets the set threshold of the precursor feature parameters.

[0164] Once the edge computing node determines that the current task is a key detection task, it begins to execute differentiated image acquisition control and model scheduling; and determines the current sampling frame rate and current exposure time based on the risk score corresponding to the defect risk level.

[0165] The edge computing node determines the corresponding target sampling frame rate and target exposure time based on the preset classification threshold of the risk score, and sends the target sampling frame rate greater than the standard frame rate and the target exposure time shorter than the standard exposure time to the image acquisition device, so that the image acquisition device switches to high frame rate acquisition higher than the standard frame rate and short exposure acquisition shorter than the standard exposure time.

[0166] High frame rates are beneficial for capturing hot spot and contour changes with short durations, while short exposures help suppress high-temperature reflections and motion blur.

[0167] Because the sum of the communication delay of the exposure adjustment command issued by the software interface and the effective time of the photosensitive element register is greater than the time window for the maintenance of the precursor feature;

[0168] Therefore, this implementation abandons the gradual approximation smooth adjustment strategy during short-term abrupt changes and instead utilizes the multi-set preset parameter configuration function of the industrial camera.

[0169] The specific action logic is as follows: When starting the preparatory state, the edge computing node injects the key detection configuration file containing the target short exposure and high frame rate parameters into the image acquisition device in advance through the underlying communication link, and completes the preloading.

[0170] When the pre-trigger value jumps further and meets the conditions for key detection tasks, the edge computing node sends a signal level jump command to the image acquisition device through a microsecond-level hardware optocoupler isolation pin, forcing the camera hardware to switch directly to the pre-loaded key detection configuration file within one sensor readout cycle;

[0171] By switching from software command to hardware level triggering, the physical lag and intermediate transition state of exposure adjustment within the extremely short precursor window are eliminated, ensuring that a clear image with correct exposure can be obtained as soon as the precursor appears.

[0172] Under key detection tasks, edge computing nodes also determine the processing priority score of the current task based on the defect risk level and pre-trigger value;

[0173] To construct specific conversion rules from function to result, the edge computing node performs linear normalization on the quantitative risk score and pre-trigger value associated with the current defect risk level, and then assigns weights according to the preset scheduling priority. For example, the weight of the quantitative risk score is set to 0.6 and the weight of the pre-trigger value is set to 0.4. The weighted sum is then performed to generate a processing priority score in the range of 0 to 1.

[0174] The processing priority score is generated based on the quantified defect risk score and pre-trigger value, and is used to indicate and schedule the execution order of the current task in system resources;

[0175] If the processing priority score is greater than the set priority threshold, such as 0.75, the edge computing node will put the current task into the priority processing queue and perform cropping and re-inspection on the image in the key detection area.

[0176] The cropping and re-inspection process involves generating a bounding box with a preset pixel size of 256×256 or 512×512, centered on the geometric center or the point of maximum contour deformation of the abnormally high temperature area. Only the key detection area corresponding to the bounding box is cropped, and the cropped area data is then sent separately to the corresponding model branch or subjected to short-window multi-frame re-inspection.

[0177] The cropping and re-inspection mechanism reduces the computational cost of non-critical image areas and increases the detection frequency of defect feature target areas by extracting the set feature bounding boxes.

[0178] If the processing priority score is not greater than the set priority threshold, the current task will still be placed in the regular queue for detection, and only the key monitoring mark will be maintained without starting the highest level processing.

[0179] As the casting transitions from the high-temperature stage to the cooling stage, the boundary of the high-temperature region in the thermal image begins to change, and the highly reflective region in the visible light image also fluctuates.

[0180] At this point, the proximity of the phase transition increases, the rate of change of the abnormal high temperature area and the contour jitter value rise simultaneously, the pre-trigger value reaches the threshold, and the edge computing node first enters the preparatory state and locks the key area.

[0181] If the defect risk level continues to rise to suspected defect, the system will immediately switch to key detection task, increase the frame rate and shorten the exposure, and put the key areas into the priority processing queue for cropping and re-inspection.

[0182] Compared to using high frame rate full-image detection from the beginning, the system extracts short-term image features by adjusting the acquisition frequency and inference parameters when triggering key detection tasks, and maintains the computing power occupancy rate of edge computing nodes within the preset load threshold range.

[0183] In this implementation, there is a clear sequential relationship between task division, pre-triggering, and data acquisition scheduling: first, it is determined whether the normal detection conditions are met based on the existing status labels; if not, the pre-trigger value is further calculated and it is decided whether to enter the preparatory state.

[0184] Based on the preparatory state, high frame rate short exposure acquisition and priority queue processing are initiated according to the key detection task conditions. This sequential arrangement can reduce the loss of key frames due to insufficient resource preparation during stage switching and improve the success rate of capturing short images of suspected defects.

[0185] In a preferred embodiment of the present invention, based on the preliminary prediction results, combined with the image quality status and the device operating status, the corrected risk value or result reliability is determined, and an online prediction result including the defect risk type, result reliability and device status identifier generated based on the device operating status is output, including: obtaining the single-frame image defect risk value within the current time and the range of a set number of frames before and after.

[0186] Based on the defect risk value of a single frame image, determine the consistency risk value of multiple frames; based on the image quality status, equipment operating status, and data synchronization indicators, determine the final defect risk value.

[0187] When the final defect risk value is greater than the risk trigger threshold determined based on the process specification, and the final defect risk values ​​are consistent for at least a preset number of consecutive frames, the defect risk type is determined.

[0188] Based on the final defect risk value, image quality status, and equipment operating status, an online prediction result containing defect risk type, result reliability, and equipment status identifier is generated and output.

[0189] It also includes: when the device is running in a degraded state, obtaining the current image quality score and comparing it with a preset quality threshold; if the image quality score is lower than the preset quality threshold, blocking the abnormal image data from entering the deep learning defect prediction model; if the image quality score is greater than or equal to the preset quality threshold, allowing it to enter the deep learning defect prediction model.

[0190] Call up backup image acquisition devices or backup acquisition sources, and mark the current batch of image data as abnormal restricted data; when outputting online prediction results, attach device abnormal event codes, and store the abnormal restricted data in the abnormal data area after marking them.

[0191] This implementation method is applicable to the result correction and anomaly isolation process after the initial prediction of the model has been completed; the execution entity includes edge computing nodes, result output units, anomaly data storage units, and backup image acquisition devices or backup acquisition sources that can be called up when the device is downgraded;

[0192] The reliability of the results here refers to the degree of credibility of the current prediction conclusion under the combined effects of image quality, device health, and data synchronization conditions; the device status identifier refers to the output label used to indicate whether the device is operating normally, slightly abnormal, or in a degraded state.

[0193] After obtaining the preliminary prediction result at a certain moment, the edge computing node does not directly output the result as the final conclusion. Instead, it first takes the defect risk value of a single frame image within the current moment and the set number of frames before and after it.

[0194] The frame range is set as a short window range around the current moment. The width of the window is determined by the duration of the defect precursor and the stability of the production line cycle. For example, 3 to 5 frames before and after the current moment are taken, and the transient noise data of a single frame is filtered out by the temporal continuity parameter of adjacent image frames.

[0195] To address the logical delay issue caused by the requirement to wait for a set number of frames to be acquired in online detection systems, edge computing nodes introduce a sliding window lag judgment mechanism: after a single frame risk value is generated, it is temporarily stored in a buffer queue until the acquisition link completes the input of subsequent set frames and completes single frame inference, and then the prediction array of the short window range is extracted with the target time as the center.

[0196] The resulting decision delay is strictly limited to the physical response time of the production line rejection mechanism, ensuring physical consistency between the multi-frame timing verification logic and the production line control link;

[0197] Edge computing nodes perform weighted combination of risk values ​​of each single frame within a short window to obtain a consistent risk value across multiple frames. The weights can be allocated according to their distance from the current time. Specifically, a distance decay weight allocation rule is adopted, that is, with the frame at the current time as the benchmark, the corresponding weight decreases by a set decay ratio of 0.8 for each additional frame from the current time. Then, all weights within the short window are summed and the allocated weight of each single frame is divided by the sum to achieve normalization, so that frames closer to the current time can have higher weights.

[0198] This can avoid abnormal increases in risk values ​​caused by single-frame reflections, slight occlusions, or short-term jitter, thereby improving the stability of timing judgment.

[0199] After obtaining the consistent risk value across multiple frames, the edge computing node combines the image quality status, device operating status, and data synchronization indicators to calculate the final defect risk value.

[0200] The multi-frame consistency risk value and the result reliability are calculated independently and output through dual channels. The multi-frame consistency risk value is the final defect risk value without correction and is not subject to attenuation calculation with image quality status or device operating status parameters to prevent the output of the multi-frame consistency risk value from falling below the risk trigger threshold.

[0201] Pre-set decreasing quality characterization coefficients for clear, lightly polluted, and heavily polluted image quality states, for example, 1.0, 0.8, and 0.5 respectively. Set decreasing equipment characterization coefficients for normal, slightly abnormal, and degraded equipment operating states. Multiply these two coefficients by the data synchronization index in the range of 0 to 1 to obtain the final reliable quantification value.

[0202] When the final defect risk value approaches or exceeds the risk trigger threshold, if the corresponding result reliability quantification value is lower than the set safety baseline, the system will no longer determine that there is no defect, but will forcibly upgrade to the data unreliability high-risk warning type.

[0203] When the device is degraded or the image quality score is lower than the preset quality threshold, the system outputs a failure risk warning signal that includes the environmental quality status limit code and the device limit code.

[0204] In this embodiment, the risk trigger threshold is determined based on the process specification; the process specification can provide the allowable range of thermal field fluctuations, contour changes, and surface texture changes at different stages, and the edge computing node compares the final defect risk value with the risk trigger threshold accordingly.

[0205] The edge computing node determines the defect risk type only when the final defect risk value is greater than the threshold and the final defect risk values ​​of at least 3 consecutive frames are consistent. Here, consistency means that the risk judgment direction of consecutive frames is the same, such as all pointing to suspected shrinkage hole, increased cold shut risk, or mixed related anomalies, rather than repeatedly switching back and forth.

[0206] By adopting a rule that is consistent for at least three consecutive frames, false triggering caused by occasional interference at a single moment can be reduced; edge computing nodes generate online prediction results based on the final defect risk value, image quality status, and device operating status.

[0207] The reliability of the results can be output in high, medium, and low levels, or in range values; the equipment status indicator directly reflects whether the equipment is currently in a normal, slightly abnormal, or degraded operating state.

[0208] In the scenario where the device is in degraded operation, this implementation method further performs anomaly isolation; degraded operation means that the edge computing node has determined based on the frame loss rate, thermal image overexposure alarm intensity, supplementary lighting anomaly value, processor frequency reduction index and cache time difference index that the current acquisition link or computing link is continuously degraded, and continuing to process according to the normal process will significantly affect the prediction results;

[0209] At this point, the edge computing node compares the current image quality score with a preset threshold; if the image quality score is lower than the threshold, the current image data is considered to be abnormal image data, and it is blocked from entering the deep learning defect prediction model.

[0210] The threshold is preferably obtained from the statistics of historical invalid samples. The boundary between the distribution of valid samples and severely interfering samples can be taken as the initial value and adjusted according to the false alarm rate.

[0211] When the device is in a degraded operating state and the image quality is lower than the preset quality threshold, abnormal image data is blocked from entering the deep learning defect prediction model in order to avoid judging the device fault characteristics as product abnormalities.

[0212] After blocking abnormal image data from entering the model, the edge computing node calls a backup image acquisition device or backup acquisition source;

[0213] The backup image acquisition device can be a redundant camera at the same workstation, or a backup input link of the same modality; the backup acquisition source can be a cached image from an adjacent time moment or other alternative input that can still be used after synchronization filtering.

[0214] The purpose of calling up backup acquisition devices or backup acquisition sources is to maintain predictive capabilities as much as possible when the main link is limited; at the same time, the edge computing node marks the current batch of image data as abnormally limited data; abnormally limited data is limited image data acquired by the device in a degraded operation state;

[0215] The above-mentioned labeling process facilitates the isolation of this restricted data from normal data in subsequent quality traceability and sample management, preventing data collected during equipment malfunctions from interfering with the accuracy of subsequent model training sample libraries and quality information databases;

[0216] When the system still needs to output online prediction results in a degraded operation state, the result output unit adds equipment abnormal event codes to the original defect risk type, result reliability and equipment status identifier;

[0217] The device anomaly event code is a structured, lightweight data exchange format message. Its underlying fields include at least: the timestamp of the anomaly occurrence, the anomaly category code (e.g., 0x01 indicates thermal imaging device saturation, 0x02 indicates processor frequency reduction, etc.), the number of milliseconds the anomaly lasts, and the current device computing power utilization rate, which are used to accurately characterize what kind of device degradation has affected the current result.

[0218] The abnormal data storage unit marks and stores abnormal restricted data in the abnormal data area. This data marking action is to add a set of identification fields at the database record level, including the source image's universally unique identification code, the restricted identification bit set to 1, and the associated abnormal category code, so as to facilitate subsequent manual review, equipment maintenance and troubleshooting, and statistical analysis. The abnormal data area is stored separately from the regular quality conclusion database, which can reduce the possibility of restricted samples entering the normal model training process.

[0219] When an edge computing node detects that the processor frequency reduction is persistent and the thermal overexposure alarm intensity increases at a certain moment, the device's operating status is determined to be degraded.

[0220] If the image quality score is lower than the threshold at this point, the system will stop sending the batch of images into the main prediction model and attempt to switch to the backup acquisition source.

[0221] If the backup acquisition source can still provide usable images, the system continues to generate online prediction results under constrained conditions according to the multi-frame consistency rule, but adds the device abnormal event code to the results and writes the corresponding data into the abnormal data area;

[0222] In this way, the production line can still obtain a risk result with limitations, rather than directly concluding that the equipment failure is a casting defect.

[0223] Through this implementation method, the model output results are first corrected for multi-frame consistency, and then corrected by image quality, device operating status and data synchronization indicators, which can improve the stability of the final results.

[0224] Meanwhile, when the equipment is degraded, blocking abnormal images, calling backup acquisition sources, and storing abnormal restricted data separately can reduce the contamination of product quality judgment by equipment abnormalities.

Claims

1. A deep learning-based online prediction method for valve casting forming defects, characterized in that, include: The image acquisition device acquires multimodal image data of valve castings and production line cycle data online. The multimodal image data includes at least visible light images and infrared thermal images, and the production line cycle data includes at least station trigger time and transmission position. Based on the multimodal image data and production line cycle data, determine the molding stage status, image quality status, defect risk level, and equipment operating status; Based on the forming stage status, image quality status, defect risk level, and equipment operating status, the current prediction task is divided into a routine detection task or a key detection task, and corresponding image acquisition control and model scheduling are executed. The multimodal image data is input into a deep learning defect prediction model. Based on the model scheduling result, the model branch corresponding to the forming stage state is called to make a prediction, and a preliminary prediction result containing candidate defect types is obtained. Based on the preliminary prediction results, combined with the image quality status and equipment operating status, the corrected risk value or result reliability is determined, and an online prediction result containing the defect risk type, result reliability, and equipment status identifier generated based on the equipment operating status is output.

2. The online prediction method for valve casting forming defects based on deep learning according to claim 1, characterized in that, Acquire multimodal image data of valve castings and production line cycle data collected online, including: Establish a unified reference time based on the system's synchronous clock; Calculate the time deviation of the multimodal image data and production line cycle time data relative to the unified reference time; Based on the time deviation, determine the data synchronization index for the current moment; When the data synchronization indicators meet the preset conditions determined based on the calibration results, the steps of determining the forming stage status, image quality status, defect risk level, and equipment operating status are executed; when the data synchronization indicators do not meet the preset conditions, the multimodal image data and production line cycle data are transferred to a low-confidence queue.

3. The online prediction method for valve casting forming defects based on deep learning according to claim 1, characterized in that, Determine the state of the molding stage, including: Obtain the current position progress value, average temperature gradient, profile shrinkage rate, and reflective area change rate; Based on the position progress value, average temperature gradient, contour shrinkage rate, and reflective area change rate, determine the indicators for the high-temperature forming stage, cooling stage, and contour stabilization stage. The forming stage state is determined based on the maximum value among the high-temperature forming stage index, the cooling stage index, and the contour stabilization stage index.

4. The online prediction method for valve casting forming defects based on deep learning according to claim 1, characterized in that, Determine the image quality status, including: The image blur, thermal overexposure ratio, high reflectivity area ratio, smoke obscuration index, and inter-frame alignment error of the multimodal image data are obtained. The image quality score is determined based on the image blur, thermal overexposure ratio, proportion of highly reflective areas, smoke obscuration index, and inter-frame alignment error. Based on the numerical range of the image quality score, the image quality status is determined to be clear, slightly polluted, or heavily polluted. Determine the defect risk level, including: The abnormal high temperature area change rate, boundary curvature abrupt change value, texture abnormal offset, contour deformation area and depression growth rate of the multimodal image data are obtained. The defect risk score is determined based on the abnormal high temperature area change rate, boundary curvature abrupt change value, texture anomaly offset, contour deformation area, and depression growth rate. Based on the numerical range of the defect risk score, the defect risk level is determined to be normal fluctuation, suspected defect, or high-risk defect.

5. The online prediction method for valve casting forming defects based on deep learning according to claim 1, characterized in that, Determine the equipment's operating status, including: Obtain the current frame drop rate, thermal image overexposure alarm intensity, abnormal fill light value, processor downclocking index, and cache time difference index; The current device health score is determined based on the frame drop rate, thermal image overexposure alarm intensity, abnormal supplementary lighting value, processor frequency reduction index, and cache time difference index. At the initial moment, the device status value is set to a preset initial value; at non-initial moments, the current device status value is determined based on the current device health score and the device status value at the previous moment. Based on the numerical range of the device status value, the device's operating status is determined to be normal, slightly abnormal, or degraded.

6. The online prediction method for valve casting forming defects based on deep learning according to claim 1, characterized in that, The current prediction tasks are divided into routine detection tasks or key detection tasks, including: If the image quality is clear, the device is operating normally, and the defect risk level is fluctuating normally, the current prediction task is classified as a routine detection task. If the conditions for dividing the current detection task are not met, but any of the following conditions are met, the current prediction task will be divided into a key detection task: the forming stage is at the critical point of stage switching, the defect risk level is a suspected defect or a high-risk defect, the image quality status is slightly contaminated, and the equipment operation status is slightly abnormal. If the conditions for a key detection task are not met, the current prediction task will be classified as a regular detection task.

7. The online prediction method for valve casting forming defects based on deep learning according to claim 6, characterized in that, Before classifying the current prediction task as the key detection task, the method further includes: Acquire the phase switching proximity, abnormal high temperature area change rate, contour jitter value, and normalized value of the distance from the target center point to the detection area boundary; The pre-trigger value is determined based on the phase switching proximity, abnormal high temperature area change rate, contour jitter value, and normalized value of the distance from the target center point to the detection area boundary. When the pre-trigger value reaches the pre-trigger threshold set based on historical statistical data, the preparatory state is initiated; The preparatory state includes pre-allocating edge computing resources, locking key detection areas, and caching images before and after triggering.

8. The online prediction method for valve casting forming defects based on deep learning according to claim 7, characterized in that, Execute the corresponding image acquisition control and model scheduling, including: When the current prediction task is a routine detection task, the image acquisition device is controlled to use a standard frame rate and standard exposure time determined based on the calibration results, and the model branch corresponding to the forming stage state is called to perform prediction; When the current prediction task is a key detection task, the current sampling frame rate and current exposure time are determined according to the defect risk level, so as to control the image acquisition device to switch to a high frame rate greater than the standard frame rate and a short exposure acquisition time less than the standard exposure time; Based on the defect risk level and / or the pre-trigger value, determine the processing priority score for the current task; When the processing priority score is greater than the set priority threshold, the current prediction task is placed in the priority processing queue, and the image in the key detection area is cropped and re-examined; when the processing priority score is not greater than the set priority threshold, the current prediction task is placed in the regular queue for detection.

9. The online prediction method for valve casting forming defects based on deep learning according to claim 2, characterized in that, Based on the preliminary prediction results, combined with the image quality status and equipment operating status, the corrected risk value or result reliability is determined, and an online prediction result is output, including the defect risk type, result reliability, and equipment status identifier generated based on the equipment operating status, including: Obtain the defect risk value of a single frame image within a set number of frames before and after the current time; Based on the single-frame image defect risk value, determine the consistency risk value for multiple frames; The final defect risk value is determined based on the image quality status, the device operating status, and the data synchronization indicators. When the final defect risk value is greater than the risk trigger threshold determined based on the process specification, and the final defect risk values ​​are consistent for at least a preset number of consecutive frames, the defect risk type is determined. Based on the final defect risk value, the image quality status, and the device operating status, an online prediction result is generated and output, which includes the defect risk type, the result reliability, and the device status identifier.

10. The online prediction method for valve casting forming defects based on deep learning according to claim 5, characterized in that, Also includes: When the device is running in a degraded state, the current image quality score is obtained and compared with a preset quality threshold. If the image quality score is lower than the preset quality threshold, the abnormal image data is blocked from entering the deep learning defect prediction model; if the image quality score is greater than or equal to the preset quality threshold, it is allowed to enter the deep learning defect prediction model. Call upon backup image acquisition equipment or backup acquisition source, and mark the current batch of image data as abnormal restricted data; When outputting online prediction results, an abnormal event code is attached, and the abnormal restricted data is marked and stored in the abnormal data area.

Images