A real-time video recognition method for surface crack defects of complex components

Abstract

The application discloses a kind of complex component surface crack defect video real-time identification method, it is related to image detection technical field, including as follows: obtain the image frame and acquisition quality metadata of multi-camera synchronous acquisition, and based on reference geometry establishes the mapping relationship of image frame and surface unit;According to process risk chart, reconnaissance uncertainty and budget constraint, select high-resolution image block;Crack candidate reasoning is executed to high-resolution image block, the result is mapped as surface unit evidence and carries out quality gate and state update;Crack object is formed by aggregation confirmation state surface unit, and output determination result and evidence package, and uncertain state surface unit is fed back as the priority object of subsequent acquisition group.The application can improve crack positioning consistency under complex curved surface condition, reduce bad frame interference, consider detection tempo and review traceability, and improve multi-view evidence merging stability, inhibit repeated alarm and cold area detection risk.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, specifically to a method for real-time video recognition of surface cracks and defects in complex components. Background Technology

[0002] For die-cast aluminum shells, stamped parts, welded parts, blades, and other complex components, on the actual production line, a robot with a handheld camera will scan the workpiece surface along the line or multiple cameras will simultaneously capture images from various angles. Then, edge computing will be used to complete defect identification and result output. The routine process is to acquire images, zoom or divide them into blocks, extract cracks, judge defects, and conduct manual review. The purpose is to complete the online screening of small surface cracks within a specified cycle time.

[0003] US Patent document US12039441B2 proposes a crack detection method and system based on fully convolutional networks. This document includes a video camera and a scanning mechanism. The scanning mechanism moves the camera along the surface to be inspected, capturing continuous video frames. There are overlapping areas between consecutive frames, and the same crack may appear repeatedly within consecutive frames. A processor-based fully convolutional network analyzes at least a portion of the video frames to obtain a crack score map. Then, a parameterized data fusion scheme is used to merge the scores of each frame in a spatiotemporal coordinate system to obtain the crack identification result. The document also mentions that this method can be used for robot detection, and for full HD and higher resolution video.

[0004] On the other hand, foreign patent document WO2024226041A1 proposes an "improved method for detecting surface features of manufactured parts on a production line". This method is for visual inspection of manufactured parts on a production line. It uses one or more cameras to acquire workpiece images and uses synthetic image training, image block extraction, multi-view inference and result fusion to complete surface feature detection. The specification also states that industrial inspection images have high resolution and model training and inference can be completed by image block extraction in different modes. US patent document US12347038B2 proposes a method for crack assessment and visualization by combining two-dimensional crack recognition results with three-dimensional mesh models.

[0005] The aforementioned technologies can detect surface cracks or surface features in applicable scenarios, but they still have limitations in high-frequency online detection scenarios for complex curved metal components. Since the fusion object of US12039441B2 is based on continuous video frames and spatiotemporal score maps, when the object to be detected is a complex curved component with varying curvature, occlusion, or strong reflection, the projection position, apparent length, and contrast of the same crack under different viewpoints or scanning positions will change over time, resulting in unstable cross-frame evidence. For the high-resolution detection route of WO2024226041A1 based on image blocks, if the original image is scaled up to adapt to the model input, sub-millimeter-level cracks will be weakened due to insufficient effective pixels. Furthermore, performing inference on a large number of fixed image blocks for large images will increase time costs and computational power consumption.

[0006] Meanwhile, multi-camera synchronous acquisition is also affected by network transmission, trigger timing, and image quality fluctuations. Basler documentation indicates that PTP synchronization accuracy depends on network hardware and configuration; frame transmission delay and inter-packet delay settings affect transmission congestion and frame rate; and (improper settings) can lead to events such as late actions, frame trigger loss, and excessive frame start triggering during acquisition. The combination of these factors can result in missed detections, false detections, duplicate alarms, increased manual review burden, and limited production line cycle time. Therefore, in multi-camera high-resolution online inspection scenarios of complex curved metal components, improving the stability and reliability of fine crack identification while ensuring inspection cycle time has become a pressing technical problem. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides a real-time video identification method for surface crack defects in complex components. The method selects high-resolution image blocks based on process risk diagrams, detection uncertainties, and budget constraints. Crack candidate inference is performed on the high-resolution image blocks, and the results are mapped to surface unit evidence, with quality gating and state updates applied. Confirmed surface units are aggregated to form crack objects, and the judgment result and evidence package are output. Uncertain surface units are fed back as priority objects for subsequent acquisition groups. This method improves crack localization consistency under complex curved surface conditions, reduces bad frame interference while balancing detection rhythm and verification traceability, and enhances the stability of multi-view evidence merging. It solves the technical problems described in the background art.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A real-time video recognition method for surface crack defects of complex components includes acquiring multi-view and / or multi-time image frames of the workpiece under test, including discretizing the surface of the workpiece under test into multiple surface units based on reference geometry, establishing the projection correspondence between each image frame and the surface unit, and acquiring acquisition quality metadata corresponding to each image frame.

[0012] Low-resolution reconnaissance is performed on each image frame, and the corresponding high-resolution image block is selected based on the risk prior of the surface unit and the reconnaissance uncertainty under the constraints of high-resolution pixel budget and time delay budget.

[0013] Crack candidate inference is performed on high-resolution image blocks, the obtained candidate crack evidence is mapped to the corresponding surface unit, and gating fusion and state update are performed based on the acquired quality metadata.

[0014] Cracked objects are formed based on adjacent surface units in the confirmed state, and judgment results and evidence packages are output. Surface units in the uncertain state are fed back as priority objects for subsequent acquisition groups.

[0015] Furthermore, the acquired quality metadata includes timestamps, late action markers, frame trigger loss markers, frame start over-trigger markers, link retransmission counts, ambiguity, saturation, and registration residuals. The edge computing device marks each image frame as low-confidence input based on the acquired quality metadata, and sends it to the downweighted path when the overall quality is below the low-confidence threshold, and to the supplementary acquisition candidate queue when it is below the abnormal threshold.

[0016] Furthermore, the reference geometry is discretized according to the local curvature and process risk map to form a surface unit map with surface index, local normal and adjacency relationship; the edge computing device performs global registration and local registration on each image frame to generate the corresponding set of visible surface units and the projection correspondence between the image frame and the surface unit.

[0017] Furthermore, the edge computing device first generates a low-resolution reconnaissance map for each image frame, then maps the coarse crack response and reconnaissance uncertainty to surface units to form surface unit-level reconnaissance results; and based on the continuous suspicious responses of adjacent surface units, it merges spatially adjacent and projection-compatible surface units into surface unit clusters to generate corresponding high-resolution image blocks.

[0018] Furthermore, the selection of high-resolution image blocks is constrained by the high-resolution pixel budget, the maximum number of image blocks, and the remaining latency budget. After priority selection is completed, the edge computing device also supplements high-resolution image blocks from low-risk surface units according to the minimum exploration ratio, so that the high-resolution image block request set covers the locations to be verified in both high-risk and low-risk areas.

[0019] Furthermore, after performing crack candidate inference on the high-resolution image block, the edge computing device extracts the crack mask, skeleton, local width, and local confidence. Based on the projection correspondence between the image frame and the surface unit, the crack mask, skeleton, and local width are mapped back to the corresponding surface unit to form surface unit evidence bound to the source image frame.

[0020] Furthermore, the edge computing device performs a three-stage quality gating on each surface unit based on the collected quality metadata and surface unit evidence. Surface unit evidence with quality higher than the second threshold enters the direct fusion path, surface unit evidence with quality between the first and second thresholds enters the low-weighted slow fusion path, and surface unit evidence with quality lower than the first threshold enters the supplementary sampling determination path. When the remaining latency budget is insufficient, it switches to the uncertain path.

[0021] Furthermore, the edge computing device maintains candidate state, confirmed state, uncertain state, and suppressed state for each surface unit. The surface unit cluster is updated to the confirmed state only when the same surface unit cluster simultaneously meets the conditions of independent support quantity, adjacency continuity, and width smoothness. For surface units supported only by a single low-quality view, the candidate state is maintained, and the surface unit is switched to the uncertain state when the remaining latency budget is insufficient.

[0022] Furthermore, the edge computing device aggregates surface units that are in a confirmed state and are topologically continuous into crack objects, and calculates the length, average width, maximum width and number of branches based on the skeleton length and local width of the crack objects. At the same time, it generates an evidence package, which includes the supporting image frame index, the corresponding surface unit position, the crack local image block, the quality metadata trajectory and the quantization result.

[0023] Furthermore, if a surface unit cluster is repeatedly in an uncertain state in the current acquisition group, the edge computing device will feed back the surface unit cluster as a high-resolution budget priority object in the subsequent acquisition group; if a certain camera continuously generates abnormal events such as delayed action, frame trigger loss, or over-triggering, the evidence weight of that camera will be reduced, and adjacent viewpoints will be prioritized for compensation acquisition of the corresponding area in the subsequent acquisition group.

[0024] (III) Beneficial Effects

[0025] This invention provides a real-time video recognition method for surface crack defects in complex components, which has the following advantages:

[0026] After discretizing the reference geometry into surface units, a projection correspondence between image frames and surface units is established, so that subsequent evidence can be focused on the same surface position, avoiding cross-view mismatch, duplicate counting, and position drift caused by obtaining image plane coordinates alone under complex curved surface conditions. At the same time, timestamps, delayed actions, frame trigger loss, over-triggering, link retransmission, ambiguity, saturation, and registration residuals are bound to form acquisition quality metadata. In step three, a pre-gating method is used to distinguish low-confidence inputs from normal inputs using acquisition quality metadata.

[0027] Low-resolution reconnaissance is then used to select high-resolution image blocks based on the process risk map, reconnaissance uncertainty, visible set of surface units, and remaining time delay budget. High-resolution processing is then applied to more worthwhile locations, leaving low-risk areas for backup exploration and avoiding missed detections in cold zones. The high-resolution inference results are then reflected back into surface unit evidence. Through a three-stage quality gating and state evolution involving candidate, confirmed, uncertain, and suppressed states, a single anomalous response will not deepen the final judgment; evidence from different angles and at different times will gradually converge according to credibility.

[0028] The topologically continuous confirmed state surface units are grouped into crack objects, and an evidence package is generated that includes supporting image frame indexes, corresponding surface unit locations, crack local image patches, quality metadata trajectories, and quantization results. The output not only provides conclusions but also complete verification evidence. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the overall architecture of the complex curved surface metal part crack detection system of the present invention;

[0030] Figure 2 This is a schematic diagram of the construction of the reference geometric surface unit map and the multi-view image frame mapping of the present invention;

[0031] Figure 3 This is a schematic diagram of the image frame data packet generation and low-confidence input splitting process of the present invention;

[0032] Figure 4 This is a schematic diagram of the process for constructing candidate attention sequences and generating high-resolution image block request sets in this invention.

[0033] Figure 5 This is a schematic diagram of the high-resolution image block crack candidate inference and surface unit evidence reflection of the present invention;

[0034] Figure 6 This is a schematic diagram of the surface unit gated current splitting and state machine evolution of the present invention;

[0035] Figure 7 This is a schematic diagram of the crack object aggregation and evidence package assembly of the present invention;

[0036] Figure 8This is a schematic diagram of the uncertain region feedback, perspective correction, and final determination process of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Please see Figures 1-8 This invention provides a real-time video recognition method for surface crack defects in complex components, comprising:

[0039] Step 1: Convert the multi-view image frames formed by the controlled acquisition of the workpiece under test in the inspection station into a set of image frame data packets with surface unit addressing relationships and acquisition quality metadata, so that subsequent steps can be carried out around the same surface position rather than around independent image planes.

[0040] For workpieces such as die-cast aluminum shells and stamped parts, the workpiece surface usually has rounded corners, ribs, hole edges, flanges, and localized reflective areas. If the crack coordinates of a simple image plane are still used to represent the crack, the same crack will often show positional shifts, length changes, and brightness reversals under different viewing angles. Even if multiple image frames are obtained in subsequent steps, it is difficult to confirm whether these fine line-like marks belong to the same surface location.

[0041] Therefore, in step one, the reference geometry of the workpiece under test is reconstructed into an addressable surface element map, and then each image frame is attached to the map, thereby restoring the same crack to repeated evidence on the same surface element.

[0042] In this embodiment, the following actions are performed by the edge computing device. The controller is responsible for sending the same acquisition group trigger command to the multi-camera acquisition unit and reading the camera status. The edge computing device pre-stores the reference geometry, process risk map, and camera calibration parameters of the workpiece to be tested. The reference geometry preferably uses a three-dimensional model of the workpiece; if there is a systematic deviation between the three-dimensional model and the actual formed surface, a reference surface obtained by scanning a gold sample is used instead. Both are referred to as the reference geometry in this step. The process risk map uses the reference geometry as a carrier to project the risk level of areas such as hole edges, rounded corner transitions, rib roots, and edge folds onto the reference geometry. The camera calibration parameters include the intrinsic parameters, extrinsic parameters, and distortion parameters of each camera.

[0043] The edge computing device first discretizes the reference geometric surface into multiple surface units. Each surface unit corresponds to a fixed surface index, local normal direction, and set of adjacent surface units. This discretization is not an average division, but rather the surface unit size varies with local curvature and process risk map: the surface units are set to be finer in locations with rapid curvature changes and high risk levels, and coarser in locations with slow curvature changes and low risk levels.

[0044] Cracks occur at rounded corners, rib roots, and hole edges, and the typical crack orientation is also greatly affected by geometry. If the surface elements are too coarse, the remaining evidence will overlap at multiple physical locations; if the discretization is too fine on a large, flat surface, it will increase the cost of mapping maintenance.

[0045] The preferred side length of the surface unit is given by the following formula:

[0046] ;

[0047] In the formula, the side length of the surface unit is... : No. The target side length of each surface element in the reference geometry, with values ​​located at the minimum side length. with the longest side length Between; reference side length : Initial discrete scale of the neutral curvature region; curvature amplitude : No. The absolute value of the local curvature at the center of each surface unit, local curvature Risk gradient can be obtained by fitting the neighborhood of a triangular mesh; The process risk diagram is in the first place. The gradient magnitude near each surface unit can be obtained by mesh difference from the process risk map;

[0048] Curvature weight and risk weight Used to balance the contributions of geometric changes and risk changes in the discrete representation, both of which are positive; truncation operator Used to limit the calculation results to Within the range.

[0049] Taking a die-cast aluminum shell as an example, after the edge computing device reads the reference geometry of the shell, it first identifies the edges of mounting holes, the roots of studs, and the turning points of reinforcing ribs. Then, according to the aforementioned edge length rules, these locations are divided into denser surface units; while in the large planar areas of the shell, only sparser surface units are retained. After processing, a surface unit map continuously covering the entire outer surface of the workpiece is obtained. The map saves the adjacency relationship and local normal of each surface unit, providing a structural basis for subsequent determination of whether cracks continue to extend along adjacent surface units.

[0050] After the surface unit map is established, step one continues processing image frames from different viewpoints within the same acquisition group. Multiple camera acquisition units form the same acquisition group under controller triggering. Each image frame, upon entering the edge computing device, undergoes initial projection based on camera calibration parameters, and then global-local registration aligns the image frame with the surface unit map. Global registration provides the approximate pose of the workpiece relative to the image frame, while local registration corrects minor deviations caused by workpiece placement errors, robot positioning errors, or glare interference.

[0051] In the preferred implementation, the edge computing device first obtains the initial pose by matching the projection contour of the reference geometry with the contour of the workpiece boundary in the image frame. Then, within a local window, it corrects the pose using edge direction consistency and grayscale similarity after reflection suppression. After registration, the system generates a set of visible surface units for each image frame, retaining only those surface units that have a projected area at the current viewpoint, are not self-occluded by the reference geometry, and do not fall into the distortion shielding zone. To avoid directly sending severely mismatched image frames into subsequent steps, the edge computing device also calculates the registration reliability of the image frames.

[0052] ;

[0053] In the formula, the registration confidence level is... Image frame The overall credibility level, with a value range of [value missing]. residual Image frame Reprojection residuals; fuzziness Image frame Ambiguity estimate; consistency measure Image frame The local edge consistency coefficient, with a value range of Residual attenuation coefficient and fuzzy attenuation coefficient It is a positive value. Reprojection residual Ambiguity and local consistency Both can be obtained through the aforementioned registration chain and image quality evaluation chain.

[0054] In another embodiment, if the surface of the workpiece under test has stable inkjet reference points or etching reference lines, local registration can also be superimposed with reference point matching; if the surface texture of the workpiece under test is extremely weak, local registration is changed to contour-normal projection alignment, maintaining the reliability of terminological registration. Each image frame obtains a one-to-one correspondence with the surface unit map and outputs the corresponding set of visible surface units.

[0055] In use, the surface unit map of the reference geometry of the workpiece under test is used. The image frame is mapped to the visible window on the surface unit map. Then, the reconnaissance and crack evidence can be combined with the fixed surface location organization. The registration confidence and the set of visible surface units can provide evidence for low confidence input judgment.

[0056] After surface unit addressing is completed, step one cannot end immediately because although the surface location has been found in the image frames within the same acquisition group, the reliability of these image frames themselves is still not explicitly stated. When multiple camera acquisition units are working simultaneously, delayed actions, frame trigger loss, frame start over-triggering, link retransmission, and local overexposure do not directly change the physical state of the workpiece surface, but they do change whether the image frames can serve as the basis for subsequent crack evidence. If these states are not bound in step one, subsequent steps can only treat them as post-event logs, making it impossible to distinguish whether this thin line structure is real crack evidence or simply appears in an abnormal image frame. Therefore, it is necessary to bind acquisition quality metadata to each image frame and pre-decentralize low-confidence image frames.

[0057] In this embodiment, the image frame data packet is not a simple image matrix, but a composite object composed of the image frame body, acquisition group index, camera index, trigger sequence number, visible surface unit set, and acquisition quality metadata. The acquisition quality metadata includes both event information from the multi-camera acquisition unit or the streaming link, and the image quality status calculated by the edge computing device in the image domain.

[0058] Edge computing devices synchronously construct acquisition quality metadata for each image frame. The acquisition quality metadata preferably includes timestamps, late action flags, frame trigger loss flags, frame start over-trigger flags, link retransmission counts, blurriness, saturation, and registration residuals. Timestamps describe the temporal position of the image frame relative to the same acquisition group; late action flags describe the execution delay state after the trigger command arrives at the camera; frame trigger loss flags and frame start over-trigger flags describe whether the camera received a trigger that was not executed or was discarded; link retransmission counts describe whether data packet retransmission occurred during image frame transmission; blurriness describes motion blur or defocus; and saturation describes whether local highlights obscure surface texture.

[0059] To enable subsequent steps to use this metadata on a uniform scale, this implementation further compresses it into a comprehensive quality score:

[0060] ;

[0061] In the formula, the comprehensive quality score Image frame Overall credibility; time consistency Image frame The degree of time consistency with the reference time of the same acquisition group, with a value range of [value range missing]. Late action marker Frame Trigger Loss Mark and frame start over trigger flag All are discrete state quantities, where 0 is taken in normal states and 1 in abnormal states; retransmission quantity : The result of normalized link retransmission count; saturation quantity Local saturation level; registration reliability Using the previous definition; time weight Action weights Triggering weight loss Over-trigger weight Retransmission weights saturation weight Registration weights All are non-negative and the sum is retransmission attenuation coefficient and saturation decay coefficient All are positive values.

[0062] In the field implementation, when a stamped part enters the inspection station, the controller simultaneously sends the same acquisition group trigger command to four cameras. The image frame from the first camera has clear edges and low registration residual; the second camera shows local reflections but triggers normally; the third camera triggers late; and the fourth camera experiences retransmission during transmission. Step one does not simply treat these four image frames as equal input, but instead writes different acquisition quality metadata and different comprehensive quality scores for each of the four image frames. In this way, even if all four image frames see the same surface unit, subsequent steps know that the reliability of each image frame is different.

[0063] After obtaining the overall quality score, step one continues with low-confidence input stream splitting. Instead of deleting all abnormal images, a splitting approach is adopted: normal images are retained, low-confidence images are downweighted, and severely abnormal images are converted to supplementary acquisition candidates. This is because certain key locations on complex curved surfaces may only appear in a limited number of viewpoints. Directly deleting abnormal image frames could completely destroy evidence for those locations in subsequent steps. In the preferred implementation, the edge computing device presets low-confidence and severely abnormal thresholds. When the overall quality score is below the low-confidence threshold but above the severely abnormal threshold, the image frame is marked as a low-confidence image frame and retained in the image frame data packet set, only entering the downweighting path in subsequent steps. When the overall quality score is below the severely abnormal threshold, or when a preset combination of key abnormalities appears in the late action flag, frame trigger loss flag, or frame start over-trigger flag, the image frame is sent to the supplementary acquisition candidate queue. If multiple key viewpoints in the same acquisition group are all marked as low-confidence image frames, the acquisition group is marked as a disturbed acquisition group.

[0064] After the data split, step one yields a ternary intermediate result that can be directly used downstream. This ternary intermediate result consists of an image frame data packet set, a surface unit visibility set, and a collection of acquisition quality metadata corresponding to each image frame. The image frame data packet set contains the original image frame itself and the triggered identity; the surface unit visibility set indicates which surfaces each image frame covers; and the acquisition quality metadata set determines whether it can be used to split crack evidence. These three data sets are linked by the same image frame index and acquisition group index.

[0065] In the parallel implementation, if the detection line consists of two high-resolution cameras and one supplementary lighting camera, then the supplementary lighting camera will participate in the subsequent crack candidate inference. However, if the detection line uses a single camera for multiple stop-and-acquire data acquisition, the acquired quality metadata fields will not be changed; only the camera index will be changed to the stop-and-acquire index. For crack candidate inference, the input is a high-resolution image patch; the first sub-network or first processing chain outputs a crack mask probability map; the second sub-network or post-processing chain refines the crack mask and extracts the skeleton; the width is obtained by searching the mask boundary according to the skeleton normal; the local confidence is composed of the crack mask probability and the skeleton continuity.

[0066] Candidates: have at least one source of evidence and Confirmed: Number of independent supports Adjacent continuous quantity Width deviation Uncertainty: Evidence is established but lacks independent support, or the replenishment budget is exhausted; Inhibition: Several consecutive times can only be supported by low-confidence sources, or the pseudo-crack score exceeds the threshold.

[0067] In practice, each acquisition group outputs not a string of unfiltered image files, but a set of ternary intermediate results. Subsequent steps reading these intermediate results can determine not only where each image frame was viewed, but also whether that frame is worth prioritizing. The acquisition process status of image frames is shifted forward to a computable object, allowing subsequent steps to differentiate between normal, low-confidence, and supplementary acquisition candidates based on overall quality scores; even in disturbed acquisition groups, key surface locations will not lose all evidence due to simple frame deletion.

[0068] Step 2: Without disrupting the beat boundary of this acquisition group, further transform the image frame data packet set, surface unit visible set, and acquisition quality metadata set obtained in Step 1 into a high-resolution image block request set, so that subsequent steps can carry out crack candidate inference around the surface location to be judged.

[0069] Step one has identified which surface units are visible in each image frame and whether these frames are reliable, but it still doesn't know which surface units in the current acquisition group are more worthy of priority magnification. If high-resolution image blocks are extracted solely based on the process risk map, high-risk areas without anomalies will continue to consume the budget; if the entire image frame is uniformly divided into blocks, flat and low-risk areas will also consume a large number of high-resolution pixels. Therefore, step two first filters out surface units in the current acquisition group that show signs, raise suspicions, and are worth closer examination from the entire visible surface, and then establishes a candidate attention order around these surface units.

[0070] Edge computing devices read the image frame body, image frame index, and overall quality score from the image frame data packet set. Simultaneously, the surface units covered by the current image frame are read from the visible set of surface units. The process risk map provides risk priors for each surface unit, and the surface unit map provides adjacency relationships for each surface unit. Subsequently, the edge computing device first performs reconnaissance at a low-resolution level, and then aggregates the reconnaissance results to the surface units.

[0071] Edge computing devices generate low-resolution reconnaissance maps for each image frame. These low-resolution reconnaissance maps are not simple thumbnails; rather, they are created by downsampling the original image frame proportionally while preserving local linear texture direction information, and then performing reflection suppression, edge preservation, and brightness compression. Complex curved metal surfaces are prone to bright spots and brushed textures. If the original image frame is directly scaled down, some fine cracks and reflective stripes will be compressed into noise. The long side of the low-resolution reconnaissance map is set to one-quarter to one-eighth of the long side of the original image frame; if it is scaled down further, narrow textures at hole edges and rib roots will be merged into the background.

[0072] After the low-resolution reconnaissance map is generated, the edge computing device assigns a reconnaissance response value to each pixel. Then, based on the image frame-surface unit mapping relationship from step one, the pixel responses falling within the same surface unit projection area are aggregated into a surface unit-level reconnaissance response. Instead of direct averaging, projection coverage weights are used to distinguish between pixels completely within the surface unit and pixels only pressed onto the edge, reducing color mixing effects near the surface unit boundaries. The surface unit-level reconnaissance response is calculated using the following formula:

[0073] ;

[0074] In the formula, the surface unit reconnaissance response Surface unit In image frame The reconnaissance response intensity below describes the suspiciousness of the surface unit from the current viewpoint, and its value range is [value range missing]. Projection area Surface unit In image frame The set of projected pixels in the image; coverage weight : pixel For surface unit The coverage contribution, which suppresses projection boundary aliasing, has a value range of [value range missing]. Coverage weight The coverage ratio of the surface unit projection area from the pixel;

[0075] Pixel reconnaissance response Image frame Medium pixel The low-resolution reconnaissance response, with a value range of [value range missing]. First, the low-resolution reconnaissance image is filtered for dark lines in eight directions to obtain eight directional responses. The largest directional response is taken as the dark line saliency. Then, combined with the local bright spot suppression map, the pixel reconnaissance response is obtained. The lightweight reconnaissance network outputs a pixel-level crack probability map, which serves as the pixel reconnaissance response. Another source.

[0076] Taking a die-cast aluminum shell as an example, the edge computing device first sees several fine line-like responses in the low-resolution reconnaissance map, and then uses the surface unit projection area to distribute these responses to specific rib root surface units and hole edge surface units.

[0077] The surface unit reconnaissance response alone is insufficient to determine where high-resolution image patches should be assigned, because some surface units have low response values ​​but are in a difficult-to-determine state; for example, some pixels may appear as cracks, while others may resemble reflective edges. Therefore, step two further constructs reconnaissance uncertainty for each surface unit to distinguish between those without abnormal signs and those that are not clearly visible from the current viewpoint.

[0078] In this embodiment, the edge computing device converges the deviation of pixel detection responses within the projection area of ​​a surface unit into detection uncertainty. If most pixel detection responses within a surface unit are significantly biased towards a dark background or significantly biased towards linear textures, the detection uncertainty is low; if conflicting responses exist simultaneously within the same surface unit, the detection uncertainty is high. The detection uncertainty is calculated using the following formula:

[0079] ;

[0080] In the formula, the surface element detection uncertainty is... Surface unit In image frame The reconnaissance uncertainty level below measures whether the surface unit needs further magnification for observation, and its value range is [value range missing]. Projection area Coverage weight Pixel reconnaissance response The meaning remains consistent with the previous expression; transformation terms Used to transfer pixel reconnaissance response from Mapped to This causes the uncertainty of the neutral response to increase during polymerization.

[0081] After the reconnaissance uncertainty is generated, the edge computing device will analyze the surface unit reconnaissance response. Surface element detection uncertainty Overall quality score The prior risks in the process risk map are written into the candidate attention order. Subsequent steps, when arranging high-resolution image blocks, no longer require blindly searching back to the image plane; instead, they can directly focus on which surface units are worth examining in detail.

[0082] After constructing the candidate attention sequence, the number of candidate surface units is usually still greater than the number of surface units allowed to enter high-resolution processing per piece. If all candidate surface units are sent to high-resolution processing, step two will degenerate into a repetitive scanning route, making it impossible to control the pixel overhead and remaining latency budget per piece. Therefore, step two also needs to compress the candidate attention sequence into a high-resolution image block request set, while retaining the ability to explore cold areas during the compression process, so that newly emerging suspicious locations outside the process risk map still have the opportunity to enter high-resolution processing.

[0083] In this embodiment, the edge computing device calculates a priority value based on the risk prior, reconnaissance uncertainty, comprehensive quality score, and adjacency relationship of the candidate attention order, along with the single-piece remaining delay budget, high-resolution pixel budget, and maximum number of image blocks. It then merges spatially adjacent, visually continuous, and projection window compatible surface units to generate a high-resolution image block request set around the surface unit cluster.

[0084] The construction of high-resolution priority values ​​simultaneously considers the risk priors given by the process risk map, the given reconnaissance uncertainty, the comprehensive quality score given in step one, the degree of continuity of suspicion between adjacent surface units, and the pixel and latency costs required for high-resolution cropping. To avoid any single factor completely dominating the ranking, this implementation adopts a combination of multiplicative enhancement and cost suppression:

[0085] ;

[0086] Corrected high-resolution priority value : Represents surface unit In image frame The priority of the selected high-resolution image patch request set is the direct basis for sorting and scheduling in step two; surface unit index : Represents a surface location unit obtained after discretizing the reference geometry; image frame index : Represents a specific image frame within the current acquisition group, used to distinguish image inputs from different viewpoints or at different times; Risk Prior : Indicates that the process risk diagram is assigned to the surface unit The risk level describes the degree to which a surface location is more prone to cracking during processing; a higher value indicates that the location deserves priority inspection; surface unit detection significance. : Represents surface unit In image frame The degree of suspicion shown in the low-resolution reconnaissance results;

[0087] Surface element detection uncertainty : Represents surface unit In image frame Is the image unclear or uncertain? (Image frame overall quality score) : Represents an image frame The reliability of a data entry point typically takes into account factors such as time synchronization status, delayed action, frame trigger loss, over-triggering, link retransmission, ambiguity, saturation, and registration residual.

[0088] Adjacent continuum : Represents surface unit Whether there have been consecutive suspicious responses between the surface elements and its adjacent surface elements; surface elements Among adjacent surface units, the detection significance is higher than the threshold and the direction is... The proportion of those in the same main direction;

[0089] Resource costs : Indicates surrounding surface unit From image frames The total cost required to obtain high-resolution image patches typically consists of the estimated pixel cost and the estimated latency cost; a weighted sum of normalized pixel count and normalized inference latency; the pixel count is the area of ​​the candidate patch; the inference latency is obtained by offline profiling, for example, by building a linear or piecewise linear model based on patch size, overlap rate, and batch size; a stable bias term. It is a small positive value to prevent the entire molecule from collapsing to 0 due to a factor approaching zero, and also to prevent sorting disorder caused by extremely small values;

[0090] Risk Prior Index : Indicates prior risk Intensity of influence on priority values; Detection significance index : Indicates the saliency of surface unit detection The strength of the influence of the priority value; the larger the value, the more likely the location in the current frame that already shows obvious signs of cracks will be selected first; reconnaissance uncertainty index : Represents the surface element detection uncertainty The strength of the influence of priority values; the larger the value, the more likely those temporarily unclear but worthwhile areas are to receive high-resolution processing; image quality index : Represents the overall quality score of an image frame. The strength of the influence of the priority value; the larger the value, the more likely the candidate position is to be ranked higher in high-quality image frames; adjacency continuity index : Indicates adjacent contiguous quantities The strength of the influence on the priority value; the larger the value, the easier it is for linear suspicious regions with good continuity to be preserved as a whole; cost suppression coefficient : Indicates the cost of resources The strength of suppression of priority values; the larger the value, the more the system tends to suppress high-cost image blocks and prioritize locations with lower processing costs but higher information value.

[0091] After priority values ​​are generated, the edge computing device scans candidate attention orders from high to low priority. Simultaneously, it merges adjacent surface units projected onto the image frame, with the distance between their projection frames not exceeding a preset boundary, into the same surface unit cluster. The outer window of the surface unit cluster is then expanded according to a fixed context boundary to obtain candidate high-resolution image patches. The expansion boundary is one-fifth to one-third of the long side of the outer window; if the expansion boundary is too small, cracks cutting off from the edge of the image patch will affect subsequent skeleton extraction; if the expansion boundary is too large, it will increase resource costs. .

[0092] Taking a stamped part as an example, two adjacent surface units at the root of the flange both have high detection uncertainty in the same image frame, while another surface unit next to the hole edge also has a high detection response. The edge computing device does not independently slice all three into three image blocks. Instead, it merges the two adjacent surface units at the flange root into one surface unit cluster, and uses the surface unit at the hole edge as another surface unit cluster, then generates two candidate high-resolution image blocks respectively. In this way, subsequent steps see the continuous texture at the flange root without mixing in the hole edge region.

[0093] After priority sorting and surface unit cluster compression are completed, step two involves budget pruning. Budget pruning includes the high-resolution pixel budget, the maximum number of image blocks, and the remaining latency budget. The edge computing device attempts to add candidate high-resolution image blocks sequentially according to priority. As long as the three boundaries are still met after addition, the image block enters the high-resolution image block request set; once any boundary is triggered, subsequent candidate high-resolution image blocks are no longer added.

[0094] However, simply suppressing all high-resolution image patches into high-risk areas through the aforementioned budget clipping prevents fine cracks suddenly appearing in low-risk areas from being amplified. Therefore, step two establishes a minimum exploration ratio after budget clipping. Low-risk surface units not selected by the current priority value are extracted from a small number of surface units to generate supplementary high-resolution image patches. The minimum exploration ratio increases with the degree of disturbance to the current acquisition group and the average reconnaissance uncertainty. The minimum exploration ratio is obtained according to the following formula:

[0095] ;

[0096] In the formula, the minimum exploration ratio is... Current data collection group The low-risk supplementary extraction ratio serves to ensure that cold areas still retain the opportunity to enter the high-resolution image patch request set; the base ratio : Initial guaranteed exploration ratio of the system under stable operating conditions; Disturbance flag Current data collection group Whether it is marked as a disturbed acquisition group in step one, with a value of 0 under normal conditions and 1 under disturbance conditions; average reconnaissance uncertainty Current data collection group The average reconnaissance uncertainty across all visible surface units, with values ​​ranging from [value range missing]. ;

[0097] Disturbance gain and uncertainty gain Non-negative coefficients; minimum proportion and maximum proportion Used to limit the guaranteed exploration ratio within a preset boundary; truncation operator Restrict the calculation result within the parentheses to a specific interval. The purpose of this is to prevent a low baseline exploration ratio from affecting the coverage of cold areas or from excessively crowding out high-resolution image blocks generated by normal sorting.

[0098] In a preferred embodiment, the edge computing device first generates a first-round high-resolution image patch request set, then supplements it with a second-round high-resolution image patch request set from low-risk surface units according to the minimum exploration ratio. Finally, the results of the two rounds are merged into a single high-resolution image patch request set and output to step three. If the current acquisition group is marked as an disturbed acquisition group, or the overall quality score of multiple key perspectives decreases, the minimum exploration ratio is increased.

[0099] In the parallel implementation, when the detection line uses a track camera instead of a robot to collect data, the logic for generating surface unit clusters is the same, except that the remaining delay budget is calculated according to the transmission cycle window. When the detection line uses a single camera for multiple exposures, the supplementary high-resolution image block request set is supplemented by subsequent exposure frames.

[0100] In practice, step two compresses the candidate attention order into a set of high-resolution image block requests coupled with the budget boundary, so that step three can directly perform crack candidate inference on the high-resolution image blocks, and still know which surface unit cluster each high-resolution image block comes from, how it was selected, and whether it belongs to the minimum exploration path.

[0101] Step 3: Transform the high-resolution image block request set formed in Step 2 into a surface unit state set with a confidence level and status label, so that subsequent steps can determine the crack object based on the confirmed surface location rather than on isolated image blocks.

[0102] Step two has focused the high-resolution pixel budget on surface unit clusters worthy of close examination. However, these surface unit clusters are currently only areas requiring further interpretation and cannot be directly used as evidence of cracks. This is because the same high-resolution image patch may contain both real cracks and scratches, wear marks, stamping and drawing marks, or reflective edges that resemble crack morphology. If high-resolution image patches are directly accumulated at the image patch level after generation, subsequent steps will revert to an image plane approach, failing to maintain the unified addressing framework of the surface unit map. Therefore, step three first performs crack candidate inference on the high-resolution image patch, and then reflects the candidate results back to surface unit evidence, ensuring that subsequent quality gating and state evolution always revolve around surface units.

[0103] In this embodiment, the following actions are performed by the edge computing device. The edge computing device reads high-resolution image blocks, their corresponding surface unit cluster identifiers, source image frame indexes, and the overall quality scores of the source image frames one by one from the high-resolution image block request set. For each high-resolution image patch, the edge computing device first performs crack candidate inference to obtain crack mask, skeleton fragment, local width and local confidence. Then, based on the image frame-surface unit mapping relationship established in step one, these candidate results are mapped back to the surface unit map.

[0104] Edge computing devices perform two levels of inference on high-resolution image patches. The first level of inference extracts elongated, continuous structures and outputs a crack mask. The second level of inference extracts skeleton fragments along the centerline of the crack mask and estimates local widths, thereby further compressing the linear structures in the high-resolution image patch from surface responses into linear structures. This two-level approach is used because real cracks in high-resolution image patches typically appear as thin, continuous, and slowly varying dark lines, while many reflective edges exhibit abrupt width changes or symmetrical double-edge structures.

[0105] In the preferred implementation, the first-level inference reads the grayscale texture, orientation gradient, and local reflection suppression map of the high-resolution image block to obtain a crack mask. The second-level inference searches for connected principal axes in the crack mask, generates skeleton fragments, and estimates the local width in the skeleton normal direction. After the crack candidate inference is completed, each pixel in the high-resolution image block is classified as background, crack region, or crack-adjacent region, and each skeleton fragment corresponds to a local width and a local confidence value.

[0106] For example, in a high-resolution image patch of a die-cast aluminum shell, the edge computing device first sees a thin dark line extending from the root of a rib and a reflective edge around a hole. After two levels of inference, the skeleton fragment corresponding to the thin dark line is retained, while the skeleton fragment length of the reflective edge around the hole is shortened due to the bimodal brightness distribution in the normal direction. Thus, the next step uses not the original image patch, but the candidate result of structural compression.

[0107] When used, the texture and edges inside the high-resolution image patch are transformed into four types of intermediate quantities: crack mask, skeleton fragment, local width, and local confidence, providing a structural basis for evidence replay.

[0108] After obtaining the candidate results within the high-resolution image patch, step three continues by mapping the results within the image patch back to the surface unit map. Since the high-resolution image patch is cropped around the surface unit cluster, each pixel within the image patch already indirectly possesses the source image frame index and the surface unit projection position in step one. Therefore, the edge computing device reprojects the crack mask and skeleton fragment onto each surface unit in the surface unit cluster according to the source image frame index of the high-resolution image patch, and generates surface unit evidence strength for each surface unit.

[0109] In this embodiment, the evidence strength of the surface unit is calculated using the following formula:

[0110]

[0111] In the formula, the strength of evidence for surface units Surface unit In image frame The strength of candidate evidence obtained from high-resolution image patch replays is , with values ​​ranging from . Skeleton projection area : Falling into surface unit The set of skeleton fragment pixels; the reflection weights Skeleton pixels For surface unit The reflection contribution suppresses aliasing near image patch boundaries and surface unit boundaries, with a value range of [value range missing]. Mask Indication Image frame Medium pixel Whether it belongs to a crack mask, set to 1 if it does, otherwise set to 0; skeleton confidence value Skeleton pixels The local confidence level, with a value range of . , is the product of the high-resolution crack mask probability and the skeleton continuity score; the skeleton continuity score is derived from the directional change rate of adjacent skeleton pixels and the fracture penalty. The skeleton continuity score means that the smaller the directional change of a skeleton pixel and its preceding and following skeleton pixels, the fewer the fractures, and the higher the score.

[0112] After the projection, each surface unit retains the surface unit evidence strength. In addition, the total length of the skeleton, average local width, and source image frame index of the surface unit are also retained. In this way, subsequent quality gating no longer faces image blocks, but rather surface unit evidence with structural properties from a specific image frame, corresponding to a specific surface unit.

[0113] In a generalized implementation, if the detection line uses a single camera with multiple stops for acquisition, the high-resolution image block is still captured around the surface unit cluster, only the source image frame index is replaced with the stop index; if the detection line uses a combination of three area scan cameras and one oblique camera, the reflection logic remains unchanged, only the camera category field is added to the source image frame index.

[0114] In practice, step three re-incorporates the local observations within the high-resolution image patch into the surface unit map, bringing the candidate information back into the unified addressing framework. Crack candidates in the high-resolution image patch no longer exist as image patch coordinates, but are transformed into surface unit evidence corresponding one-to-one with surface units. Therefore, subsequent steps can compare evidence from different image frames at a unified surface location and describe crack candidates using local width, skeleton length, and local confidence.

[0115] Furthermore, the sources of surface unit evidence vary. Some evidence comes from image frames with high overall quality scores, some from image frames with delayed actions or numerous link retransmissions, and some, while possessing high local confidence, only appear briefly in a single viewpoint. If these surface unit pieces of evidence are treated equally, the overall quality scores, high-resolution priority values, and disturbed acquisition group markers established in steps one and two become meaningless. Therefore, step three must consider both whether the evidence resembles a crack and the acquisition conditions under which it originated, and advance each surface unit to one of the following states: candidate state, confirmed state, uncertain state, or suppressed state.

[0116] In this embodiment, the edge computing device first calculates a gating score for each surface unit piece of evidence, and then performs a three-stage quality gating based on the gating score. After gating, the edge computing device updates the surface unit state based on the number of independent supports, adjacency continuity, and width smoothness. The three-stage quality gating and state evolution are not two isolated actions, but a continuous single chain: quality gating determines how surface unit evidence enters the state update, and the state update determines whether re-sampling or suppression is needed.

[0117] Combined surface unit evidence strength And introduce the high-resolution priority value in step two. The gating score is used to form the surface unit evidence. A higher gating score indicates that the evidence originates from a reliable image frame and has a high degree of local structural confidence. In this embodiment, the gating score is calculated using the following formula:

[0118]

[0119] In the formula, the gate control score Surface unit In image frame The credibility of evidence participating in subsequent fusion determines whether it enters strong fusion, low-weight easing fusion, or supplementary fusion; a positive value is assigned. (Overall quality score) Source image frame The overall credibility level, with a value range of [value range missing]. Surface unit evidence strength Surface unit The strength of candidate evidence, with values ​​ranging from 1 to 10. Width deviation Surface unit The average local width deviates from the historical width reference of its adjacent surface elements, and is a non-negative real number; for surface elements... The relative deviation between the average local width and the median width of its adjacent confirmed / candidate surface cells.

[0120] High resolution priority The degree of interest of the surface unit in step two is indicated by a positive value; stable bias. A positive value is used to avoid zero terms under low priority conditions; quality index Evidence Index Width suppression coefficient Priority Index All are positive values; exponential function The monotonically decaying mapping with the natural constant as the base has the effect of reducing the gating score as the width deviation increases.

[0121] After the gating score is generated, the edge computing device compares it with the preset upper and lower thresholds. If the gating score is greater than the upper threshold, the surface unit evidence enters the strong fusion path; if the gating score is between the upper and lower thresholds, the surface unit evidence enters the low-weight slow fusion path and is not confirmed as a candidate; if the gating score is less than the lower threshold, the surface unit evidence does not enter the forward fusion path, but enters the supplementary sampling candidate path or the uncertain path. The surface unit evidence is diverted to the path according to the reliability of the source or the stability of the structure.

[0122] For example, in the flanged area of ​​a stamped part, two image frames generated the same surface unit evidence. One image frame had a higher overall quality score and a higher gating score, so it entered the strong fusion path; the other image frame, although the surface unit evidence strength was not low, had a gating score between the upper and lower thresholds due to delayed action and local overexposure, so it only entered the low-weighted slow fusion path and did not forcibly push the confirmation process higher.

[0123] After completing the three-stage quality gating, step three continues to update the surface unit state machine. The surface unit state machine includes at least candidate states, confirmed states, uncertain states, and suppressed states. A candidate state indicates that suspicious evidence has appeared on the surface unit but has not yet been confirmed; a confirmed state indicates that the surface unit and its neighboring surface units together constitute an acceptable chain of crack evidence; an uncertain state indicates that the evidence is insufficient but still requires attention from the subsequent acquisition team; and a suppressed state indicates that the evidence is close to the reflective edge, an isolated scratch, or occasional noise.

[0124] During state updates, the edge computing device accumulates strong fusion evidence and low-weighted sparse fusion evidence from different image frames for each surface unit and checks three conditions. First, the number of independent supports: whether the surface unit is supported by surface unit evidence from at least two different sources (different sources refer to different image frames or different camera viewpoints). Second, adjacency continuity: whether there is any suspicious evidence in the adjacent surface units that extends continuously along its skeleton direction. Third, width smoothness: whether the average local width of the surface unit and its adjacent surface units changes slowly rather than abruptly. Only when all three conditions are met simultaneously is the surface unit or cluster of surface units promoted to a confirmed state.

[0125] Under boundary conditions, if the evidence for a surface unit comes only from a single low-quality view, it remains in the candidate or uncertain state even if the evidence strength of the surface unit is high. If neither of the adjacent supporting views is available, the edge computing device stops pushing the surface unit to the confirmed state and instead transfers it to the uncertain state and writes it into the subsequent supplementary acquisition candidate table. If the current remaining latency budget for a single unit is insufficient to support supplementary acquisition, the surface unit remains in the uncertain state and is output to step four along with the acquisition group.

[0126] In one embodiment, if the detection line adopts a single-camera multi-stop scheme, the different sources in the number of independent supports are replaced by different image frames or different camera angles with different stop times; if the detection line faces the aluminum-magnesium alloy high-reflectivity housing, in addition to the width smoothness condition, reflective symmetry filtering is added, but the terms surface unit state machine and gating score remain unchanged.

[0127] In practice, the surface unit no longer simply stores a string of scores, but acquires a defined state. Each state carries a different meaning for subsequent actions: a confirmed state can proceed to crack object generation, an uncertain state can proceed to the fourth step of the feedback path, and a suppressed state is excluded from the positive result chain.

[0128] Step 4: Converge the surface unit state set output in Step 3 into a crack object with spatial location, length, width and origin trajectory, and organize the crack object together with the feedback information of the uncertain area into the final judgment result, so that the detection station can provide the next acquisition group with the parameter correction basis while completing the current workpiece processing.

[0129] After step three is completed, the edge computing device has mastered the status and gating score of each surface unit. Surface unit evidence strength Although the average local width and the source image frame index are distributed across multiple surface units, if these distributed surface units are output as the result, the controller cannot eliminate or manually verify these surface units, and the human-machine interface cannot view a complete crack.

[0130] Therefore, step four first involves aggregating the topologically continuous confirmed state surface units into crack objects, then calculating the position coordinates and geometric quantization results around the crack objects, and simultaneously organizing an evidence package that explains why the object was identified.

[0131] In this embodiment, the following actions are performed by the edge computing device, the controller receives the final judgment result, and the human-machine interface receives the evidence package and displays the source image frame, surface position, and quality trajectory. The edge computing device first extracts all surface units in the confirmed state from the surface unit state set, and then performs aggregation based on the adjacency relationships in the surface unit graph; each aggregated surface unit cluster corresponds to a candidate crack object. For each candidate crack object, the edge computing device continues to perform object confidence aggregation, geometric quantization, and evidence package assembly, ultimately obtaining a set of crack objects that can be used simultaneously by the controller and the human-machine interface.

[0132] Edge computing devices first examine the continuity of the skeleton direction and the width gradient relationship between adjacent surface units before deciding whether to merge them into the same crack object. Linear structures in different directions often appear simultaneously on the same complex curved surface; for example, a real crack extends along the root of a rib, while a stamping drawing crack is distributed along another direction. If aggregation is based solely on distance, both would be incorrectly merged into the same object. Therefore, edge computing devices establish connection criteria between adjacent surface units: on the one hand, they must be directly adjacent or adjacent across one layer in the surface unit map; on the other hand, the included angle of their main skeleton directions must be within a preset turning range, and the change in average local width must not exceed a preset width jump range. Surface units that meet the connection criteria are merged into the same object surface unit set.

[0133] Once the object surface unit set is formed, the edge computing device performs gating scores on the object surface unit set from different image frames. and surface unit evidence strength Aggregate the data to obtain the object confidence values. :

[0134]

[0135] In the formula, the object confidence value : Represents a cracked object The overall reliability level serves to provide a unified measure for final judgment and human-computer interface ranking, and its value range is [value range missing]. ; Object surface unit set : Represents a cracked object All surface units included; support frame set Surface unit In the cracked object Which image frames support the content; consistent weights Surface unit In image frame symbols The degree of consistency between the lower and overall orientation and width patterns of the object is used to improve the contribution of continuous and consistent evidence; the value range is [value range missing]. The weight is calculated as: directional consistency weight × width smoothness weight. Directional consistency is determined by the angle between the local skeleton direction of the surface unit and the main direction of the object. The width smoothness weight is determined by... Decide.

[0136] Gating score Surface unit In image frame symbols The credibility of participation in fusion is positive; surface unit evidence strength. Surface unit In image frame symbols The strength of candidate evidence is determined by a value. .

[0137] In one embodiment of a die-cast aluminum housing, the edge computer obtains continuously validated surface units from the rib root corners. These units appear in two front views and one oblique view, with a consistent skeleton orientation and an average local width that slowly varies along the object's length. The edge computer synthesizes these surface units into a cracked object and uses the object confidence value. To coordinate contributions from different perspectives.

[0138] In practice, it is confirmed that the state surface elements are grouped into a small number of interpretable crack objects, each with a uniform object confidence value. .

[0139] After object aggregation is completed, the edge computing device continues to calculate the length, average width, maximum width, number of branches, and position coordinates of each crack object. The length is accumulated along the main line of the object skeleton on the surface element map; the average width and maximum width are derived from the average local width and maximum local width in the object's surface element set; the number of branches is given based on the skeleton intersection points and topological bifurcation in the object's surface element set; the position coordinates are determined by the central surface position and the main extension direction of the object's surface element set. Since the length and width are measured on the reference surface corresponding to the surface element map, local observations from different perspectives can be unified into the same surface coordinate system.

[0140] After quantization, the edge computing device will package an evidence package for each crack object. The evidence package needs to include: supporting image frame index, object surface unit, crack local image patch, and object confidence value. Gating score Source, corresponding surface unit location, length, average width, maximum width, number of branches.

[0141] If the supporting image frames of the cracked object come from multiple camera views, the evidence packets are arranged in view order; if the supporting image frames come from a single camera with multiple stops, the evidence packets are arranged in stop time.

[0142] For example, in the hole edge region of a stamped part, after the edge computing device generates a crack object, it will write the local image block of the object in the first camera image frame, the local image block in the second camera image frame, the corresponding object surface unit set, and the geometric quantization result into the evidence package.

[0143] In practice, step four further transforms the surface unit state set into a crack object set and provides a complete evidence package for each crack object, enabling the controller to perform workpiece handling and the human-machine interface to perform verification and display. The controller obtains executable workpiece handling information, the human-machine interface obtains a traceable support chain, and subsequent feedback can also rely on the already aggregated object information.

[0144] Although crack objects and evidence packages have been generated, the detection process cannot end immediately after object generation. Complex curved metal parts may still retain some surface units in uncertain states within the same acquisition group. These surface units are neither sufficient to directly form crack objects nor should they be completely ignored. At the same time, certain camera perspectives may continuously exhibit delayed actions, lost frame triggers, or over-triggered frame starts within this acquisition group. If these sources are not recorded and reported, the next acquisition group will continue to waste budget on low-confidence perspectives. Therefore, step four also needs to adjust the high-resolution priority order of surface unit clusters and camera evidence weights in the next acquisition group based on the results of this acquisition group, and establish clear exit criteria.

[0145] In this embodiment, the edge computing device extracts all surface units in uncertain states and their corresponding source image frame indices from the surface unit state set, and then combines this with the disturbed acquisition group marker from step one and the high-resolution priority value from step two. and the gating score in step three Feedback is generated. The feedback operates through two paths: the first path affects the surface unit-level scheduling of the next acquisition group, giving repeatedly uncertain surface units a higher priority in high-resolution image patch selection in the next acquisition cycle; the second path affects the camera-level viewpoint weights, reducing the contribution of views that continuously generate anomalous events in the next acquisition group and prioritizing the use of adjacent viewpoint compensation. Subsequently, the edge computing device determines the processing flow for this task—whether to terminate, re-acquire, or output with uncertain tags—based on exit criteria.

[0146] The edge computing device first calculates the degree of uncertainty persistence of each surface unit on the current workpiece. If a surface unit enters an uncertain state multiple times within the current acquisition group, and there is a confirmed crack object in its adjacent surface unit, it indicates that the surface unit is closer to the crack edge extension area; if a camera in the current acquisition group repeatedly experiences delayed action, frame trigger loss, or over-triggering, it indicates that the direct contribution of that viewpoint to the next acquisition group should be reduced.

[0147] To write these two types of information back to the next data collection group, the edge computing device generates updated feedback values. :

[0148]

[0149] In the formula, the updated feedback value Surface unit Updated priority criteria for scheduling in the next acquisition group; risk prior. : This indicates that the original process risk map in step two is assigned to the surface unit. The risk level, with a value range of [value missing]. Uncertain duration Surface unit The cumulative degree to which the current workpiece is judged to be in an uncertain state, with a value range of [value missing]. The cumulative number of times the surface unit within the current workpiece enters an uncertain state is obtained after normalization.

[0150] Neighborhood confirmation Surface unit The degree of aggregation of cracked objects in adjacent surface units serves to move the crack edge region forward to the forefront of the next acquisition group, with a value range of [value missing]. The proportion of adjacent surface units in the confirmed state; the amount of anomaly source suppression. : Indicates the supporting surface unit The primary perspective is the cumulative level of abnormal events in the current data collection group. Its purpose is to prevent low-confidence perspectives from being excessively prioritized. The value range is... The normalized cumulative amount of anomalous events occurring in the current workpiece, representing the primary viewpoint supporting this surface unit. Uncertain gain. Neighborhood gain and abnormal inhibition coefficient All coefficients are non-negative; truncation operator Used to restrict the result within parentheses to a range. Inside.

[0151] Updated feedback value After generation, the edge computing device writes this value into the surface unit scheduling cache before the next acquisition group starts, to replace or superimpose the original risk prior. Meanwhile, edge computing devices write viewpoint correction flags for continuously abnormal camera viewpoints. If a camera experiences delayed action, frame trigger loss, or over-triggering in multiple consecutive acquisition groups, the source weight of that camera in the next acquisition group is reduced, while adjacent cameras that are spatially close to it and whose coverage areas intersect are pre-selected into the compensation sequence.

[0152] If a surface element at the end of a rib of a housing remains in an uncertain state within this acquisition group, and a nearby surface element has been identified as a confirmed crack, and the main-view camera covering that surface element experiences multiple frame start over-triggers, step four will not end there; it will still provide the updated value of that surface element back. Increase, and decrease the source weight of the main-view camera in the next acquisition group, so that the area is covered by its neighboring oblique-view camera.

[0153] When used, the uncertain information of this acquisition group no longer remains at the result display level, but is transformed into an executable scheduling basis and perspective correction basis for the next acquisition group.

[0154] After feedback is complete, step four continues to generate the final determination for the current workpiece. The edge computing device first checks whether an object confidence value exists. Cracks exceeding a preset threshold: If a crack exists, the current workpiece is written into the failure judgment along with a set of crack objects and an evidence package; If no crack objects exist, but the surface unit state set contains uncertain state surface units exceeding a preset range, the current workpiece is written into the uncertainty judgment along with an uncertain region evidence package; If neither crack objects nor uncertain state surface units exceeding a preset range exist, the current workpiece is written into the pass judgment.

[0155] Meanwhile, the edge computing device determines whether to perform supplementary sampling on the current workpiece based on exit criteria. Exit criteria include at least four: whether a final judgment can be generated and the proportion of uncertain state surface units is below the minimum supplementary sampling threshold; whether the maximum number of supplementary sampling attempts has been reached; whether the number is below the minimum supplementary sampling threshold; whether the evidence package for the current workpiece has been fully written into the memory and human-machine interface cache; when any one of the exit conditions is met, the current workpiece is processed, and the controller sends the final judgment to the execution mechanism.

[0156] In the preferred method, if the current workpiece has already been written with an uncertain judgment and the number of re-sampling attempts is less than the minimum re-sampling threshold, while the remaining delay budget is higher than the minimum re-sampling threshold, the controller sends a re-sampling trigger command to the multi-camera acquisition unit again. This time, the high-resolution image block re-sampling can only overwrite the update feedback value. The area of ​​high uncertainty cannot cover the entire workpiece. If the number of re-samples exceeds the upper limit, the current workpiece along with the evidence packet will be output.

[0157] In a generalized implementation, if the production line adopts an offline verification mode, the evidence package corresponding to the uncertain determination is sent to the verification workstation; if the production line adopts an online rejection mode, the controller sends the failure determination to the sorting mechanism, the uncertain determination to the buffer station, and the pass determination to the main conveyor line. Different execution paths do not change the crack object. Evidence package Updated feedback value These terms and data objects.

[0158] In practice, step four determines whether the current workpiece passes, fails, or is uncertain, and establishes a closed loop from the current acquisition group back to the next acquisition group. This ensures that the method not only outputs results but also provides the scheduling correction basis for the next round of processing. By connecting the current result chain with the next round of acquisition chain, the entire method forms a continuous identification process with feedback.

[0159] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0163] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A real-time video recognition method for surface crack defects in complex components, comprising acquiring multi-view and / or multi-timeframe image frames of the workpiece under test, characterized in that: include, Based on reference geometry, the surface of the workpiece under test is discretized into multiple surface units, and the projection correspondence between each image frame and the surface unit is established. At the same time, the acquisition quality metadata corresponding to each image frame is obtained. Low-resolution reconnaissance is performed on each image frame, and the corresponding high-resolution image block is selected based on the risk prior of the surface unit and the reconnaissance uncertainty under the constraints of high-resolution pixel budget and time delay budget. Crack candidate inference is performed on high-resolution image blocks, the obtained candidate crack evidence is mapped to the corresponding surface unit, and gating fusion and state update are performed based on the acquired quality metadata. Cracked objects are formed based on adjacent surface units in the confirmed state, and judgment results and evidence packages are output. Surface units in the uncertain state are fed back as priority objects for subsequent acquisition groups. The reference geometry is discretized according to the local curvature and process risk map to form a surface unit map with surface index, local normal and adjacency relationship; the edge computing device performs global registration and local registration on each image frame to generate the corresponding set of visible surface units and the projection correspondence between the image frame and the surface unit; The edge computing device first generates a low-resolution reconnaissance map for each image frame, and then maps the coarse crack response and reconnaissance uncertainty to surface units to form surface unit-level reconnaissance results. Based on the continuous suspicious responses of adjacent surface units, spatially adjacent and projection-compatible surface units are merged into surface unit clusters to generate corresponding high-resolution image patches. The selection of high-resolution image blocks is constrained by the high-resolution pixel budget, the maximum number of image blocks, and the remaining latency budget. After the priority selection is completed, the edge computing device also supplements the high-resolution image blocks from the low-risk surface units according to the minimum exploration ratio, so that the high-resolution image block request set covers the locations to be verified in both high-risk and low-risk areas. After performing crack candidate inference on high-resolution image blocks, the edge computing device extracts crack mask, skeleton, local width and local confidence. Based on the projection correspondence between image frames and surface units, the crack mask, skeleton and local width are mapped back to the corresponding surface units to form surface unit evidence bound to the source image frame. The edge computing device performs three-stage quality gating on each surface unit based on the collected quality metadata and surface unit evidence. Surface unit evidence with quality higher than the second threshold enters the direct fusion path, surface unit evidence with quality between the first and second thresholds enters the low-weight slow fusion path, and surface unit evidence with quality lower than the first threshold enters the supplementary sampling determination path. When the remaining delay budget is insufficient, it switches to the uncertain path. The edge computing device maintains candidate state, confirmed state, uncertain state and suppressed state for each surface unit. The surface unit cluster is updated to the confirmed state only when the same surface unit cluster simultaneously meets the conditions of independent support number, adjacency continuity and width smoothness. For surface units supported by only a single low-quality view, the candidate state is maintained and the surface unit is switched to the uncertain state when the remaining delay budget is insufficient. The edge computing device aggregates surface units that are in a confirmed state and are topologically continuous into crack objects, and calculates the length, average width, maximum width and number of branches based on the skeleton length and local width of the crack objects. At the same time, it generates an evidence package, which includes the supporting image frame index, the corresponding surface unit position, the crack local image block, the quality metadata trajectory and the quantization result. Edge computing devices read the image frame body, image frame index, and overall quality score from the image frame data packet set. Simultaneously, the surface units covered by the current image frame are read from the visible set of surface units; the process risk map provides risk priors for each surface unit, and the surface unit map provides adjacency relationships for each surface unit; Edge computing devices first conduct reconnaissance at a low-resolution level, and then aggregate the reconnaissance results to the surface unit; Edge computing devices converge the deviation of pixel detection responses within the projection area of ​​a surface unit into detection uncertainty; if most pixel detection responses within a surface unit are biased towards a dark background or linear texture, the detection uncertainty is low; if conflicting responses exist simultaneously within the same surface unit, the detection uncertainty is high. The uncertainty of reconnaissance is calculated using the following formula: ； In the formula, the surface element detection uncertainty is... Surface unit In image frame The reconnaissance uncertainty level below measures whether the surface unit needs further magnification for observation, and its value range is [value range missing]. Projection area Coverage weight Pixel reconnaissance response The meaning remains consistent with the previous expression; transformation terms Used to transfer pixel reconnaissance response from Mapped to This causes the uncertainty of the neutral response to increase during polymerization; After the reconnaissance uncertainty is generated, the edge computing device will analyze the surface unit reconnaissance response. Surface element detection uncertainty Overall quality score Risk priors from the process risk diagram are written into the candidate attention order.

2. The real-time video recognition method for surface crack defects of complex components according to claim 1, characterized in that: The acquired quality metadata includes timestamps, late action markers, frame trigger loss markers, frame start over-trigger markers, link retransmission counts, ambiguity, saturation, and registration residuals. The edge computing device marks each image frame as a low-confidence input based on the acquired quality metadata, and sends it to a downweighted path when the overall quality is below the low-confidence threshold, and to a supplementary acquisition candidate queue when it is below the abnormal threshold.

3. The real-time video recognition method for surface crack defects of complex components according to claim 2, characterized in that: If a surface unit cluster is repeatedly in an uncertain state in the current acquisition group, the edge computing device will feed back the surface unit cluster as a high-resolution budget priority object in the subsequent acquisition group; if a certain camera continuously generates abnormal events such as delayed action, frame trigger loss, and over-triggering, the evidence weight of that camera will be reduced, and the adjacent viewpoints will be used first in the subsequent acquisition group to compensate for the acquisition of the corresponding area.

Images