An image recognition-based lost foam mold cavity defect detection method and system

Abstract

The present application relates to the field of computer vision and visual inspection technology, and more particularly to a lost foam mold cavity defect detection method and system based on image recognition. Obtain the digital reference model of the cavity and divide the cavity surface elements, plan the initial detection viewpoint and the paired opposite directional illumination direction, collect the structured light image and the directional illumination image; Recover the cavity surface space point, map the image to the cavity surface coordinate domain, generate the signed illumination response evidence and the signed geometric deviation evidence; Construct the defect evidence graph based on the evidence node, the support edge, the conflict edge and the occlusion relationship, identify the single viewpoint support, the opposite conflict and the occlusion fracture topology, determine the supplementary detection viewpoint and the supplementary directional illumination direction accordingly, and incrementally update the local evidence subgraph until convergence; Input the convergence subgraph and the associated image into the image recognition model, and output the defect area, the defect category and the cavity surface coordinate. The present application can improve the defect determination reliability under the conditions of curved surface reflection and deep cavity occlusion.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and visual inspection technology, and in particular to a method and system for detecting cavity defects in lost foam casting molds based on image recognition. Background Technology

[0002] The surface quality of the cavity in a lost foam casting mold affects the molding accuracy of the foam model. During processing, assembly, and use, defects such as dents, protrusions, scratches, deposits, and boundary defects may occur on the cavity surface. Because the cavity typically contains curved surfaces, deep cavities, corners, parting lines, venting structures, and insert boundaries, it is difficult to reliably obtain the surface condition of each location within the cavity through manual observation. Therefore, current technologies typically use industrial cameras and lighting components to acquire images of the cavity surface and extract grayscale, edge, texture, or shape features from the images using image recognition technology to determine the presence of defects on the cavity surface.

[0003] For example, Chinese invention patent application CN118429219A, published on August 2, 2024, discloses a visual inspection method for surface defects in injection molds. The method acquires an image capturing component, a lighting component, a sample injection mold, and a corresponding standard injection mold. The working path of the lighting component is determined based on the inner cavity shape of the sample injection mold. Images of both the sample and standard injection molds are acquired. The acquired images are then processed for grayscale conversion, noise reduction, and deblurring. Defect information is determined through edge detection and the correspondence between the image to be inspected and the standard image. For images that do not meet preset image threshold conditions, the corresponding image is fed back to the operator, and the image is re-acquired after inspecting the image capturing component and the lighting component.

[0004] The existing technology can adjust the working path of the lighting component according to the cavity shape and use standard mold images as a comparison basis for defect detection. However, such methods are mainly based on processing two-dimensional images obtained from different shooting positions. When there are large curvature changes, deep cavity obstruction, or local reflections on the cavity surface, the same cavity surface position may present different bright and dark boundaries and texture features under different viewing directions and different lighting directions. The image response formed by actual depressions or protrusions may also be similar to the image response formed by shadows, highlights, and normal cavity process structures, making it difficult to distinguish between actual geometric defects and appearance anomalies caused by lighting conditions based solely on two-dimensional image features.

[0005] For obstructed cavity surface areas, images obtained from different shooting positions may only cover a portion of the defect area. The existing technology primarily processes the image to be inspected and compares defects with corresponding standard images. It struggles to determine whether abnormal areas in different images correspond to the same physical location on the cavity surface, and it's also difficult to determine whether differences in detection results under different viewing directions are caused by the viewing direction, lighting conditions, or obstruction, or by actual defects on the cavity surface. Re-acquiring images solely based on image clarity or lighting uniformity also makes it difficult to determine appropriate subsequent acquisition conditions for the specific cavity surface areas causing the differences in detection results.

[0006] Therefore, the main technical problem that the existing technology needs to solve is: how to improve the reliability of defect determination at the same cavity surface position when there are curved surface reflections, deep cavity occlusions, and different image responses produced by different viewing directions and lighting directions in the lost foam mold cavity. Summary of the Invention

[0007] To overcome the aforementioned technical deficiencies, the present invention aims to provide a method and system for detecting defects in the cavity of lost foam casting molds based on image recognition. This invention maps multi-viewpoint directional illumination images and structured light 3D data uniformly to the cavity surface coordinate domain. Based on the same reference surface normal, it generates signed illumination response evidence and signed geometric deviation evidence, constructing a defect evidence map to characterize multi-viewpoint evidence support, sign conflicts, and occlusion / fracture relationships. Furthermore, it jointly determines supplementary detection viewpoints and supplementary directional illumination directions for different anomalous topologies. After incrementally updating the local evidence sub-maps, image recognition is performed to improve the reliability of defect determination at the same cavity surface location.

[0008] This invention discloses an image recognition-based method for detecting defects in the cavity of lost foam casting molds, comprising:

[0009] S1. Obtain the digital reference model of the cavity of the lost foam mold and divide it into cavity surface units. Determine the reference surface normal, the direction of maximum principal curvature, the initial detection viewpoint, and the paired opposite directional lighting directions. Acquire structured light images and directional lighting images.

[0010] S2. Recover the spatial points on the cavity surface based on the structured light image, map the directional illumination image to the coordinate domain of the cavity surface, and establish the set of visible pixels and occlusion relationships;

[0011] S3. Generate signed illumination response evidence based on pixel responses under opposite directional illumination directions, calculate the directed distance of spatial points on the cavity surface relative to the digital reference model of the cavity along the reference surface normal, and generate signed geometric deviation evidence to form an evidence state.

[0012] S4. Set the evidence records as evidence nodes, establish support edges or conflict edges according to coordinate relationships and evidence symbols, construct a defective evidence map by combining the set of visible pixels and occlusion relationships, identify single-viewpoint support topology, conflict topology with opposite signs and occlusion fracture topology, and determine the set of evidence nodes connected by support edges and having defective evidence states as defective candidate regions.

[0013] S5. Based on the identified topology, determine the supplementary detection viewpoint and supplementary directional lighting direction, acquire supplementary images, limit the influence closure with stable boundary nodes, and incrementally update the local evidence subgraph within the closure until the connected component node set, evidence symbol set and stable boundary node set are the same in two consecutive updates, and each defect candidate region is supported by at least two detection viewpoints with an angle greater than 0° and less than 180° between the observation directions.

[0014] S6. Extract node features from the converged local evidence sub-image and the associated cavity surface image. Determine the node gating weights based on the number of detection viewpoints, the proportion of visible pixels, the support weight, and the conflict weight. Enable the support edges to transmit node features and enable the conflict edges to block feature transmission. Determine the defect region, defect category, and cavity surface coordinates based on the fused features.

[0015] Preferably, determining the initial detection viewpoint and the paired opposite directional illumination directions includes:

[0016] Candidate acquisition combinations are generated based on the reference surface normal, the direction of maximum principal curvature, and the occlusion boundary of each cavity surface unit. Each candidate acquisition combination includes a candidate detection viewpoint and at least two pairs of opposite directional illumination directions.

[0017] Determine the visible cavity surface unit, imaging resolution, and projection direction of each directional illumination direction onto the tangent plane of the corresponding cavity surface unit for each candidate acquisition combination;

[0018] Select an initial acquisition combination from the candidate acquisition combinations, such that each cavity surface unit is covered by at least two initial detection viewpoints, the angle between the observation directions of at least two initial detection viewpoints is greater than 0° and less than 180°, and the projections of at least one pair of opposite directional illumination directions on the corresponding tangent plane are respectively located on both sides of the direction of maximum principal curvature.

[0019] Preferably, dividing the cavity digital reference model into cavity surface units and establishing a process structure reference evidence template includes:

[0020] Curvature change boundary and cavity process structure label are extracted from the cavity digital reference model. Cavity surface unit with continuous reference surface normal is formed according to the curvature change boundary and cavity process structure label. The cavity process structure label includes at least one of parting boundary label, venting structure label, insert boundary label and demolding structure action area label.

[0021] By combining the initial data collection and based on the repeated inspection results of the mold design documents or the defect-free standard mold, the expected lighting response symbol sequence and expected geometric deviation range corresponding to the process structure label of each cavity are determined, and the expected lighting response symbol sequence and expected geometric deviation range are associated as the process structure benchmark evidence template.

[0022] Preferably, mapping the directional illumination image to the cavity surface coordinate domain includes:

[0023] The spatial points on the cavity surface recovered from the structured light image are registered with the digital reference model of the cavity to obtain the pose transformation relationship between each initial detection viewpoint and the digital reference model of the cavity.

[0024] Based on the pose transformation relationship, the effective pixels in each directional lighting image are converted into line-of-sight rays, and the intersection points of each line-of-sight ray and the cavity digital reference model are determined.

[0025] When a line of sight intersects with multiple cavity surface units, the cavity surface unit to which the intersection point has the smallest distance from the corresponding cavity surface spatial point and the angle between the reference surface normal and the direction pointing to the corresponding initial detection viewpoint is less than 90° is determined as the pixel belonging unit. The intersecting cavity surface units located after the intersection point along the propagation direction of the line of sight are determined as occlusion units, thus forming a set of visible pixels and occlusion relationships between cavity surface units.

[0026] Preferably, generating evidence of a signed illumination response includes:

[0027] Exposure response normalization and pixel position registration are performed on each directional illumination image acquired under the same initial detection viewpoint;

[0028] Two pairs of directional illumination directions with non-parallel projection directions on the tangent plane of the corresponding cavity surface unit are selected from the pairs of opposite directional illumination directions. The directional illumination images corresponding to each pair of directional illumination directions are then subjected to pixel-by-pixel difference to obtain the directional response components distributed along the two tangential directions.

[0029] A directed lighting response vector is formed based on two directed response components and their corresponding tangential directions. The local response center is determined based on the zero-value boundary of the two directed response components. The directed lighting response vector pointing to the local response center is assigned an inward sign, and the directed lighting response vector deviating from the local response center is assigned an outward sign.

[0030] Associating inward or outward signs, the direction of the directed lighting response vector, and the response amplitude with evidence of the signed lighting response.

[0031] Preferably, the formation of an evidentiary state includes:

[0032] When a spatial point on the cavity surface is located on the positive side of the normal of the reference surface relative to the corresponding reference surface point, the corresponding directed distance is assigned an outward sign; when a spatial point on the cavity surface is located on the negative side of the normal of the reference surface relative to the corresponding reference surface point, the corresponding directed distance is assigned an inward sign.

[0033] When the signs of the signed lighting response evidence and the signed geometric deviation evidence are the same and both fall within the preset defect evidence range, a state of dual evidence in the same direction is formed.

[0034] When the signs of the signed lighting response evidence and the signed geometric deviation evidence are opposite and both fall within the preset defect evidence range, a state of directional conflict is formed.

[0035] When one type of evidence, either the signed lighting response evidence or the signed geometric deviation evidence, falls into a pre-defined defective evidence interval, and the other type of evidence falls into a pre-defined benchmark evidence interval, a single evidence state is formed.

[0036] Preferably:

[0037] The evidence node records the coordinate range of the cavity surface, the source of the evidence, the evidence symbol, the evidence status, the detection viewpoint, the proportion of visible pixels, and the occlusion relationship;

[0038] It supports two evidence nodes whose edge-connected cavity surface coordinate ranges are adjacent or overlapping, whose evidence symbols are consistent, and whose boundary spacing of visible pixel sets meets the preset connectivity conditions.

[0039] Two evidence nodes whose coordinate ranges overlap and whose evidence signs are opposite are connected by conflicting edges on the cavity surface.

[0040] The support weights of the supporting edges are determined based on the overlap ratio of the cavity surface coordinates of the two evidence nodes, the proportion of visible pixels, and the angle between the observation directions of the corresponding detection viewpoints. The conflict weights of the conflict edges are determined based on the overlap ratio of the cavity surface coordinates, the similarity of the geometric deviation amplitude, and the similarity of the illumination response amplitude.

[0041] Preferably, identifying single-viewpoint support topology, conflicting topology, and occlusion-breaking topology includes:

[0042] A connected component in which all evidence nodes originate from the same detection viewpoint, or a connected component in which evidence nodes originate from multiple detection viewpoints with an angle of 0° or 180° between their observation directions, is defined as a single-viewpoint support topology.

[0043] Evidence nodes with inward and outward signs respectively within the same cavity surface coordinate range and connected by conflict edges are identified as opposite-sign conflict topologies;

[0044] Two sets of evidence nodes with consistent evidence symbols, adjacent cavity surface coordinate boundaries, and separated by occlusion relationships are defined as the occlusion fracture topology.

[0045] Starting from evidence nodes in a single-viewpoint support topology, conflict topology, or occlusion-break topology, extend along support and conflict edges to evidence nodes with consistent evidence signs and supported by detection viewpoints with at least two viewing directions having an angle greater than 0° and less than 180°, forming a local evidence subgraph.

[0046] Preferably, determining the supplementary detection viewpoint and the supplementary directional illumination direction includes:

[0047] Within the reachable pose space and controllable illumination direction range of the detection device, supplementary acquisition combinations are generated. Each supplementary acquisition combination includes a candidate supplementary detection viewpoint and at least two pairs of candidate supplementary directional illumination directions.

[0048] Select supplementary acquisition combinations that can cover the evidence nodes in the identified topology and form image overlaps with the cavity surface areas corresponding to adjacent evidence nodes that are consistent with the evidence symbols.

[0049] Determine the newly added visible cavity surface units corresponding to each supplementary acquisition combination, the angle of the newly added observation direction relative to the existing detection viewpoint, the expected symbol distinction relationship corresponding to the opposite directional illumination direction, and the coverage of the local evidence sub-image;

[0050] Based on the identified topology type, supplementary detection viewpoints and supplementary directional lighting directions are determined from the supplementary acquisition combinations.

[0051] Preferably, determining the supplementary detection viewpoint and supplementary directional illumination direction based on the identified topology type includes:

[0052] Determine the direction of maximum principal curvature and the tangent plane of the cavity surface unit corresponding to the identified topology;

[0053] For single-viewpoint support topology, supplementary detection viewpoints are selected that have an angle greater than 0° and less than 180° with the observation direction of the existing detection viewpoints and simultaneously cover the corresponding evidence node and the adjacent evidence symbol. Supplementary directional illumination directions are also selected whose projections on the tangent plane are located on both sides of the direction of maximum principal curvature.

[0054] For conflicting topologies, determine the main response direction of the conflict evidence nodes, select two pairs of supplementary directional illumination directions whose projections on the tangent plane are parallel and perpendicular to the main response direction, and select supplementary detection viewpoints whose angles with the observation directions corresponding to each existing conflict evidence node are greater than 0° and less than 180°.

[0055] For the occlusion fracture topology, supplementary detection viewpoints are selected that cover the occlusion interval and the evidence nodes on both sides of the occlusion interval, and supplementary directional lighting directions are selected whose projection directions on the tangent plane intersect with the boundary of the occlusion interval.

[0056] Preferably, incremental updates to the local evidence subgraph include:

[0057] The supplementary image is mapped to the cavity surface coordinate domain to generate supplementary evidence nodes. Supplementary support edges or supplementary conflict edges are generated based on the cavity surface coordinate relationship and evidence symbol relationship between the supplementary evidence nodes and the existing evidence nodes.

[0058] Starting from the supplementary evidence nodes and the evidence nodes in the identified topology, the process traverses along newly added supporting edges, newly added conflicting edges, supporting edges or conflicting edges whose weights have changed, and evidence nodes whose evidence symbols have changed.

[0059] The first evidence node reached during the traversal process, supported by at least two observation viewpoints with an angle greater than 0° and less than 180° between them, and with consistent evidence symbols, is identified as a stable boundary node.

[0060] The supplementary evidence nodes, the evidence nodes in the identified topology, and the evidence nodes that the identified evidence nodes pass through before reaching the stable boundary node along the support edge or conflict edge are identified as the influence closure, and the local evidence subgraph is updated within the influence closure.

[0061] Preferably, determining that a local evidence subgraph has reached a convergence state includes:

[0062] After each incremental update, compare the set of connected component nodes, the evidence symbols of each evidence node, and the set of stable boundary node identifiers before and after the update.

[0063] When an incremental update causes a connected component to split, a connected component to merge, or a change in the evidence sign inside a stable boundary node, the corresponding stable boundary node is included in the influence closure, and the influence closure is extended along the changed supporting edge or conflicting edge.

[0064] When two consecutive incremental updates yield the same set of connected component nodes, evidence symbols, and stable boundary node identifiers, and each defect candidate region is supported by at least two detection viewpoints with an angle greater than 0° and less than 180° between their observation directions, the corresponding local evidence subgraph is determined to have reached a convergence state.

[0065] Preferably, inputting the local evidence sub-image and the associated cavity surface image into the image recognition model includes:

[0066] Surface texture features, illumination response features, and geometric deviation features are extracted from the cavity surface image and the corresponding cavity surface spatial points, and the surface texture features, illumination response features, and geometric deviation features are associated with the corresponding evidence nodes;

[0067] Based on the number of detection viewpoints, the proportion of visible pixels, the support weight of supporting edges, and the conflict weight of conflicting edges corresponding to each evidence node, determine the node gating weight and edge propagation coefficient.

[0068] Multiply the node features corresponding to each evidence node by the corresponding node gating weight, set the edge propagation coefficient of the supporting edge to a value greater than 0 determined according to the corresponding supporting weight, and set the edge propagation coefficient of the conflicting edge to 0.

[0069] The node features are transmitted and fused within the same connected component based on the edge propagation coefficient, and the image recognition model determines the defect region and defect category based on the fused node features.

[0070] Preferably, it further includes:

[0071] From the local evidence sub-image fused with node features by the image recognition model, extract the evidence nodes that split the corresponding connected components after removal, the evidence nodes with the largest signed geometric deviation, and the evidence nodes from each detection viewpoint that supports the defect candidate region, to form a set of evidence reproduction nodes.

[0072] A re-inspection acquisition sequence is generated based on the detection viewpoint and directional lighting direction recorded in the evidence reproduction node set;

[0073] Collect re-examination images according to the re-examination acquisition sequence, reconstruct re-examination evidence sub-images based on the re-examination images, and compare the connected component node sets, evidence symbols, and cavity surface coordinate boundaries of the re-examination evidence sub-images with those of the original local evidence sub-images.

[0074] When the set of connected component nodes and evidence symbols of the re-inspection evidence sub-graph are the same as those of the original local evidence sub-graph, and the difference in the coordinate boundary of the cavity surface does not exceed the preset re-inspection boundary difference threshold, the corresponding defect area will be marked as a re-inspection consistent state; when at least one of these conditions is not met, the corresponding defect area will be marked as a re-inspection abnormal state.

[0075] In view of this, the present invention also provides an image recognition-based lost foam die cavity defect detection system, including a rigid calibration acquisition head, a pose adjustment mechanism, a motion controller, an illumination controller, and an image processing device;

[0076] The rigid calibration acquisition head is set at the execution end of the pose adjustment mechanism. The rigid calibration acquisition head includes an image sensor, a structured light projection component, and a directional illumination array. The directional illumination array includes at least four independently controlled illumination units arranged around the image sensor, and the illumination units form at least two pairs of opposite directional illumination directions.

[0077] The motion controller is connected to the posture adjustment mechanism, the lighting controller is connected to the directional lighting array, and the image processing device is communicatively connected to the image sensor, the motion controller, and the lighting controller, respectively.

[0078] Image processing devices include a processor and a memory. The memory stores program instructions, which are configured and formed when the processor executes them.

[0079] The model processing unit is used to obtain the digital reference model of the cavity of the lost foam mold, divide the digital reference model of the cavity into cavity surface units, establish the cavity surface coordinate domain, and determine the reference surface normal and the direction of maximum principal curvature of each cavity surface unit.

[0080] The initial acquisition planning unit is used to determine the initial detection viewpoint and the paired opposite directional illumination directions based on the reference surface normal, the direction of maximum principal curvature and the occlusion boundary, and send pose control commands and illumination control commands to the motion controller and the illumination controller respectively.

[0081] The surface mapping unit is used to acquire structured light images through an image sensor when the structured light projection component projects structured light, recover the spatial points of the cavity surface based on the structured light images, map the directional illumination images acquired by the image sensor to the coordinate domain of the cavity surface, and establish the set of visible pixels and occlusion relationships.

[0082] The signed evidence generation unit is used to generate signed illumination response evidence based on the pixel response of the same cavity surface unit under opposite directional illumination directions, calculate the directed distance of the cavity surface spatial point relative to the cavity digital reference model along the corresponding reference surface normal and generate signed geometric deviation evidence to form an evidence state.

[0083] The evidence graph processing unit is used to set the evidence records of each detection viewpoint as evidence nodes, establish support edges or conflict edges according to the coordinate relationship of the cavity surface and the evidence symbols, construct a defect evidence graph by combining the set of visible pixels and the occlusion relationship, identify single-viewpoint support topology, opposite-sign conflict topology and occlusion fracture topology, and determine the set of evidence nodes connected by support edges and having defect evidence status as defect candidate regions.

[0084] The joint supplementary acquisition unit is used to determine the supplementary detection viewpoint and supplementary directional lighting direction based on the identified topology, send supplementary pose control commands to the motion controller to control the pose adjustment mechanism, send supplementary lighting control commands to the lighting controller to control the directional lighting array, acquire supplementary images through the image sensor, and limit the influence closure with stable boundary nodes, and incrementally update the local evidence subgraph within the influence closure;

[0085] The convergence determination and identification unit is used to determine the local evidence subgraph as convergent when the set of connected component nodes, the set of evidence symbols, and the set of stable boundary nodes are the same in two consecutive incremental updates, and each defect candidate region is supported by at least two detection viewpoints with an angle greater than 0° and less than 180° between the observation directions. The unit extracts node features from the converged local evidence subgraph and the associated cavity surface image, determines the node gating weight based on the number of detection viewpoints, the proportion of visible pixels, the support weight, and the conflict weight, enables the supporting edges to transmit node features and enables the conflicting edges to block feature transmission, and determines the defect region, defect category, and cavity surface coordinates based on the fused features.

[0086] Compared with existing technologies, the above technical solution has the following advantages:

[0087] 1. This invention maps structured light 3D data and directional illumination images obtained from different detection viewpoints to the cavity surface coordinate domain, enabling abnormal responses in different images to be associated with the same cavity surface location, and using multi-viewpoint detection results for mutual verification, thereby improving the reliability of locating and determining defects on complex cavity surfaces.

[0088] 2. This invention utilizes paired opposite directional lighting directions to generate signed lighting response evidence, and generates signed geometric deviation evidence based on the normal directional distance of spatial points on the cavity surface relative to the digital reference model of the cavity. By judging the state of depression, convexity and directional conflict through the amplitude and sign relationship of the two types of evidence, it can reduce the possibility that shadows, specular highlights and local reflections are identified as actual geometric defects.

[0089] 3. This invention identifies the topology of obstructed fractures through defect evidence maps, and determines supplementary detection viewpoints and supplementary directional lighting directions based on the obstruction interval and the spatial relationship between evidence nodes on both sides of the obstruction interval. This allows the supplementary line of sight to cover the cavity surface area that was not fully observed by the original detection viewpoint, thereby reducing the loss of detection information caused by deep cavities, corners and local structural obstructions.

[0090] 4. This invention identifies single-viewpoint support topology, conflicting symbol topology, and occlusion-fracture topology respectively, and selects corresponding supplementary detection viewpoints and supplementary directional illumination directions according to different abnormal topologies. It can perform supplementary acquisition for different reasons such as insufficient multi-viewpoint support, conflicting evidence symbols, and occlusion separation, avoiding repeated acquisition of images with low information increment.

[0091] 5. This invention writes the cavity surface coordinate range, evidence source, evidence symbol, detection viewpoint, visible pixel ratio and occlusion relationship into the evidence node, and records the relationship between different pieces of evidence through supporting edges and conflicting edges, so that the formation process of the defect area, the evidence support source and the conflict change process can be traced along the defect evidence map.

[0092] 6. The present invention determines the influence closure based on newly added evidence nodes, changed support edges, and conflicting edges, and updates the local evidence subgraph only within the influence closure. This eliminates the need to recalculate the complete defect evidence graph after each supplementary collection, thereby reducing the repeated processing of areas not affected by supplementary evidence.

[0093] 7. In this invention, after obtaining the same set of connected component nodes, evidence symbols, and stable boundary nodes in two consecutive incremental updates, and after the defect candidate region obtains support from at least two non-collinear detection viewpoints, the local evidence subgraph is determined to have reached a convergence state, and a re-inspection is performed through the evidence reproduction node, so that the final output defect region, defect category, and cavity surface coordinates have a multi-viewpoint evidence basis. Attached Figure Description

[0094] Figure 1 This is a schematic diagram of the overall process of the image recognition-based lost foam die cavity defect detection method of the present invention.

[0095] Figure 2 This is a schematic diagram showing the overall structure and connection relationship of the image recognition-based lost foam mold cavity defect detection system of the present invention.

[0096] Figure 3 This is a schematic diagram of the cavity surface unit division, reference surface normal, maximum principal curvature direction, and cavity process structure label of the cavity digital reference model of the present invention.

[0097] Figure 4 This is a schematic diagram illustrating the generation principle of the signed illumination response evidence and the signed geometric deviation evidence of the present invention.

[0098] Figure 5 This is a schematic diagram of the node connection relationship of the defect evidence diagram of the present invention, as well as the single-view support topology, the opposite-sign conflict topology, and the occlusion break topology.

[0099] Figure 6 This is a schematic diagram illustrating the determination of supplementary detection viewpoints and supplementary directional lighting directions for single-viewpoint support topology, conflicting topology, and occlusion-breaking topology according to the present invention.

[0100] Figure 7 This is a schematic diagram illustrating the process of determining the influence closure of the local evidence subgraph, incremental updates, and convergence status determination in this invention.

[0101] Figure 8 This is a schematic diagram of the image feature fusion and image recognition model based on local evidence subgraphs of the present invention. Detailed Implementation

[0102] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. These specific embodiments are used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0103] Without altering the mapping of the cavity surface coordinate domain, the generation of signed illumination response evidence and signed geometric deviation evidence, the construction of defect evidence maps, the identification of abnormal topology, the joint determination of supplementary detection viewpoints and supplementary directional illumination directions, the incremental updating of local evidence sub-maps, and the main line of image recognition technology, those skilled in the art can adaptively adjust the image acquisition parameters, illumination parameters, pose adjustment methods, judgment thresholds, and image recognition model parameters based on the cavity size, cavity material, surface reflection characteristics, processing accuracy, and detection environment of the lost foam mold to be inspected.

[0104] Unless otherwise explicitly stated, the geometric dimension threshold and positional difference threshold in the specific implementation are determined based on the spatial measurement accuracy of the detection equipment, the design tolerance of the lost foam casting mold to be detected, and the repeated detection results of the defect-free standard mold; the pixel ratio threshold, illumination response threshold, and evidence judgment threshold are determined based on the grayscale response range of the image sensor, the illumination calibration results, and the response statistics of the defect-free standard mold; the support weight, conflict weight, supplementary acquisition combination evaluation weight, and image recognition model training parameters are determined based on the recognition results of the verification data. Specific values ​​are used to illustrate an implementable parameter configuration and are not intended to limit the scope of protection of this invention.

[0105] Example 1

[0106] This embodiment provides a method for detecting cavity defects in lost foam casting molds based on image recognition.

[0107] like Figure 2 As shown, the detection system executing the detection method includes a rigid calibration acquisition head, a pose adjustment mechanism, a motion controller, an illumination controller, and an image processing device. The rigid calibration acquisition head is located at the execution end of the pose adjustment mechanism and includes an image sensor, a structured light projection assembly, and a directional illumination array. The image sensor, structured light projection assembly, and directional illumination array are fixedly connected by a rigid bracket, maintaining a fixed relative position and orientation during detection. The rigid bracket remains fixed after the equipment is installed to reduce external parameter variations caused by repeated assembly.

[0108] The pose adjustment mechanism can employ a six-axis industrial robotic arm. In one embodiment, the working radius of the six-axis industrial robotic arm is... Rated load is The repeatability accuracy is For cavity opening size smaller than For lost foam molds with a small number of detection directions and fixed detection paths, the pose adjustment mechanism can also consist of three linear motion axes and two rotary motion axes.

[0109] The motion controller receives the target pose from the image processing device, performs inverse kinematics solving, trajectory interpolation, speed control, and collision detection, and feeds back the actual position, actual attitude, arrival flag, motion anomaly flag, and collision detection results of the execution end to the image processing device.

[0110] The image sensor uses an area-array industrial camera. The resolution of the image sensor can be... Pixels, cell size can be The lens focal length can be .

[0111] A structured light projection assembly is positioned on one side of the image sensor and forms a known structured light triangulation baseline with the image sensor. The structured light projection assembly can be a phase-shifting fringe structured light projector, a coded fringe structured light projector, or a laser line projector. In one embodiment, the structured light projection assembly uses a projection wavelength of... A phase-shifting fringe structured light projector.

[0112] The directional illumination array is arranged around the image sensor and includes a first illumination unit, a second illumination unit, a third illumination unit, and a fourth illumination unit. The first and third illumination units are arranged on opposite sides of the image sensor and form a first pair of opposite directional illumination directions. The second and fourth illumination units are arranged in a direction that is not parallel to the first pair of opposite directional illumination directions and form a second pair of opposite directional illumination directions.

[0113] The first pair of opposite directional illumination directions and the second pair of opposite directional illumination directions can be projected onto the imaging plane of the image sensor and can be perpendicular to each other or form an angle of 60° to 120°.

[0114] Each lighting unit is independently controlled by a lighting controller, and its on / off status, output illuminance, pulse duration, and trigger delay can be set separately. The output illuminance of a single lighting unit can be set to 8000 lux. Illuminance adjustment can be achieved using 256 levels of pulse width modulation.

[0115] To reduce the impact of specular highlights on the surface of the metal cavity on the directional illumination image, a first polarizer can be placed in front of the image sensor, and a second polarizer can be placed in front of each illumination unit, with the polarization directions of the first polarizer and the second polarizer being orthogonal to each other.

[0116] The structured light projection component and the directional illumination array operate in a time-division multiplexing mode. When acquiring structured light images, the directional illumination array is turned off and the structured light projection component is turned on; when acquiring directional illumination images, the structured light projection component is turned off, and each illumination unit is turned on sequentially in the order of the first illumination unit, the third illumination unit, the second illumination unit, and the fourth illumination unit. The two illumination units in opposite directions acquire images continuously to reduce the impact of changes in the lost foam casting mold or the detection environment over a long time interval on the differential results.

[0117] The image processing device includes a processor, a memory, an image acquisition interface, a motion communication interface, and an illumination communication interface. The image processing device communicates with the image sensor, motion controller, and illumination controller, respectively. The memory stores program instructions executed by the processor. When these instructions are executed, they configure a model forming processing unit, an initial acquisition planning unit, a surface mapping unit, a signed evidence generation unit, an evidence map processing unit, a joint supplementary acquisition unit, and a convergence determination and recognition unit.

[0118] The model processing unit is used to acquire the cavity digital reference model of the lost foam mold to be inspected, divide the cavity digital reference model into cavity surface units, and determine the reference surface normal, the direction of maximum principal curvature, and the cavity process structure label of each cavity surface unit.

[0119] The initial acquisition planning unit is used to determine the initial detection viewpoint and paired opposite directional illumination directions based on the reference surface normal of the cavity surface unit, the direction of maximum principal curvature, the occlusion boundary, and the reachable pose space of the pose adjustment mechanism.

[0120] The surface mapping unit is used to recover the spatial points of the cavity surface from the structured light image and map the directional illumination image to the coordinate domain of the cavity surface.

[0121] The signed evidence generation unit is used to generate signed lighting response evidence and signed geometric deviation evidence.

[0122] The evidence graph processing unit is used to establish evidence nodes, support edges, and conflict edges, and to identify single-view support topology, conflict topology with different signs, and occlusion-breaking topology.

[0123] The joint supplementary acquisition unit is used to determine the supplementary detection viewpoint and supplementary directional lighting direction based on the abnormal topology.

[0124] The convergence determination and recognition unit is used to incrementally update the local evidence subgraph, determine whether the local evidence subgraph has reached the convergence state, and output the defect region, defect category, and cavity surface coordinates.

[0125] One type of hardware and acquisition parameter configuration is shown in Table 1.

[0126] Table 1 Hardware and Acquisition Parameter Configuration Table

[0127] Image resolution Pixels Directional illumination image and structured light image acquisition Pixel size Camera Intrinsic Calibration Lens focal length Imaging and Field of View Determination working distance 180 to Feasibility constraints of inspection viewpoints Structured light reconstruction accuracy 0.03 to Signed geometric deviation calculation Illuminance output of a single lighting unit 8000 to Directional illumination image acquisition Pose repeatability positioning accuracy Multi-view spatial correspondence Image exposure time 0.5 to Prevent underexposure or overexposure Allowed observation direction angle 20° to 150° Multi-view independence constraint Maximum number of supplementary collections Second-rate Supplementary data collection exit conditions

[0128] Before the detection system is put into use, the image sensor, structured light projection component, rigid calibration acquisition head and pose adjustment mechanism are jointly calibrated.

[0129] First, the checkerboard calibration board is set within the working distance range of the image sensor. Images of the calibration board are acquired from no fewer than 15 different positions and orientations. The image coordinates and the plane coordinates of the calibration board are obtained through corner detection. The camera intrinsic parameter matrix, radial distortion coefficient, and tangential distortion coefficient of the image sensor are then calculated.

[0130] The camera intrinsic parameter matrix of the image sensor is represented as follows:

[0131]

[0132] in, The camera intrinsic parameter matrix representing the image sensor; This represents the equivalent focal length of the image sensor in the horizontal direction, expressed in pixels. This represents the equivalent focal length of the image sensor in the vertical direction, expressed in pixels. Represents the x-coordinate of the principal point in the image coordinate system, in pixels; This represents the ordinate of the principal point in the image coordinate system, in pixels.

[0133] When calibrating the structured light projection component, the structured light projection component is used as a reverse imaging device. The correspondence between the projected pixels and the spatial points of the calibration board is established based on the phase value of the projected stripes on the calibration board. The equivalent intrinsic parameters of the structured light projection component and the extrinsic parameters of the structured light projection component relative to the image sensor are calculated.

[0134] Furthermore, the rigid calibration acquisition head is moved to at least 10 different postures, and images of the fixed calibration plate are acquired in each posture, while the actual posture of the actuator of the posture adjustment mechanism is recorded. Based on the posture changes of the image sensor relative to the calibration plate and the posture changes of the actuator of the posture adjustment mechanism, the fixed transformation matrix of the rigid calibration acquisition head relative to the actuator of the posture adjustment mechanism is solved.

[0135] After calibration, the pose of the image sensor relative to the coordinate domain of the cavity surface can be calculated from the actual pose of the actuator of the pose adjustment mechanism under any detection pose.

[0136] During the inspection process, the following coordinate systems are established: model coordinate system, image sensor coordinate system, actuator coordinate system, pose adjustment mechanism base coordinate system, and image coordinate system. The model coordinate system is fixed to the digital reference model of the cavity and serves as the three-dimensional reference coordinate system for the cavity surface coordinate domain. The image sensor coordinate system has its origin at the optical center of the image sensor and its third coordinate axis at the optical axis of the image sensor. The actuator coordinate system is fixed to the actuator end of the pose adjustment mechanism. The pose adjustment mechanism base coordinate system is fixed to the pose adjustment mechanism base. The image coordinate system has its origin at the upper left corner of the image.

[0137] The relationship between the spatial points on the cavity surface in the model coordinate system and the image sensor coordinate system is expressed as follows:

[0138]

[0139] in, Represents the homogeneous coordinates of a point on the cavity surface in the image sensor coordinate system; This represents the fixed transformation matrix from the execution end coordinate system to the image sensor coordinate system; This represents the real-time transformation matrix from the base coordinate system to the actuator coordinate system of the posture adjustment mechanism; The installation transformation matrix represents the transformation from the model coordinate system to the coordinate system of the pose adjustment mechanism base. This represents the homogeneous coordinates of a point on the cavity surface in the model coordinate system.

[0140] The actual illumination direction of each illumination unit is calibrated using a diffuse reflection plane. The diffuse reflection plane is placed within the working distance of the image sensor, each illumination unit is activated, and the illuminance distribution on the diffuse reflection plane is collected. The center illumination direction of each illumination unit is calculated based on the illuminance peak position and illuminance gradient direction. The illuminance coefficient of each illumination unit is determined by the average grayscale value of the same diffuse reflection plane under the same exposure time.

[0141] After system calibration is completed, the image processing equipment acquires the digital reference model of the cavity of the lost foam mold to be inspected. The digital reference model of the cavity can be derived from the STEP file, IGES file, or STL file formed during the mold design stage, or it can be obtained through the reconstruction of the 3D scanning point cloud of a defect-free standard mold.

[0142] When reconstructing the digital reference model of the cavity using 3D scanning point cloud, outliers are first removed and holes are filled in the 3D scanning point cloud, and then a triangular mesh model is generated by Poisson surface reconstruction or triangulation.

[0143] For STEP or IGES files, discretization is first performed according to preset chord height error and normal angle error. In one embodiment, the chord height error is set to... The maximum normal angle error between adjacent triangular facets is set to 5°. After discretization, the vertices of the triangular facets are aligned in a uniform direction facing the cavity space, so that the reference surface normals of each triangular facet point towards the cavity space.

[0144] The image processing equipment establishes local neighborhoods for each grid vertex in the cavity digital reference model and calculates the principal curvature using a local quadratic surface fitting method. Taking the grid vertex to be calculated as the local coordinate origin, a local quadratic surface is fitted based on adjacent grid vertices:

[0145]

[0146] in, Indicates the height along the normal to the local reference surface, in millimeters; Represents the fitting coefficients of the first and second quadratic terms; The first coordinate of the local tangent plane is represented in millimeters. Represents the fitting coefficients of the cross quadratic terms; The second coordinate of the local tangent plane, in millimeters; Represents the fitting coefficient of the second quadratic term; Represents the fitting coefficients for the first-order term; Represents the fitting coefficient of the second-order term; This represents the constant fitting coefficient.

[0147] Substitute each grid vertex in the local neighborhood into the local quadratic surface and solve for the fitting coefficients using the least squares method. Based on the second derivative matrix of the local quadratic surface, determine the two principal curvatures and two principal curvature directions. The direction corresponding to the principal curvature with the larger absolute value is determined as the direction of maximum principal curvature.

[0148] When the included angle between the reference surface normals of adjacent mesh vertices reaches 15°, or the maximum principal curvature difference between adjacent mesh vertices reaches... When this occurs, the corresponding position is determined as the curvature abrupt change boundary.

[0149] The curvature abrupt change threshold is determined based on the design curvature change of the cavity to be tested and the statistical results of the grid curvature of a defect-free standard mold. In another embodiment, the median and median absolute deviation of the included normal angles of all adjacent grid vertices are calculated, and the sum of the median and three times the median absolute deviation is used as the adaptive included normal angle threshold.

[0150] Cavity process structure labels include at least one of the following: parting boundary label, venting structure label, insert boundary label, and demolding structure action area label. Cavity process structure labels can be directly read from the design features in the cavity digital reference model, or they can be manually annotated on the cavity digital reference model. For cylindrical holes, long slots, closed assembly boundaries, and straight parting contours, cavity process structure labels can be generated based on the shape, size, and topological boundaries of the corresponding geometric entities.

[0151] The image processing device performs region growth on the triangular mesh along the curvature abrupt boundary and the cavity process structure label boundary. Region growth starts from any unassigned triangular facet and adds triangular facets that are adjacent to the current region, do not cross the curvature abrupt boundary and the cavity process structure label boundary, and whose reference surface normal angle is less than a preset continuity threshold to the current region, thereby forming cavity surface units with continuous reference surface normals.

[0152] The area of ​​a single cavity surface unit can be set to 20 to When the area is greater than When the area is less than the maximum principal curvature, the corresponding region is further divided; when the area of ​​the region is less than the maximum principal curvature, the corresponding region is further divided. If no independent cavity process structure label is included, the corresponding area will be incorporated into the adjacent cavity surface unit with the largest shared boundary length.

[0153] Each cavity surface unit is assigned a unique cavity surface unit identifier, and at least the following information is recorded: the set of triangular facets, center coordinates, surface area, reference surface normal, maximum principal curvature, direction of maximum principal curvature, cavity process structure label, identifiers of adjacent cavity surface units, and design tolerances.

[0154] The data structure of the cavity surface unit is shown in Table 2.

[0155] Table 2 Data Structure Table for Cavity Surface Unit

[0156] Cavity surface unit marking Unique Integer Number Regional division results Triangular facet set Triangle facet index set Cavity digital reference model center coordinates 3D coordinates Weighted calculation of triangular facet area Reference surface normal Three-dimensional unit vector Triangular facet normal weighted calculation Maximum principal curvature Curvature value Local quadratic surface fitting maximum principal curvature direction Tangent plane unit vector Principal curvature calculation Cavity process structure label Tag Encoding Design model or manual annotation Adjacent unit identifier Adjacent number set Mesh topology Design tolerances Length tolerance Mold design documents

[0157] The image processing device generates candidate detection viewpoints based on the reference surface normal, the direction of maximum principal curvature, the occlusion boundary, and the reachable pose space of the pose adjustment mechanism for each cavity surface unit.

[0158] For each cavity surface unit, with the center of the cavity surface unit as the target point and the opposite direction of the reference surface normal as the initial observation direction, candidate observation directions are generated at fixed angular intervals within a conical range surrounding the initial observation direction.

[0159] In one implementation, the angle between the candidate observation direction and the reference surface normal is taken at 10° intervals from 0° to 65°, and the azimuth angle around the reference surface normal is taken at 15° intervals. The distance between the image sensor and the center of the cavity surface unit ranges from 180° to... according to Interval values ​​are used. Candidate detection viewpoints are determined by the observation direction, observation distance, and image sensor roll angle.

[0160] For each candidate detection viewpoint, perform inverse kinematics solution of the pose adjustment mechanism and collision detection. If there is no inverse kinematic solution that satisfies the joint limit for the candidate detection viewpoint, or if the motion path from the current pose to the candidate detection viewpoint collides with the lost foam mold entity, then the corresponding candidate detection viewpoint is eliminated.

[0161] For candidate viewpoints that pass accessibility detection, a ray is emitted from the optical center of the image sensor towards uniformly selected sampling points within the cavity surface unit. If the ray intersects the corresponding cavity surface unit first, the corresponding sampling point is determined to be a visible point.

[0162] No. The candidate detection viewpoints for the first The visible proportions of each cavity surface unit are expressed as follows:

[0163]

[0164] in, Indicates the first The candidate detection viewpoints for the first The visible proportion of each cavity surface unit; Indicates from the first The candidate detection viewpoint is visible to the first Number of sampling points per cavity surface unit; Indicates the first The total number of sampling points for each cavity surface unit; Indicates the candidate detection viewpoint index; Indicates the cavity surface unit index.

[0165] When the visible ratio is greater than or equal to 0.60, the candidate detection viewpoint is considered to effectively cover the corresponding cavity surface unit; when the visible ratio is less than 0.15, the candidate detection viewpoint is considered to not provide valid evidence for the corresponding cavity surface unit; when the visible ratio is between 0.15 and 0.60, the corresponding cavity surface unit is recorded as a locally visible unit.

[0166] Calculate the projected size of an image pixel on the cavity surface based on the image sensor's focal length, working distance, and observation angle. When the side length of the projected image pixel on the cavity surface is greater than... When the corresponding candidate detection viewpoint is not used to detect sizes smaller than 10,000 pixels, the corresponding candidate detection viewpoint is not used to detect sizes smaller than 10,000 pixels. The defects.

[0167] Each candidate acquisition combination includes a candidate detection viewpoint and at least two pairs of opposite directional illumination directions. For any candidate detection viewpoint, the calibrated illumination directions of each illumination unit are transformed to the model coordinate system and projected onto the tangent plane of the corresponding cavity surface unit. The two projection directions of the first pair of opposite directional illumination directions are located on both sides of the direction of maximum principal curvature, and the projection directions of the second pair of opposite directional illumination directions are not parallel to the first pair of projection directions.

[0168] When selecting the initial acquisition combination from the candidate acquisition combinations, the basic constraint is to cover all cavity surface units, and each cavity surface unit is covered by at least two initial detection viewpoints with an angle between the observation directions greater than 0° and less than 180°. The angle between the observation directions between the initial detection viewpoints can be set from 30° to 120°.

[0169] The initial acquisition combination is determined using a greedy coverage method. First, the candidate acquisition combination that can cover the most uncovered cavity surface units is selected. Then, the number of detection viewpoints for each cavity surface unit is updated, and the candidate acquisition combination that reduces the number of cavity surface units that do not meet the dual viewpoint coverage condition is selected the most, until all cavity surface units meet the coverage requirements.

[0170] If multiple candidate acquisition combinations have the same coverage, they are sorted in order of motion path length, visibility ratio, and angle of observation direction, and the candidate acquisition combination with shorter motion path, higher visibility ratio, and larger angle of observation direction is selected.

[0171] The image processing equipment establishes a process structure benchmark evidence template based on the cavity process structure label. The process structure benchmark evidence template records the expected lighting response symbol sequence, expected geometric deviation range, contour shape, and size range of a normal process structure.

[0172] The expected lighting response symbol sequence is determined based on the directional lighting direction in the initial acquisition combination and the repeated inspection results of defect-free standard molds. The expected geometric deviation range can be directly derived from the mold design documents, or it can be determined by repeatedly acquiring data from no fewer than 10 defect-free standard molds.

[0173] A template for process structure benchmark evidence is shown in Table 3.

[0174] Table 3. Template for Process Structure Benchmark Evidence

[0175] Fractal boundary label Extroversion—Introversion—Extroversion to Position, orientation, and continuous contour matching Exhaust structure label Introversion to Hole position, diameter and depth matching Patch border label Extroversion-Introversion or Introversion-Extroversion to Closed contour matching Demolding structure action area label Continuous change along the demolding direction to Curvature variation continuity matching

[0176] After completing the initial acquisition planning, the motion controller controls the pose adjustment mechanism to sequentially reach each initial detection viewpoint. When the distance between the actual position reported by the pose adjustment mechanism and the target position is less than... When the angle error between the actual posture and the target posture is less than 0.05° and continues for two control cycles, it is determined that the rigid calibration acquisition head has reached the target detection viewpoint.

[0177] At each initial detection viewpoint, image acquisition is performed according to a fixed time sequence. First, all illumination units are turned off, and a dark-field image is acquired. Then, the structured light projection component is turned on, and structured light images are acquired in phase-shift order. After the structured light projection component is turned off, the first, third, second, and fourth illumination units are turned on in sequence, and directional illumination images are acquired respectively. Finally, all illumination units are turned off.

[0178] Before acquiring directional illumination images, the exposure status is checked using pre-acquired images. If the proportion of pixels with gray values ​​greater than 95% of the image sensor's maximum gray value in the pre-acquired image is greater than 2%, the exposure time or illumination intensity is reduced; if the proportion of pixels with gray values ​​less than 10% of the image sensor's maximum gray value is greater than 60%, the exposure time or illumination intensity is increased.

[0179] If the exposure conditions cannot be met after three exposure adjustments, the current detection viewpoint is marked as an abnormal acquisition viewpoint, and an alternative detection viewpoint that is closest in observation direction and meets the accessibility condition is selected from the candidate detection viewpoints.

[0180] Each image is stored in association with the cavity number, cavity surface coordinate domain version, detection viewpoint identifier, actual pose of the pose adjustment mechanism, illumination unit identifier, illumination intensity, exposure time, and acquisition time.

[0181] The data structure of the image acquisition and recording is shown in Table 4.

[0182] Table 4. Image Acquisition Record Data Structure Table

[0183] Cavity number Current lost foam casting mold markings to be inspected Detection viewpoint markers The current detection viewpoint has a unique identifier. Actual pose Position coordinates and attitude parameters Image type Dark field images, structured light images, or directional illumination images Lighting direction signs Current lighting direction number Exposure time Image sensor exposure parameters Lighting intensity Lighting controller output level Image timestamp Image acquisition time Data validity status Effective, underexposed, overexposed, or abnormal motion

[0184] After the structured light image is acquired, phase-shift decoding is used to obtain the wrapper phase of each effective pixel, and then the absolute phase is obtained through coded stripes or multi-frequency phase unwrapping. The spatial points on the cavity surface are recovered by intersecting the projection plane of the image sensor rays and the structured light projection component.

[0185] For phase shift number Structured light images, the first The wrapper phase of each pixel is represented as:

[0186]

[0187] in, Indicates the first The wrapping phase of each pixel; Represents the image pixel index; Represents the arctangent function in the four quadrants; Indicates the number of phase-shifted images; Indicates the phase-shifted image sequence number; Indicates the first The pixel in the first Gray values ​​in an amplitude-phase-shifted image; It represents pi (π).

[0188] For pixels with a modulation index lower than a preset modulation index threshold, the corresponding pixels are marked as invalid structured light pixels. The structured light modulation degree of each pixel is represented as:

[0189]

[0190] in, Indicates the first Structured light modulation of each pixel; Indicates the number of phase-shifted images; Indicates the phase-shifted image sequence number; Indicates the first The pixel in the first Gray values ​​in an amplitude-phase-shifted image; It represents pi (π).

[0191] When the structured light modulation degree is less than 5% of the maximum gray value of the image sensor, the corresponding pixel is not used to recover the cavity surface space point.

[0192] After restoring the spatial points on the cavity surface, the structured light point cloud is registered with the cavity digital reference model. First, coarse registration is performed based on the cavity positioning holes, the intersection of the parting boundary, or the positioning reference plane, and then fine registration is performed using the iterative nearest point algorithm from point to plane.

[0193] The registration error from point to plane is expressed as:

[0194]

[0195] in, This represents the root mean square error of registration, in millimeters. Indicates the number of spatial points on the cavity surface involved in registration; Indicates the spatial point index of the cavity surface involved in registration; The registration rotation matrix represents the registration of the structured light point cloud to the cavity digital reference model; Indicates the first One measured spatial point on the cavity surface; This represents the registration translation vector from the structured light point cloud to the cavity digital reference model; Indicates the relationship with the first The reference surface point corresponding to each measured cavity surface spatial point; Represents the reference surface normal corresponding to the reference surface point; symbol This represents the dot product of vectors.

[0196] When the root mean square error of registration is greater than At this time, structured light data from the current detection viewpoint is not used to generate signed geometric bias evidence. The image processing device re-executes structured light image acquisition; the root mean square error of registration after re-acquisition is still greater than [value missing]. At that time, the detection viewpoint is redefined.

[0197] The mapping relationship between pixels in a directional illumination image and spatial points on the cavity surface is expressed as:

[0198]

[0199] in, Indicates the first The scale factor corresponding to each effective pixel; Indicates the first Homogeneous image coordinates of 1 effective pixel; The camera intrinsic parameter matrix representing the image sensor; The rotation matrix representing the cavity surface coordinate domain to the image sensor coordinate system; Indicates the first The coordinates of the spatial points on the cavity surface corresponding to each effective pixel; This represents the translation vector from the cavity surface coordinate domain to the image sensor coordinate system.

[0200] For each valid pixel in the directional illumination image, a gaze ray is generated based on the camera intrinsic matrix and the image sensor pose. The intersection point of the gaze ray and the cavity digital reference model is then calculated using a bounding box hierarchy to accelerate the computation. The line of sight corresponding to each pixel is represented as:

[0201]

[0202] in, Indicates the first The spatial point on the line of sight corresponding to each pixel; Indicates the ray distance parameter; This represents the coordinates of the optical center of the image sensor in the coordinate domain of the cavity surface; Indicates the first Each pixel corresponds to a unit of viewing direction.

[0203] When a line of sight intersects multiple cavity surface units, the cavity surface unit belonging to the intersection point with the smallest distance to the spatial point of the cavity surface recovered by structured light, and with the angle between the reference surface normal and the direction pointing to the current detection viewpoint being less than 90°, is determined as the pixel-assigned unit. Cavity surface units located after the pixel-assigned intersection point along the propagation direction of the line of sight are determined as occlusion units.

[0204] When all intersections of the line-of-sight rays do not satisfy the reference surface normal angle condition, or the distance between the nearest intersection and the spatial point on the cavity surface recovered by structured light is greater than... When this happens, the corresponding pixel is determined to be an invalid mapped pixel.

[0205] A set of visible pixels is established for each cavity surface unit. The set of visible pixels records the pixel coordinates, detection viewpoint identifier, illumination direction identifier, pixel-assigned triangular facet, cavity surface coordinates, and visibility status. The visible pixel ratio is the ratio of the effective pixel area actually assigned to the cavity surface unit to the theoretical projected area of ​​the corresponding cavity surface unit under the current detection viewpoint.

[0206] Before generating signed illumination response evidence, distortion correction, pixel registration, dark field subtraction, and exposure response normalization are performed on four directional illumination images from the same detection viewpoint. Pixel registration is based on the cavity surface coordinates, rather than directly using two-dimensional image coordinates for translation registration, in order to reduce mapping errors caused by curved surface perspective distortion.

[0207] No. The cavity surface unit in the first The normalized pixel response under each directional illumination direction is represented as follows:

[0208]

[0209] in, Indicates the first The cavity surface unit in the first Normalized pixel response under each directional lighting direction; Indicates the cavity surface unit index; Indicates the direction index of directional lighting; Indicates the first The set of visible pixels of each cavity surface unit in the first... The average grayscale value of the original pixel under each directional lighting direction; Indicates the first The average gray level of the dark field corresponding to each directional lighting direction; Indicates the first Exposure time corresponding to each directional lighting direction; Indicates the first The calibrated illuminance coefficient of each lighting unit.

[0210] When localized saturated pixels exist within the cavity surface unit, pixels with grayscale values ​​exceeding 98% of the image sensor's maximum grayscale value are first removed, and then the average grayscale value of the original pixels is calculated. If the number of effective pixels after removal is less than 50% of the original visible pixel count, the corresponding directional lighting image is marked as an overexposed image, and the exposure time or lighting intensity is reduced before re-acquiring.

[0211] The normalized pixel responses corresponding to the first illumination unit and the third illumination unit are differentially analyzed to obtain the first directed response component:

[0212]

[0213] The normalized pixel responses corresponding to the second illumination unit and the fourth illumination unit are differentiated to obtain the second directed response component:

[0214]

[0215] in, Indicates the first The directional response components of each cavity surface unit along the first tangential direction; Indicates the first The directional response components of each cavity surface unit along the second tangential direction; This represents the normalized pixel response corresponding to the first illumination unit; This represents the normalized pixel response corresponding to the second illumination unit; This represents the normalized pixel response corresponding to the third lighting unit; Indicates the normalized pixel response corresponding to the fourth illumination unit; superscript Indicates lighting response; subscript and These represent two tangential directions.

[0216] The first pair of opposite directional lighting directions in the... The unit projection direction on the tangent plane of the cavity surface unit is denoted as The second pair of opposite directional lighting directions will be in the first The unit projection direction on the tangent plane of the cavity surface unit is denoted as . No. The directed illumination response vector of each cavity surface element is expressed as:

[0217]

[0218] in, Indicates the first The directed illumination response vector of each cavity surface unit; Indicates the first directed response component; Indicates the first pair of opposite directional lighting directions in the... The unit projection direction on the tangent plane of the cavity surface unit; Indicates the second directed response component; Indicates the second pair of opposite directional lighting directions in the first The unit projection direction on the tangent plane of the cavity surface unit.

[0219] The image processing device extracts the zero-value boundaries of the transition from the positive response region to the negative response region in two directed response component maps. When the response values ​​of adjacent pixels have different signs, the zero-value boundaries are obtained by linear interpolation. Establish zero-value boundary points at the locations.

[0220] When two sets of zero-value boundaries have a unique intersection point, the corresponding intersection point is determined as the local response center. When two sets of zero-value boundaries have multiple intersection points, the sum of the norms of the directed lighting response vectors in the preset neighborhood around each intersection point is calculated, and the intersection point with the largest sum of norms is determined as the local response center. When two sets of zero-value boundaries do not intersect, the geometric center of the pixel position where the response amplitude reaches the lighting defect judgment threshold is determined according to the response amplitude weighting.

[0221] The response amplitude weighted geometric center is represented as:

[0222]

[0223] in, Indicates the first Local response center of each cavity surface unit; Indicates the first The set of pixel locations in a cavity surface unit whose response amplitude reaches the illumination defect determination threshold; Represents a set Pixel position index in; Indicates the first The directed illumination response vector at each pixel location; Indicates the first The norm of the directed illumination response vector at each pixel location; Indicates the first The coordinates of the cavity surface corresponding to each pixel.

[0224] No. The illumination response direction discriminant of each cavity surface unit is expressed as:

[0225]

[0226] in, Indicates the first The illumination response direction discrimination of each cavity surface unit; Indicates the local response center; Indicates the first Each cavity surface unit has an area-weighted center at the location where the illumination response vector acts; Indicates the first The directed illumination response vector of each cavity surface unit; symbol This represents the dot product of vectors.

[0227] When the direction discrimination value of the lighting response is greater than the lower limit of direction discrimination, the signed lighting response evidence is assigned an inward sign; when the direction discrimination value of the lighting response is less than the opposite of the lower limit of direction discrimination, the signed lighting response evidence is assigned an outward sign; when the absolute value of the direction discrimination value of the lighting response does not exceed the lower limit of direction discrimination, the evidence status is marked as an undetermined direction state.

[0228] No. The lower limit for determining the orientation of a cavity surface unit is expressed as follows:

[0229]

[0230] in, Indicates the first Lower limit for orientation discrimination of individual cavity surface units; Indicating the first defect-free standard mold The median of the absolute value of the direction discriminant of the illumination response of each cavity surface unit; This represents the median absolute deviation of the absolute value of the corresponding lighting response direction discrimination quantity; a constant. It represents the statistical dispersion coefficient.

[0231] Signed geometric deviation evidence is generated based on the directed distance between spatial points on the cavity surface and the digital reference model of the cavity. For the first... For each cavity surface element, project all effective cavity surface spatial points onto the nearest reference triangular facet, and calculate the directed distance of each cavity surface spatial point relative to the corresponding reference surface point along the reference surface normal. Then, use the median of the directed distances as the first... The directional geometric deviation of each cavity surface unit.

[0232] No. The directed geometric deviation of each cavity surface unit is expressed as:

[0233]

[0234] in, Indicates the first Directed geometric deviation of each cavity surface unit; This represents median operations; Indicates the first The set of effective cavity surface spatial points in each cavity surface unit; Represents a set Spatial point index; Indicates the first One measured spatial point on the cavity surface; Indicates the relationship with the first The reference surface point corresponding to each measured cavity surface spatial point; Represents the reference surface normal corresponding to the reference surface point; symbol This represents the dot product of vectors.

[0235] When the directed geometric deviation is greater than When the corresponding signed geometric deviation evidence is outward sign; when the directed geometric deviation is less than At that time, the corresponding evidence of signed geometric deviation is inward sign.

[0236] No. The threshold for determining geometric defects in a cavity surface unit is expressed as follows:

[0237]

[0238] in, Indicates the first The threshold for determining geometric defects in each cavity surface unit; This indicates the operation of finding the maximum value. Indicates the first Absolute value of design tolerance for each cavity surface unit; Indicating the first defect-free standard mold The median of the absolute values ​​of the directional geometric deviations of each cavity surface unit; Indicates the geometric threshold coefficient; This represents the median absolute deviation corresponding to the absolute value of the directed geometric deviation.

[0239] The geometric threshold coefficient can be between 2.5 and 4.0. In one embodiment, the geometric threshold coefficient is 3.0.

[0240] No. The threshold for determining illumination defects in each cavity surface unit is expressed as follows:

[0241]

[0242] in, Indicates the first The threshold for determining illumination defects in each cavity surface unit; Indicating the first defect-free standard mold The median of the norm of the directed illumination response vector of each cavity surface element; Indicates the lighting threshold coefficient; This represents the median absolute deviation of the norm of the corresponding directed lighting response vector.

[0243] The lighting threshold coefficient can be between 3.0 and 4.5. In one embodiment, the lighting threshold coefficient is 3.5.

[0244] When the norm of the directed lighting response vector reaches the lighting defect determination threshold, the absolute value of the directed geometric deviation reaches the geometric defect determination threshold, and the signs of the signed lighting response evidence and the signed geometric deviation evidence are consistent, a state of dual evidence in the same direction is formed.

[0245] When the norm of the directed lighting response vector reaches the lighting defect threshold and the absolute value of the directed geometric deviation reaches the geometric defect threshold, but the signs of the signed lighting response evidence and the signed geometric deviation evidence are opposite, a state of directional conflict is formed.

[0246] When one type of evidence reaches the corresponding defect judgment threshold, and another type of evidence falls into the preset benchmark evidence interval, a single evidence state is formed. The geometric benchmark evidence interval is defined as the absolute value of the directed geometric deviation not exceeding 0.4 times the geometric defect judgment threshold, and the lighting benchmark evidence interval is defined as the norm of the directed lighting response vector not exceeding 0.4 times the lighting defect judgment threshold.

[0247] When one type of evidence reaches the corresponding defect judgment threshold, while another type of evidence is between the benchmark evidence range and the defective evidence range, the corresponding evidence status is determined to be pending confirmation.

[0248] Both the "direction undetermined" and "confirmation pending" states indicate that the current evidence is insufficient to reliably determine the defect direction. The image processing device combines the evidence nodes corresponding to the "direction undetermined" or "confirmation pending" states with the evidence nodes of adjacent cavity surface units to form a local evidence sub-image, and determines the supplementary detection viewpoint and supplementary directional illumination direction according to the supplementary acquisition rules corresponding to the single-viewpoint support topology.

[0249] The image processing device sets the evidence records formed under each detection viewpoint as evidence nodes. Each evidence node records at least the evidence node identifier, cavity surface coordinate range, cavity surface unit identifier, evidence source, evidence symbol, evidence status, detection viewpoint identifier, visible pixel ratio, occlusion relationship, registration error, directed illumination response vector, and directed geometric deviation.

[0250] The data structure of the evidence nodes is shown in Table 5.

[0251] Table 5. Evidence Node Data Structure Table

[0252] Evidence node identifier Unique node number Cavity surface unit marking Corresponding cavity surface unit number Cavity surface coordinate range Three-dimensional area covered by evidence nodes Source of evidence Lighting, geometry, or dual evidence Evidence symbols Introverted, extroverted, or undetermined Evidence status Dual evidence in the same direction, conflicting directions, single evidence, or pending confirmation Detection viewpoint markers Evidence Source Testing Perspective Visible pixel ratio Effective pixel coverage ratio occlusion relationship occlusion unit and occlusion boundary Registration error Registration error of the current detection viewpoint Response amplitude Directed lighting response vector norm Geometric deviation Directed geometric deviation

[0253] When the cavity surface coordinate ranges corresponding to two evidence nodes are adjacent or overlap, the evidence symbols are consistent, and the shortest distance between the visible pixel boundaries of the two evidence nodes in the cavity surface coordinate domain is less than or equal to At that time, a support edge is established between the two evidence nodes.

[0254] For regions with significant differences in the size of cavity surface units, the minimum distance threshold at the boundary can be set to 20% of the smaller of the equivalent side lengths of the two cavity surface units.

[0255] When the coordinate ranges of the cavity surface corresponding to two evidence nodes overlap and the evidence signs are opposite, a conflict edge is established between the two evidence nodes.

[0256] A set of evidence nodes connected by supporting edges and containing dual-evidence states, single-evidence states, or pending confirmation states in the same direction is identified as a defect candidate region. A set containing only evidence nodes that match the process structure baseline evidence template is not considered a general defect candidate region.

[0257] No. The evidence node and the first The overlap ratio of cavity surface coordinates between each evidence node is expressed as follows:

[0258]

[0259] in, Indicates the first The evidence node and the first The overlap ratio of cavity surface coordinates between evidence nodes; and These represent the indices of the two evidence nodes; This represents the intersection area of ​​the cavity surface regions corresponding to the two evidence nodes; This indicates the minimum value operation; Indicates the first The area of ​​the cavity surface region corresponding to each evidence node; Indicates the first The area of ​​the cavity surface region corresponding to each evidence node.

[0260] The support weights of the supporting edges are represented as follows:

[0261]

[0262] in, Indicates the first The evidence node and the first Support weights of the edges between each evidence node; Indicates the overlap ratio weighting coefficient; Indicates the overlap ratio of the cavity surface coordinates; This represents the visible pixel ratio weighting coefficient; This represents the smaller of the visible pixel ratios of the two evidence nodes; This represents the weighting coefficient of the angle between the observation directions; This represents the angle factor of the observation direction.

[0263] The sum of the overlap ratio weighting coefficient, the visible pixel ratio weighting coefficient, and the viewing direction angle weighting coefficient is: In one implementation, the three weighting coefficients are set to 0.45, 0.30, and 0.25, respectively.

[0264] The angle factor of the observation direction is expressed as:

[0265]

[0266] in, Indicates the first The evidence node and the first Angle factor between observation directions of each evidence node; Represents the sine function; This represents the angle between the observation directions of the corresponding detection viewpoints of the two evidence nodes, and is calculated in radians.

[0267] Geometric deviation amplitude similarity is expressed as:

[0268]

[0269] in, Indicates the first The evidence node and the first Geometric deviation magnitude similarity between evidence nodes; This indicates the operation of finding the maximum value. Indicates the first The directed geometric deviation corresponding to each evidence node; Indicates the first The directed geometric deviation corresponding to each evidence node; This indicates that the denominator is to prevent the denominator from being... The lower limit of geometric deviation.

[0270] The lower limit of geometric deviation can be taken as .

[0271] The similarity of lighting response amplitudes is expressed as:

[0272]

[0273] in, Indicates the first The evidence node and the first Similarity of lighting response amplitude among evidence nodes; This indicates the operation of finding the maximum value. Indicates the first The directed lighting response vector corresponding to each evidence node; Indicates the first The directed lighting response vector corresponding to each evidence node; and Let represent the norms of the two directed illumination response vectors, respectively. This indicates that the denominator is to prevent the denominator from being... The lower limit of the lighting response.

[0274] The lower limit of the lighting response can be set to 0.001.

[0275] The conflict weight of a conflicting edge is represented as:

[0276]

[0277] in, Indicates the first The evidence node and the first The conflict weight of conflict edges between evidence nodes; Indicates the overlap weighting coefficient of the conflict area; Indicates the overlap ratio of the cavity surface coordinates; This represents the geometric deviation similarity weighting coefficient; Indicates the similarity of geometric deviation magnitudes; This represents the weighting coefficient for the similarity of lighting responses; This indicates the similarity of the lighting response amplitude.

[0278] The sum of the overlap weighting coefficient of the conflict area, the geometric deviation similarity weighting coefficient, and the lighting response similarity weighting coefficient is: In one implementation, the three weighting coefficients are set to 0.50, 0.25, and 0.25, respectively.

[0279] Image processing equipment uses breadth-first or depth-first traversal to determine the connected components in the defect evidence graph. Connected components of evidence nodes provided only by the same inspection viewpoint, or connected components of evidence nodes provided only by multiple inspection viewpoints with viewing direction angles of 0° or 180°, are identified as single-viewpoint supporting topology.

[0280] In digital calculations, an angle of less than or equal to 1° is considered 0°, and an angle of greater than or equal to 179° is considered 180°.

[0281] When both inward and outward evidence nodes exist simultaneously within the same cavity surface coordinate range, and conflict edges are established between the corresponding evidence nodes, it is identified as an opposite-sign conflict topology.

[0282] Two sets of evidence nodes with identical evidence symbols and adjacent cavity surface coordinate boundaries, but separated by an occlusion relationship, are identified as occlusion fracture topologies. When identifying occlusion fracture topologies, it is also required that there be a continuous cavity surface path between the two sets of evidence nodes on the cavity digital reference model, in order to exclude two independent cavity surface regions from being identified as occlusion fractures.

[0283] For isolated evidence nodes whose corresponding area is less than 20% of the average area of ​​the cavity surface unit, whose visible pixel ratio is less than 15%, and which have not established supporting edges or conflicting edges with other evidence nodes, the corresponding evidence nodes are marked as low-confidence isolated nodes.

[0284] Low-confidence isolated nodes are not directly output as defects. The image processing device combines the low-confidence isolated nodes with the evidence nodes of adjacent cavity surface units to form a local evidence sub-graph, and determines the supplementary detection viewpoint and supplementary directional illumination direction according to the supplementary acquisition rules corresponding to the single-view support topology.

[0285] Starting from an evidence node in the anomalous topology, the process expands outward along supporting and conflicting edges until it reaches an evidence node with consistent evidence symbols and supported by at least two observation viewpoints with an angle greater than 0° and less than 180° between their observation directions. The evidence nodes traversed and the evidence nodes reached during the expansion together constitute a local evidence subgraph.

[0286] After identifying the anomalous topology, the image processing device generates supplementary acquisition combinations within the reachable pose space of the pose adjustment mechanism. Candidate supplementary detection viewpoints satisfy constraints related to joint limits, anti-collision distance, image sensor working distance, imaging incident angle, and expected visible scale. The minimum safe distance between the rigid calibration acquisition head and the lost foam die to be detected can be set to... .

[0287] Each supplementary acquisition combination includes one candidate supplementary detection viewpoint and at least two pairs of candidate supplementary directional illumination directions. The supplementary acquisition combination covers evidence nodes in the anomalous topology and forms an image overlap region of not less than 20% with the cavity surface region corresponding to adjacent evidence nodes that are consistent with at least one evidence symbol and have been supported by two valid detection viewpoints.

[0288] No. The comprehensive evaluation value of the supplementary collection combination is expressed as:

[0289]

[0290] in, Indicates the first The comprehensive evaluation value of each supplementary collection combination; Indicates the candidate supplementary collection combination index; The evaluation weight represents the proportion of newly added visible cavity surface units; Indicates the proportion of newly added visible cavity surface units; This indicates the evaluation weight of the angle between the newly added observation directions; This indicates the normalized value of the angle between the newly added observation directions; The evaluation weights represent the ability to distinguish between expected symbols; This indicates the ability to distinguish between expected symbols; Evaluation weights representing the coverage ratio of local evidence sub-maps; Indicates the coverage ratio of a local evidence submap; The evaluation weights represent the normalized cost of motion; This represents the cost of normalization.

[0291] The normalized value of the newly added observation direction angle is expressed as follows:

[0292]

[0293] in, Indicates the first Normalized values ​​of the included angle of the new observation direction for each supplementary acquisition combination; Represents the sine function; Indicates the first The angle between the observation direction of each candidate supplementary detection viewpoint and the detection viewpoint whose observation direction is closest among the existing detection viewpoints is calculated in radians.

[0294] Projecting the two pairs of candidate supplementary directional illumination directions onto the tangent plane of the cavity surface element corresponding to the anomalous topology yields two unit vectors of the projection directions. and .

[0295] When the norm of the average directed lighting response vector corresponding to the anomalous topology is greater than the lower bound for direction discrimination, the first... The main response direction corresponding to each candidate supplementary acquisition combination is represented as follows:

[0296]

[0297] in, Indicates the first The main response direction corresponding to each candidate supplementary acquisition combination; Indicates the first The average value of the directed illumination response vectors corresponding to the anomalous topological evidence nodes covered by the candidate supplementary detection viewpoints; This represents the norm of the average directed lighting response vector.

[0298] When the norm of the average directed illumination response vector is not greater than the lower limit of direction discrimination, the direction of the maximum principal curvature of the cavity surface element corresponding to the abnormal topology is determined as the principal response direction.

[0299] The ability to distinguish between expected symbols is represented as:

[0300]

[0301] in, Indicates the first The expected symbol discrimination capability of each supplementary collection combination; This indicates the unit projection direction of the first pair of candidate supplementary directional lighting directions onto the corresponding tangent plane; This indicates the unit projection direction of the second pair of candidate supplementary directional lighting directions onto the corresponding tangent plane; Indicates the direction of the main response; symbol Dot product of vectors; symbol It represents the absolute value of a scalar.

[0302] The local evidence sub-image coverage ratio is the ratio of the cavity surface area corresponding to the local evidence sub-image visible from the candidate supplementary detection viewpoint to the total cavity surface area of ​​the local evidence sub-images. The normalized motion cost is the ratio of the joint spatial trajectory length from the current detection pose to the candidate supplementary detection viewpoint to the maximum joint spatial trajectory length among all candidate supplementary detection viewpoints.

[0303] In one implementation, the evaluation weight for the proportion of newly added visible cavity surface units is 0.25, the evaluation weight for the angle of newly added observation direction is 0.20, the evaluation weight for the expected symbol discrimination ability is 0.25, the evaluation weight for the coverage proportion of local evidence sub-graphs is 0.20, and the evaluation weight for the normalized motion cost is 0.10.

[0304] Candidate supplementary data collection combinations are ranked from largest to smallest based on their comprehensive evaluation values, and the feasible supplementary data collection combination with the largest comprehensive evaluation value is selected for supplementary data collection.

[0305] Different supplementary acquisition constraints are adopted for different abnormal topologies, as shown in Table 6.

[0306] Table 6. Abnormal Topology Supplementary Collection Strategy

[0307] Abnormal topology type Supplemental detection viewpoint constraints Supplemental directional lighting direction constraints Single-view support topology It forms an observation angle of 30° to 120° with the existing detection viewpoint, and covers anomalous nodes and adjacent stable nodes. The projection directions are located on both sides of the direction of maximum principal curvature. Conflict topology The observation directions of all conflict detection viewpoints are not collinear. The two pairs of projection directions are parallel and perpendicular to the principal response directions, respectively. Shielding fracture topology The line of sight simultaneously covers the obstruction interval and the evidence nodes on both sides of the obstruction interval. The projection direction intersects with the boundary of the occlusion interval.

[0308] For single-viewpoint support topologies, supplementary detection viewpoints are selected that form an angle of 45° to 90° with the existing detection viewpoints in the viewing direction. For conflicting topologies, directional illumination parallel to the main response direction is used to highlight the bright and dark responses in the corresponding direction, and directional illumination perpendicular to the main response direction is used to verify whether the corresponding response changes with the illumination direction. For occlusion-fracture topologies, the line of sight of the candidate supplementary detection viewpoints passes through the occlusion interval and covers the evidence nodes on both sides of the occlusion interval.

[0309] After the supplementary images are acquired, they are mapped to the cavity surface coordinate domain in the same way as the initial images, generating supplementary evidence nodes, supplementary support edges, and supplementary conflict edges. Then, a graph traversal is performed starting from the supplementary evidence nodes and the original evidence nodes in the anomaly topology.

[0310] Graph traversal only propagates along newly added supporting edges, newly added conflicting edges, supporting edges with a change in supporting weight greater than 0.05, conflicting edges with a change in conflicting weight greater than 0.05, and evidence nodes whose evidence symbols have changed.

[0311] Evidence nodes that are first reached during the traversal, supported by at least two observation viewpoints with an angle greater than 0° and less than 180° between their observation directions, and whose evidence symbols are consistent, are identified as stable boundary nodes.

[0312] The set of evidence nodes in the anomalous topology is denoted as the starting node set, and the set of stable boundary nodes is denoted as the ending node set. Starting from any evidence node in the starting node set, all evidence nodes that can be reached along supporting or conflicting edges and passed before reaching the ending node set, together with supplementary evidence nodes, constitute the influence closure.

[0313] After reaching a stable boundary node during graph traversal, the stable boundary node is recorded, and no further expansion beyond the stable boundary node is performed.

[0314] The image processing device only recalculates the state of evidence nodes, supporting edges, conflicting edges, and connected components within the affected closure, and does not recalculate the defect evidence graph outside the affected closure.

[0315] When supplementary evidence causes the original connected component to split, two connected components to merge, the evidence symbol inside the stable boundary node to change, or a conflict edge to be added, the influence closure is expanded and the update continues.

[0316] The convergence state of the local evidence subgraph is determined based on the results of two consecutive incremental updates. When two consecutive incremental updates have the same set of connected component node identifiers, the same set of evidence symbols, and the same set of stable boundary node identifiers, and each defect candidate region is supported by at least two detection viewpoints with an angle between the observation directions greater than 0° and less than 180°, the corresponding local evidence subgraph is determined to have reached a convergence state.

[0317] The maximum number of supplementary acquisitions for each local evidence subgraph is set to 5. If convergence is not achieved even after reaching the maximum number of supplementary acquisitions, the corresponding local evidence subgraph is marked as an acquisition anomaly, automatic supplementary acquisition is stopped, and the abnormal topology type, the detection viewpoints that have been executed, the directional illumination directions that have been executed, the conflict edges that have not been eliminated, and the invisible cavity surface units are output.

[0318] For a local evidence sub-image that has reached convergence, surface texture features, illumination response features, and geometric deviation features are extracted from the associated cavity surface image and the corresponding cavity surface spatial points.

[0319] Surface texture features include gray-level mean, gray-level standard deviation, local binary mode features, and gray-level co-occurrence matrix features. Illumination response features include the first directed response component, the second directed response component, the directed illumination response vector norm, the response sign, the distance between local response centers, and the zero-value boundary density. Geometric deviation features include directed geometric deviation, local average deviation, maximum deviation, local curvature variation, and point cloud fitting residuals.

[0320] Each evidence node constructs a 64-dimensional node input feature vector, of which surface texture features account for 24 dimensions, illumination response features account for 16 dimensions, geometric deviation features account for 16 dimensions, and node attributes corresponding to the number of detection viewpoints, visible pixel ratio, evidence status, evidence symbols, cavity process structure labels, and registration residuals account for 8 dimensions.

[0321] No. The node input feature vector of each evidence node is represented as follows:

[0322]

[0323] in, Indicates the first The initial node input feature vector of each evidence node; Indicates the evidence node index; Indicates the first Surface texture feature vectors of each evidence node; Indicates the first The illumination response feature vector of each evidence node; Indicates the first Geometric deviation feature vectors of each evidence node; Indicates the first The node attribute vector of each evidence node; superscript This represents the initial layer before graph convolution processing.

[0324] No. The node gating weights of each evidence node are represented as follows:

[0325]

[0326] in, Indicates the first The node gating weight of each evidence node; Represents the natural exponential function; Indicates the node gating bias parameter; This represents the gating parameter corresponding to the number of valid detection viewpoints; Indicates the first Normalized value of the number of valid detection viewpoints for each evidence node; This represents the gate parameter corresponding to the proportion of visible pixels; Indicates the first The proportion of visible pixels for each evidence node; This indicates the gating parameters corresponding to the supported weights; Indicates the relationship with the first The average weight of the supporting edges connected to each evidence node; This represents the gating parameter corresponding to the conflict weight; Indicates the relationship with the first The average conflict weight of the conflict edges connected to each evidence node.

[0327] The node gating bias parameters and each gating parameter are all determined through training the image recognition model.

[0328] The edge propagation coefficient of supporting edges is determined based on the support weight normalization, while the edge propagation coefficient of conflicting edges is set to... :

[0329]

[0330] in, Indicates the first The evidence node points to the first The edge propagation coefficient when a piece of evidence node transmits node features. Indicates the index of adjacent evidence nodes; Evidence node index representing the characteristics of the receiving node; Indicates the first The evidence node and the first Support weights of the edges between each evidence node; Indicates the relationship with the first A set of evidence nodes connected by supporting edges; Represents a set The evidence node index in the middle; Indicates the first The evidence node and the first The support weights of the support edges between each evidence node.

[0331] When set When empty, the first Each evidence node does not receive features transmitted by neighboring evidence nodes and only retains its own node's feature update items.

[0332] The image recognition model employs a two-layer gated graph convolutional network. Layer to the first The node feature update method of the layer is represented as follows:

[0333]

[0334] in, Indicates the first The evidence node at the _ Layer node feature vectors; Represents the linear rectification activation function; Indicates the first Layer's self-node feature transformation matrix; Indicates the first The node gating weight of each evidence node; Indicates the first The evidence node at the _ Layer node feature vectors; Indicates the relationship with the first A set of evidence nodes connected by a single evidence node; Represents a set The evidence node index in the middle; Indicates the first The evidence node points to the first The edge propagation coefficient when a piece of evidence node transmits node features. Indicates the first The feature transformation matrix of adjacent nodes in the layer; Indicates the first The node gating weight of each evidence node; Indicates the first The evidence node at the _ Layer node feature vectors; Indicates the first The layer's feature update bias vector; This represents the graph convolutional layer index.

[0335] After completing two layers of gated graph convolution, max pooling and average pooling are performed on the node features in the same connected component, respectively. The two pooling results are then concatenated and input into the defect region segmentation head and the defect category recognition head, respectively.

[0336] The defect region segmentation head outputs the probability that each cavity surface unit belongs to a defect region based on the fused node features, and generates a defect region mask based on the cavity surface coordinate connectivity. The defect category identification head outputs the probability of different defect categories based on the fused connected component features.

[0337] Defect categories include dent defects, protrusion defects, linear damage defects, attachment defects, and boundary defects.

[0338] The training samples for the image recognition model consist of actual detection samples, standard test block samples, and synthetic samples. Actual detection samples are provided with defect area and defect category labels based on coordinate measuring machine (CMM) measurements or manual verification. Standard test block samples are obtained from standard test blocks with known depressions, protrusions, linear damage, attachments, or boundary defects. Synthetic samples are generated by adding depressions or protrusions of known depth, width, and orientation to the cavity digital reference model and rendering the image using an illumination direction corresponding to the directional illumination array.

[0339] The manual verification results are annotated by the verifiers based on the original images, structured light 3D data, cavity surface coordinates, and measurement results. When the manual verification results are inconsistent with the coordinate measuring machine (CMM) measurement results, the CMM measurement results shall be used as the basis for determining the geometric defect areas and geometric deviation labels.

[0340] The training samples were divided into training, validation, and test sets in a 7:2:1 ratio. Each training batch contained 16 local evidence subgraphs, with an initial learning rate of 0.001 and 150 training epochs. Training stopped when the validation set loss failed to decrease for 15 consecutive epochs.

[0341] The image recognition model updates node gating parameters, node feature transformation matrix, bias vector, defect region segmentation head parameters, and defect category recognition head parameters through backpropagation, and determines the stopping point for training based on the validation set loss.

[0342] The total loss of the image recognition model is expressed as:

[0343]

[0344] in, This represents the total loss of the image recognition model; Indicates the weight of the loss for defect region segmentation; This represents the defect region segmentation loss, which aims to optimize the prediction of the difference between the defect region and its label. This indicates the loss weight for defect category identification; This represents the defect category identification loss, which optimizes the difference between the predicted defect category probability and the defect category label. Indicates the weight of the loss of evidence consistency; This indicates a loss of consistency in evidence.

[0345] In one implementation, the defect region segmentation loss weight is set to 0.50, the defect category identification loss weight is set to 0.35, and the evidence consistency loss weight is set to 0.15.

[0346] Loss of consistency of evidence is expressed as:

[0347]

[0348] in, This indicates a loss of consistency in the evidence; Indicates the set of supporting edges; This indicates the number of supporting edges in the supporting edge set; and These represent the evidence node indices at both ends of an edge; Indicates the first The node features of each evidence node after graph convolution; Indicates the first The node features of each evidence node after graph convolution; This represents the L2 distance between the features of two nodes; Represents the set of conflicting edges; Indicates the number of conflicting edges in the set of conflicting edges; This indicates the operation of finding the maximum value. This indicates the feature interval of conflicting nodes.

[0349] In one implementation, the feature interval of conflicting nodes is set to 1.0. When the support edge set is empty, the support edge loss term in the evidence consistency loss is set to... When the set of conflicting edges is empty, the conflicting edge loss term in the evidence consistency loss is set to... .

[0350] The image recognition model outputs a defect region mask, defect category confidence score, and the corresponding cavity surface coordinates of the defect region. Pixels in the defect region mask are mapped back to the cavity surface coordinate domain, and the three-dimensional boundary of the defect region is generated based on the corresponding set of triangular facets.

[0351] The output data for the defect area should include at least the defect number, defect category, category confidence score, defect center coordinates, defect area, maximum directed geometric deviation, evidence node set, detection viewpoint set, and convergence status. The category confidence score is recorded as a field in the detection results and is used to sort the order of manual review for different candidate defect areas.

[0352] After image recognition is completed, evidence nodes that split the corresponding connected components after removal, evidence nodes with the largest signed geometric deviation, and evidence nodes from each supporting detection viewpoint are extracted from the local evidence subgraph to form an evidence reproduction node set. The evidence nodes that split the corresponding connected components after removal are determined using a keypoint recognition algorithm.

[0353] A re-inspection acquisition sequence is generated based on the detection viewpoint and directional lighting direction recorded in the evidence reproduction node set. The original detection viewpoint and original directional lighting direction are prioritized for re-inspection. If the original detection viewpoint is unreachable, an alternative detection viewpoint with an angle of less than 10° to the original detection viewpoint's observation direction and the closest coverage ratio is selected.

[0354] The re-inspection image is processed using the same steps as the original inspection image to generate a re-inspection evidence sub-image. A re-inspection evidence sub-image is considered valid when it shares the same set of connected component nodes and evidence symbols as the original local evidence sub-image, and the Hausdorff distance between the cavity surface coordinate boundaries of the two defect regions is less than or equal to... If the condition is met, the corresponding defect area will be marked as consistent with the re-inspection; otherwise, the corresponding defect area will be marked as abnormal with the re-inspection.

[0355] For abnormal re-inspection conditions, the image processing equipment saves the original image, supplementary image, re-inspection image, actual pose of each inspection viewpoint, defect evidence image change record and image recognition model output results, and outputs manual review task.

[0356] The following shows the calculation process for signed evidence. Choose an area of... The cavity surface unit has a design dimensional tolerance absolute value of The median absolute value of the directed geometric deviation obtained from 20 defect-free standard molds is: The median absolute deviation is The median norm of the directed lighting response vector is 0.08, and the median absolute deviation is 0.04. The geometric threshold coefficient is set to 3.0, and the lighting threshold coefficient is set to 3.5.

[0357] The corresponding thresholds and detection data are shown in Table 7.

[0358] Table 7. Calculation Process of Signed Evidence

[0359] Absolute value of design dimensional tolerance As the lower limit of the geometric threshold Median of absolute geometric deviation Defect-free standard mold statistics Geometric deviation, median absolute deviation Defect-free standard mold statistics Geometric defect determination threshold Median of lighting response amplitude 0.08 Defect-free standard mold statistics Median absolute deviation of lighting response 0.04 Defect-free standard mold statistics Lighting defect judgment threshold 0.22 Measured Directed Geometric Deviation Evidence of inward geometric deviation Measured Lighting Response Vector Norm 0.37 Reaching the threshold for judging lighting defects Lighting response direction discrimination Greater than the lower limit of direction discrimination Evidence of inward lighting response Evidence status Same direction dual evidence status Both types of evidence are introverted.

[0360] The incremental update process of the local evidence subgraph is recorded in Table 8.

[0361] Table 8. Incremental Update Process of Local Evidence Subgraphs

[0362] Initial detection 2 3 0 1 Not formed Single-view support topology After the first supplementary collection 1 7 1 2 It has been formed Continue to update After the second supplementary collection 1 9 0 3 remain unchanged Waiting for confirmation of consecutive results Third update confirmed 1 9 0 3 remain unchanged Reaching convergence

[0363] In the process shown in Table 8, the initial detection is supported by evidence from only one detection viewpoint, forming a single-viewpoint support topology. After the first supplementary acquisition, the two connected components merge, but conflicting edges still exist. After the second supplementary acquisition, the conflicting edges are eliminated, and three valid detection viewpoints are obtained. During the third update confirmation, the set of connected component node identifiers, the set of evidence symbols, and the set of stable boundary node identifiers all remain unchanged, thus confirming that the local evidence subgraph has reached a convergent state.

[0364] When local brightness and darkness anomalies appear in the deep cavity curved surface area of ​​a lost foam mold, the directional lighting image obtained from the initial inspection viewpoint may show an inward lighting response, but a single observation direction cannot rule out the appearance anomalies caused by surface highlights or occlusion boundaries.

[0365] The image processing device first maps the directional illumination image onto the corresponding cavity surface unit, and then registers the cavity surface spatial points recovered by structured light with the cavity digital reference model. The directed geometric deviation of the corresponding cavity surface unit is negative, and the absolute value of the directed geometric deviation reaches the geometric defect judgment threshold; the directed illumination response vector points to the local response center, and the norm of the directed illumination response vector reaches the illumination defect judgment threshold, thus forming an inward unidirectional dual evidence state.

[0366] When the initial evidence nodes originate from only one detection viewpoint, the defect evidence map forms a single-viewpoint support topology. The joint supplementary acquisition unit selects a supplementary detection viewpoint that is not collinear with the observation direction of the existing detection viewpoint, and selects supplementary directional illumination directions whose projection directions are located on both sides of the direction of maximum principal curvature.

[0367] After the supplementary image is mapped to the same cavity surface coordinate domain, the supplementary evidence node and the original evidence node have adjacent or overlapping cavity surface coordinate ranges and consistent inward signs, and support edges are established between the two evidence nodes. Graph traversal starts from the supplementary evidence node and the original anomalous evidence node, and propagates to stable boundary nodes supported by two non-collinear detection viewpoints and with consistent evidence signs, thereby determining the influence closure.

[0368] After obtaining the same set of connected component nodes, evidence symbols, and stable boundary nodes through two consecutive updates, the local evidence subgraph is determined to be in a convergent state. The image recognition model transmits node features based on supporting edges and blocks feature transmission corresponding to conflicting edges, ultimately outputting the concave defects, defect regions, and corresponding cavity surface coordinates.

[0369] After image recognition is completed, a re-inspection is performed based on the evidence reproduction node set. When the re-inspection evidence subgraph satisfies the conditions of the connected component node set, evidence symbol set, and cavity surface coordinate boundary difference, the defect area is marked as a re-inspection consistent state; if the re-inspection consistent conditions are not met, a manual review task is output.

[0370] This embodiment, through the above steps, can form a complete industrial inspection closed loop, from the acquisition of surface brightness and darkness anomalies, physical surface coordinate association, dual evidence verification of lighting and geometry, anomaly topology identification, joint planning of supplementary viewpoints and lighting, local incremental update, graph structure identification to re-inspection and confirmation.

[0371] Example 2

[0372] This embodiment provides an image recognition-based cavity defect detection system for lost foam casting molds. The detection system is used to perform the cavity defect detection method for lost foam casting molds described in Embodiment 1.

[0373] The detection system includes a rigid calibration acquisition head, a pose adjustment mechanism, a motion controller, an illumination controller, and an image processing device.

[0374] The rigid calibration acquisition head is mounted on the actuator of the pose adjustment mechanism and moves with the pose adjustment mechanism to the initial detection viewpoint or the supplementary detection viewpoint. The rigid calibration acquisition head includes an image sensor, a structured light projection component, and a directional illumination array, and the image sensor, structured light projection component, and directional illumination array maintain a fixed relative positional relationship.

[0375] The structured light projection assembly projects structured light onto the cavity of the lost foam casting mold to be inspected, and an image sensor acquires the structured light image. The directional illumination array includes at least four illumination units that can be independently turned on and off. The at least four illumination units form at least two pairs of opposite directional illumination directions, which are used to sequentially create different directional illumination conditions under the same inspection viewpoint, and the image sensor acquires the corresponding directional illumination images.

[0376] The posture adjustment mechanism can be a six-axis industrial robotic arm, or a multi-degree-of-freedom posture adjustment mechanism formed by combining linear motion mechanism and rotary motion mechanism.

[0377] The motion controller is connected to the pose adjustment mechanism and the image processing equipment. It is used to receive the target detection viewpoint sent by the image processing equipment, and control the pose adjustment mechanism to move the rigid calibration acquisition head to the corresponding detection viewpoint. At the same time, it feeds back the actual position, actual posture and positioning status of the rigid calibration acquisition head to the image processing equipment.

[0378] The lighting controller is communicatively connected to both the directional lighting array and the image processing equipment. It controls the activation sequence, lighting intensity, and duration of each lighting unit according to lighting control commands issued by the image processing equipment. The lighting controller also coordinates the time-sharing operation of the structured light projection component and the directional lighting array, separating the structured light image acquisition process from the directional lighting image acquisition process.

[0379] The image processing device is communicatively connected to the image sensor, motion controller, and lighting controller. The image processing device includes a processor and a memory. The memory stores program instructions, which, when executed by the processor, configure the following units: a model processing unit, an initial acquisition planning unit, a surface mapping unit, a signed evidence generation unit, an evidence map processing unit, a joint supplementary acquisition unit, and a convergence determination and recognition unit.

[0380] The model processing unit is used to acquire the cavity digital reference model of the lost foam mold to be inspected, divide the cavity digital reference model into multiple cavity surface units, and determine the reference surface normal and the direction of maximum principal curvature of each cavity surface unit.

[0381] The initial acquisition planning unit is used to determine the initial detection viewpoint and the paired opposite directional lighting directions based on the cavity digital reference model, cavity surface unit, reference surface normal, maximum principal curvature direction and occlusion boundary, and sends pose control commands and lighting control commands to the motion controller and lighting controller respectively.

[0382] The surface mapping unit is used to recover the spatial points of the cavity surface based on the structured light image, map the directional illumination image to the coordinate domain of the cavity surface, and establish the set of visible pixels and occlusion relationships of each cavity surface unit.

[0383] The signed evidence generation unit is used to generate signed illumination response evidence based on pixel responses under paired opposite directional illumination directions, and to calculate the directed distance of the cavity surface spatial point relative to the cavity digital reference model along the reference surface normal to generate signed geometric deviation evidence.

[0384] The evidence graph processing unit is used to set evidence records as evidence nodes, establish support edges between adjacent or overlapping evidence nodes with the same evidence symbols, establish conflict edges between evidence nodes with overlapping coordinate ranges on the cavity surface and opposite evidence symbols, and construct a defect evidence graph by combining the visible pixel set and occlusion relationship. From the defect evidence graph, it identifies single-viewpoint support topology, conflict topology with opposite signs, and occlusion break topology.

[0385] The joint supplementary acquisition unit is used to determine the supplementary detection viewpoint and supplementary directional lighting direction based on the identified abnormal topology, and sends corresponding pose control commands and lighting control commands to the motion controller and lighting controller, respectively.

[0386] The motion controller controls the pose adjustment mechanism to move the rigid calibration acquisition head to the supplementary detection viewpoint according to the pose control command, and the lighting controller controls the directional lighting array to form a supplementary directional lighting direction according to the lighting control command, and the image sensor acquires the supplementary image.

[0387] The joint supplementary acquisition unit is also used to generate supplementary evidence nodes, supplementary support edges, and supplementary conflict edges based on the supplementary images, so as to stabilize the boundary nodes to limit the influence closure, and to incrementally update the local evidence subgraph within the influence closure.

[0388] The convergence determination and identification unit is used to compare the set of connected component nodes, the set of evidence symbols, and the set of stable boundary nodes after two consecutive incremental updates. When the results of two consecutive incremental updates are consistent, and each defect candidate region is supported by at least two detection viewpoints with an angle between the observation directions greater than 0° and less than 180°, the corresponding local evidence subgraph is determined to have reached the convergence state.

[0389] The convergence determination and recognition unit further extracts node features from the local evidence sub-image and the associated cavity surface image that have reached the convergence state. It determines the node gating weight based on the number of detection viewpoints, the proportion of visible pixels, the support weight, and the conflict weight, so that the support edge can transmit node features and the conflict edge can block feature transmission. Based on the fused node features, it outputs the defect area, defect type, and the cavity surface coordinates corresponding to the defect area.

[0390] When the detection system is running, the image processing equipment completes the establishment of the detection task and the initial acquisition plan, the motion controller controls the rigid calibration acquisition head to reach the detection viewpoint, the illumination controller controls the structured light projection component and the directional illumination array to work according to the predetermined timing, and the image sensor sends the acquired images to the image processing equipment.

[0391] The image processing device sequentially completes cavity surface mapping, signed evidence generation, defect evidence map construction, abnormal topology identification, supplementary acquisition control, incremental update of local evidence sub-maps, and defect identification. The threshold determination, calculation method, anomaly handling, and convergence judgment for each data processing step adopt the corresponding methods described in Example 1.

[0392] Comparative Example 1

[0393] This comparative example uses the same image sensor, directional illumination array, and initial detection viewpoint as Example 1 to acquire cavity surface images, but does not recover the cavity surface spatial points based on the structured light images, nor does it uniformly map the directional illumination images under different detection viewpoints to the cavity surface coordinate domain.

[0394] This comparative example directly extracts the light and dark boundaries, textures, and grayscale differences from the two-dimensional image coordinates of each directional lighting image, and compares the abnormal regions in each image with the standard image.

[0395] Because the perspective projection relationships differ across different detection viewpoints, the pixel coordinates of the same cavity surface position in different images do not have a direct correspondence. Therefore, this comparative example cannot establish cross-viewpoint evidence nodes based on unified cavity surface coordinates, nor can it generate signed geometric deviation evidence along the reference surface normal.

[0396] The difference between this comparative example and Example 1 lies in whether a unified mapping of the cavity surface coordinate domain is implemented and whether signed geometric deviation evidence is generated. This comparative example is used to illustrate the role of unified physical surface coordinate correlation and signed dual evidence in distinguishing between geometric defects and lighting appearance anomalies, and does not record performance values ​​obtained without actual testing as experimental results.

[0397] Comparative Example 2

[0398] This comparative example uses the cavity surface coordinate domain, signed illumination response evidence, signed geometric deviation evidence, evidence nodes, support edges, conflict edges, and anomalous topology formed in Example 1, and acquires supplementary images according to the supplementary acquisition rules of Example 1.

[0399] After each supplementary image acquisition, instead of determining the stable boundary nodes and influence closures of the original scale, the system recalculates the state of all evidence nodes, all supporting edges, all conflicting edges, and all connected components in the complete defect evidence graph.

[0400] Example 1 performs incremental updates only within the influence closure defined by the anomalous topology evidence node, supplementary evidence node, and stable boundary node. The difference between this comparative example and Example 1 lies in whether or not stable boundary nodes are used to define the local update range.

[0401] This comparative example is used to illustrate the effect of incremental updates of closures on reducing redundant calculations in regions not affected by supplementary evidence. Processing time or computational load values ​​not obtained from actual operation statistics are not recorded as experimental results.

[0402] As can be seen from Examples 1, 2, Comparative Examples 1 and 2, the present invention uses the cavity digital reference model and the cavity surface coordinate domain as the basis for cross-viewpoint data association, uses signed illumination response evidence and signed geometric deviation evidence to form evidence states, identifies anomalous topology based on support edges, conflict edges and occlusion relationships, and drives the joint determination of supplementary detection viewpoints and supplementary directional illumination directions by the anomalous topology.

[0403] After supplementing the image to form new evidence, this invention limits the influence closure by stable boundary nodes, incrementally updates the local evidence subgraph only within the influence closure, and implements node gating and graph structure feature fusion after the multi-view evidence meets the continuous convergence condition, thereby forming a defect detection technology solution corresponding to the vanishing foam mold cavity surface reflection, deep cavity occlusion and multi-directional imaging conditions.

[0404] It should be noted that the embodiments of the present invention have better implementability and are not intended to limit the present invention in any way. Any person skilled in the art may use the above-disclosed technical content to change or modify it into equivalent effective embodiments. However, any modifications or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting defects in the cavity of a lost foam casting mold based on image recognition, characterized in that, include: S1. Obtain the digital reference model of the cavity of the lost foam mold and divide it into cavity surface units. Determine the reference surface normal, the direction of maximum principal curvature, the initial detection viewpoint, and the paired opposite directional lighting directions. Acquire structured light images and directional lighting images. S2. Recover the spatial points on the cavity surface based on the structured light image, map the directional illumination image to the coordinate domain of the cavity surface, and establish the set of visible pixels and occlusion relationships; S3. Generate signed illumination response evidence based on pixel responses under opposite directional illumination directions, calculate the directed distance of spatial points on the cavity surface relative to the digital reference model of the cavity along the reference surface normal, and generate signed geometric deviation evidence to form an evidence state. S4. Set the evidence records as evidence nodes, establish support edges or conflict edges according to coordinate relationships and evidence symbols, construct a defective evidence map by combining the set of visible pixels and occlusion relationships, identify single-viewpoint support topology, conflict topology with opposite signs and occlusion fracture topology, and determine the set of evidence nodes connected by support edges and having defective evidence states as defective candidate regions. S5. Based on the identified topology, determine the supplementary detection viewpoint and supplementary directional lighting direction, acquire supplementary images, limit the influence closure with stable boundary nodes, and incrementally update the local evidence subgraph within the closure until the connected component node set, evidence symbol set and stable boundary node set are the same in two consecutive updates, and each defect candidate region is supported by at least two detection viewpoints with an angle greater than 0° and less than 180° between the observation directions. S6. Extract node features from the converged local evidence sub-image and the associated cavity surface image. Determine the node gating weights based on the number of detection viewpoints, the proportion of visible pixels, the support weight, and the conflict weight. Enable the support edges to transmit node features and enable the conflict edges to block feature transmission. Determine the defect region, defect category, and cavity surface coordinates based on the fused features.

2. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 1, characterized in that, Determining the initial detection viewpoint and paired opposite directional illumination directions includes: Candidate acquisition combinations are generated based on the reference surface normal, the direction of maximum principal curvature, and the occlusion boundary of each cavity surface unit. Each candidate acquisition combination includes a candidate detection viewpoint and at least two pairs of opposite directional illumination directions. Determine the visible cavity surface unit, imaging resolution, and projection direction of each directional illumination direction onto the tangent plane of the corresponding cavity surface unit for each candidate acquisition combination; Select an initial acquisition combination from the candidate acquisition combinations, such that each cavity surface unit is covered by at least two initial detection viewpoints, the angle between the observation directions of at least two initial detection viewpoints is greater than 0° and less than 180°, and the projections of at least one pair of opposite directional illumination directions on the corresponding tangent plane are respectively located on both sides of the direction of maximum principal curvature.

3. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 2, characterized in that, Dividing the cavity digital reference model into cavity surface units and establishing a process structure reference evidence template includes: Curvature change boundary and cavity process structure label are extracted from the cavity digital reference model. Cavity surface unit with continuous reference surface normal is formed according to the curvature change boundary and cavity process structure label. The cavity process structure label includes at least one of parting boundary label, venting structure label, insert boundary label and demolding structure action area label. By combining the initial data collection and based on the repeated inspection results of the mold design documents or the defect-free standard mold, the expected lighting response symbol sequence and expected geometric deviation range corresponding to the process structure label of each cavity are determined, and the expected lighting response symbol sequence and expected geometric deviation range are associated as the process structure benchmark evidence template.

4. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 1, characterized in that, Mapping the directional illumination image to the cavity surface coordinate domain includes: The spatial points on the cavity surface recovered from the structured light image are registered with the digital reference model of the cavity to obtain the pose transformation relationship between each initial detection viewpoint and the digital reference model of the cavity. Based on the pose transformation relationship, the effective pixels in each directional lighting image are converted into line-of-sight rays, and the intersection points of each line-of-sight ray and the cavity digital reference model are determined. When a line of sight intersects with multiple cavity surface units, the cavity surface unit to which the intersection point has the smallest distance from the corresponding cavity surface spatial point and the angle between the reference surface normal and the direction pointing to the corresponding initial detection viewpoint is less than 90° is determined as the pixel belonging unit. The intersecting cavity surface units located after the intersection point along the propagation direction of the line of sight are determined as occlusion units, thus forming a set of visible pixels and occlusion relationships between cavity surface units.

5. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 4, characterized in that, Evidence for generating signed lighting responses includes: Exposure response normalization and pixel position registration are performed on each directional illumination image acquired under the same initial detection viewpoint; Two pairs of directional illumination directions with non-parallel projection directions on the tangent plane of the corresponding cavity surface unit are selected from the pairs of opposite directional illumination directions. The directional illumination images corresponding to each pair of directional illumination directions are then subjected to pixel-by-pixel difference to obtain the directional response components distributed along the two tangential directions. A directed lighting response vector is formed based on two directed response components and their corresponding tangential directions. The local response center is determined based on the zero-value boundary of the two directed response components. The directed lighting response vector pointing to the local response center is assigned an inward sign, and the directed lighting response vector deviating from the local response center is assigned an outward sign. Associating inward or outward signs, the direction of the directed lighting response vector, and the response amplitude with evidence of the signed lighting response.

6. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 5, characterized in that, The conditions for forming evidence include: When a spatial point on the cavity surface is located on the positive side of the normal of the reference surface relative to the corresponding reference surface point, the corresponding directed distance is assigned an outward sign; when a spatial point on the cavity surface is located on the negative side of the normal of the reference surface relative to the corresponding reference surface point, the corresponding directed distance is assigned an inward sign. When the signs of the signed lighting response evidence and the signed geometric deviation evidence are the same and both fall within the preset defect evidence range, a state of dual evidence in the same direction is formed. When the signs of the signed lighting response evidence and the signed geometric deviation evidence are opposite and both fall within the preset defect evidence range, a state of directional conflict is formed. When one type of evidence, either the signed lighting response evidence or the signed geometric deviation evidence, falls into a pre-defined defective evidence interval, and the other type of evidence falls into a pre-defined benchmark evidence interval, a single evidence state is formed.

7. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 1, characterized in that: The evidence node records the coordinate range of the cavity surface, the source of the evidence, the evidence symbol, the evidence status, the detection viewpoint, the proportion of visible pixels, and the occlusion relationship; It supports two evidence nodes whose edge-connected cavity surface coordinate ranges are adjacent or overlapping, whose evidence symbols are consistent, and whose boundary spacing of visible pixel sets meets the preset connectivity conditions. Two evidence nodes whose coordinate ranges overlap and whose evidence signs are opposite are connected by conflicting edges on the cavity surface. The support weights of the supporting edges are determined based on the overlap ratio of the cavity surface coordinates of the two evidence nodes, the proportion of visible pixels, and the angle between the observation directions of the corresponding detection viewpoints. The conflict weights of the conflict edges are determined based on the overlap ratio of the cavity surface coordinates, the similarity of the geometric deviation amplitude, and the similarity of the illumination response amplitude.

8. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 7, characterized in that, Identifying single-viewpoint supported topologies, conflicting topologies, and occlusion-fractured topologies includes: A connected component in which all evidence nodes originate from the same detection viewpoint, or a connected component in which evidence nodes originate from multiple detection viewpoints with an angle of 0° or 180° between their observation directions, is defined as a single-viewpoint support topology. Evidence nodes with inward and outward signs respectively within the same cavity surface coordinate range and connected by conflict edges are identified as opposite-sign conflict topologies; Two sets of evidence nodes with consistent evidence symbols, adjacent cavity surface coordinate boundaries, and separated by occlusion relationships are defined as the occlusion fracture topology. Starting from evidence nodes in a single-viewpoint support topology, conflicting topology, or occlusion-fracture topology, extend along support edges and conflict edges to evidence nodes with consistent evidence signs and supported by detection viewpoints with at least two viewing directions having an angle greater than 0° and less than 180°, forming a local evidence subgraph.

9. The method for detecting defects in the cavity of a lost foam casting mold based on image recognition according to claim 1, characterized in that, Determining the supplementary detection viewpoint and supplementary directional illumination direction includes: Within the reachable pose space and controllable illumination direction range of the detection device, supplementary acquisition combinations are generated. Each supplementary acquisition combination includes a candidate supplementary detection viewpoint and at least two pairs of candidate supplementary directional illumination directions. Select supplementary acquisition combinations that can cover the evidence nodes in the identified topology and form image overlaps with the cavity surface areas corresponding to adjacent evidence nodes that are consistent with the evidence symbols. Determine the newly added visible cavity surface units corresponding to each supplementary acquisition combination, the angle of the newly added observation direction relative to the existing detection viewpoint, the expected symbol distinction relationship corresponding to the opposite directional illumination direction, and the coverage of the local evidence sub-image; Based on the identified topology type, supplementary detection viewpoints and supplementary directional lighting directions are determined from the supplementary acquisition combinations.

10. A cavity defect detection system for lost foam casting molds based on image recognition, characterized in that, It includes a rigid calibration acquisition head, a pose adjustment mechanism, a motion controller, an illumination controller, and an image processing device; The rigid calibration acquisition head is set at the execution end of the pose adjustment mechanism. The rigid calibration acquisition head includes an image sensor, a structured light projection component, and a directional illumination array. The directional illumination array includes at least four independently controlled illumination units arranged around the image sensor, and the illumination units form at least two pairs of opposite directional illumination directions. The motion controller is connected to the posture adjustment mechanism, the lighting controller is connected to the directional lighting array, and the image processing device is communicatively connected to the image sensor, the motion controller, and the lighting controller, respectively. Image processing devices include a processor and a memory. The memory stores program instructions, which are configured and formed when the processor executes them. The model processing unit is used to obtain the digital reference model of the cavity of the lost foam mold, divide the digital reference model of the cavity into cavity surface units, establish the cavity surface coordinate domain, and determine the reference surface normal and the direction of maximum principal curvature of each cavity surface unit. The initial acquisition planning unit is used to determine the initial detection viewpoint and the paired opposite directional illumination directions based on the reference surface normal, the direction of maximum principal curvature and the occlusion boundary, and send pose control commands and illumination control commands to the motion controller and the illumination controller respectively. The surface mapping unit is used to acquire structured light images through an image sensor when the structured light projection component projects structured light, recover the spatial points of the cavity surface based on the structured light images, map the directional illumination images acquired by the image sensor to the coordinate domain of the cavity surface, and establish the set of visible pixels and occlusion relationships. The signed evidence generation unit is used to generate signed illumination response evidence based on the pixel response of the same cavity surface unit under opposite directional illumination directions, calculate the directed distance of the cavity surface spatial point relative to the cavity digital reference model along the corresponding reference surface normal and generate signed geometric deviation evidence to form an evidence state. The evidence graph processing unit is used to set the evidence records of each detection viewpoint as evidence nodes, establish support edges or conflict edges according to the coordinate relationship of the cavity surface and the evidence symbols, construct a defect evidence graph by combining the set of visible pixels and the occlusion relationship, identify single-viewpoint support topology, opposite-sign conflict topology and occlusion fracture topology, and determine the set of evidence nodes connected by support edges and having defect evidence status as defect candidate regions. The joint supplementary acquisition unit is used to determine the supplementary detection viewpoint and supplementary directional lighting direction based on the identified topology, send supplementary pose control commands to the motion controller to control the pose adjustment mechanism, send supplementary lighting control commands to the lighting controller to control the directional lighting array, acquire supplementary images through the image sensor, and limit the influence closure with stable boundary nodes, and incrementally update the local evidence subgraph within the influence closure; The convergence determination and identification unit is used to determine the local evidence subgraph as convergent when the set of connected component nodes, the set of evidence symbols, and the set of stable boundary nodes are the same in two consecutive incremental updates, and each defect candidate region is supported by at least two detection viewpoints with an angle greater than 0° and less than 180° between the observation directions. The unit extracts node features from the converged local evidence subgraph and the associated cavity surface image, determines the node gating weight based on the number of detection viewpoints, the proportion of visible pixels, the support weight, and the conflict weight, enables the supporting edges to transmit node features and enables the conflicting edges to block feature transmission, and determines the defect region, defect category, and cavity surface coordinates based on the fused features.

Images