Multi-lens visual inspection image processing method and system

Abstract

The application discloses a kind of multi-lens visual inspection image processing method and system, it is related to intelligent visual inspection technical field, the present application adopts multiple lenses in the same synchronous trigger period is completed multi-view collection, and is completed in one processing link, fusion and decoding output, avoid the multiple exposure sequence collection introduced for the sake of different material imaging, so that under the premise of not increasing the number of acquisition frames, multi-attribute synchronous determination is realized;In the unified imaging condition, the problem that the reflection and texture difference of composite material is significant, leading to shared features being dominated by single material, the present application aggregates the information of multiple observation directions into shared features through cross-view fusion, so that the reflection, shielding or low-contrast area can obtain supplementary clues from other views, reducing the overall impact of single-view imaging defects on defect detection and assembly evaluation.

Description

Technical Field

[0001] This invention relates to the field of intelligent visual inspection technology, and in particular to a multi-lens visual inspection image processing method and system. Background Technology

[0002] In modern industrial manufacturing such as consumer electronics and automotive parts, workpieces are often composed of composite materials such as plastics, metals, glass, and rubber. On the production line, it is usually necessary to quickly determine the appearance defects (such as scratches, dents, burrs, stains, etc.) and assembly relationships (such as mounting misalignment, gaps / coaxiality, seal positioning, etc.) of the workpieces at the same station in order to meet the full inspection requirements under the cycle time requirements.

[0003] In existing technologies, some solutions employ deep learning-based multi-task networks, which achieve joint prediction of defect category identification and assembly offset regression by sharing a main feature extraction backbone and setting multiple task branches. However, composite material workpieces often exhibit differences in reflectivity, texture, and specular highlights under the same lighting conditions, leading to insufficient contrast or glare in some areas, thus making the defect imaging features unstable. Furthermore, multi-task learning may experience gradient conflicts and negative transfer during shared parameter updates, resulting in a decrease in the detection rate of some defect categories or small / low-contrast defects.

[0004] Some solutions utilize programmable multi-channel light sources to achieve multi-angle / multi-spectral time-division imaging, optimizing imaging conditions for different materials or defects, and performing fusion analysis on multiple frames. These solutions typically require multiple exposures (e.g., multi-wavelength sequence acquisition) and are more sensitive to trigger synchronization, inter-frame registration, and misalignment / blurring caused by motion. In high-speed online inspection scenarios with extremely short cycle times or continuous workpiece movement, the additional acquisition and processing links may compress the effective detection window, affecting system stability.

[0005] In addition, although existing multi-camera / multi-lens synchronous inspection systems can obtain multi-view information, they often require the establishment of a unified coordinate system and high-precision calibration and uncertainty control in metrology-level assembly measurement, resulting in high system integration and maintenance costs.

[0006] Therefore, there is still a need for a visual inspection image processing method and system that can improve the synchronization, stability and integrity of defect detection and assembly measurement of composite material workpieces while meeting production cycle requirements. Summary of the Invention

[0007] In view of the aforementioned existing problems, the present invention is proposed.

[0008] This invention provides a multi-lens visual inspection image processing method and system to solve the problems of simultaneous defect and assembly inspection of composite material workpieces, easy mutual interference of single-light multi-task, and difficulty in meeting the cycle time of time-division imaging.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a multi-lens visual detection image processing method, comprising: Step S1: At least two images of the composite workpiece from different perspectives are acquired by at least two image acquisition devices within the same trigger cycle. The composite workpiece contains at least two different material components. Step S2: Perform viewpoint alignment or coordinate mapping on the images from different viewpoints based on preset camera calibration parameters; Step S3: Input the aligned multi-view images into the fusion feature extraction network to obtain a shared feature map; Step S4: Input the shared feature map into the region perception task modulation module, generate spatial weight maps for at least two types of preset detection tasks, and weight the shared feature map accordingly to obtain the corresponding task feature map. The preset detection tasks include defect detection tasks for components of various materials and assembly relationship evaluation tasks between components of different materials. Step S5: Input the feature maps of each task into the corresponding task decoding network, and output the defect detection results and assembly offset or assembly status results.

[0010] As a preferred embodiment of the multi-lens visual detection image processing method of the present invention, the fusion feature extraction network includes a shared encoder and a cross-view fusion module. The shared encoder extracts initial feature maps for each view image, and the cross-view fusion module performs information interaction and aggregation on each initial feature map based on an interactive attention mechanism to generate the shared feature map.

[0011] As a preferred embodiment of the multi-lens visual detection image processing method of the present invention, the interactive attention mechanism includes: generating query features from an initial feature map of any viewpoint, and generating key features and value features from an initial feature map of another viewpoint; calculating the correlation based on the query features and key features and obtaining attention weights; using the attention weights to weightedly aggregate the value features to obtain cross-viewpoint enhanced features, and fusing them with the initial features of the any viewpoint to update the viewpoint features, which are used to merge the updated features of each viewpoint to form the shared feature map; In the interactive attention mechanism, the correlation between query features and key features is scaled dot product and temperature control is introduced to adjust the sharpness of the attention distribution. The correlation is superimposed on the geometrically corresponding region mask obtained by viewpoint alignment or coordinate mapping and then exponentially normalized to obtain the attention weight. The attention weight is weighted and converged on the value features to obtain cross-viewpoint enhanced features, and residual fusion is performed with the initial features on the query side before layer normalization is performed to update the viewpoint features. The updated features of each viewpoint are backfilled into spatial feature maps and then mean fusion is performed to obtain a shared feature map. The mask is used to limit the normalization of the attention weight to the geometrically corresponding region.

[0012] As a preferred embodiment of the multi-lens visual detection image processing method of the present invention, the region perception task modulation module includes a weight prediction sub-network. The weight prediction sub-network takes a shared feature map as input and outputs a spatial weight map for each preset detection task and for each spatial location of the shared feature map. The weight values ​​of the spatial weight map are constrained within a preset numerical range.

[0013] As a preferred embodiment of the multi-lens visual detection image processing method of the present invention, the generation of task feature map includes: for the k-th preset detection task, the shared feature map and the spatial weight map corresponding to the task are weighted element by element to obtain the k-th task feature map, and the element-by-element weighting includes element-by-element multiplication or residual fusion with the shared feature map after element-by-element multiplication. During the task feature map generation process, the weight response output by the weight prediction sub-network is constrained to form a spatial weight map. The spatial weight map is multiplied element-wise with the shared feature map to obtain the task feature map, or the task feature map is obtained by residual fusion with the shared feature map after element-wise multiplication, so as to retain the general representation when the gating is weak.

[0014] As a preferred embodiment of the multi-lens visual inspection image processing method of the present invention, the defect detection result is a pixel-level defect segmentation mask or a defect category label, the assembly offset is a physical quantity offset calculated based on the camera calibration parameters, or the assembly status result is a classification label indicating whether the assembly is qualified or not.

[0015] As a preferred embodiment of the multi-lens visual inspection image processing method of the present invention, the viewpoint alignment or coordinate mapping includes at least one of the following: planar mapping based on homography matrix, epipolar correction based on binocular extrinsic parameters, or a mapping table based on pixel-to-workpiece coordinates obtained by calibration.

[0016] Secondly, the present invention provides a multi-lens visual inspection image processing system, comprising, A multi-lens acquisition unit, comprising at least two image acquisition devices and a synchronization triggering module; The processing unit is communicatively connected to the multi-lens acquisition unit. The processing unit includes a processor and a memory. The memory stores program instructions, and the processor executes the program instructions to realize the fusion feature extraction network, the region-aware task modulation module, and the task decoding network.

[0017] As a preferred embodiment of the multi-lens visual inspection image processing system of the present invention, a prism assembly is provided in the optical path of at least one image acquisition device. The prism assembly is used to change the incident light path to form an oblique viewing angle imaging, and the installation attitude parameters of the prism assembly are recorded as part of the camera calibration parameters.

[0018] As a preferred embodiment of the multi-lens visual inspection image processing system of the present invention, it further includes a light source control unit. The light source control unit is used to set parameters such as illumination intensity, illumination angle or spectral channel under the control of the synchronization trigger module, and to store the parameters in association with the camera calibration parameters for the processing unit to call.

[0019] Through the above technical solution, the present invention can achieve at least the following beneficial effects: 1) To address the issue of high-speed production lines being sensitive to detection cycle time, this invention employs multiple lenses to complete multi-view acquisition within the same synchronous trigger cycle, and completes alignment, fusion, and decoding output within a single processing link. This avoids the need for multiple exposure sequences to accommodate imaging of different materials, thereby achieving synchronous determination of multiple attributes without increasing the number of acquisition frames.

[0020] 2) Addressing the issue that significant differences in reflection and texture between composite materials under unified imaging conditions lead to the dominance of shared features by a single material, this invention aggregates information from multiple observation directions into shared features through cross-view fusion. This allows reflective, occluded, or low-contrast areas to obtain supplementary clues from other viewpoints, reducing the overall impact of single-view imaging defects on defect detection and assembly evaluation.

[0021] 3) To address the problem that multi-task shared backbones can easily lead to mutual constraints between tasks and a decrease in sensitivity to some defect categories, this invention introduces a region-aware task modulation mechanism after sharing features. This mechanism generates spatial weight maps for different detection tasks and performs gating or residual gating, enabling each task to form differentiated spatial attention on the same shared representation. This structurally reduces gradient pull in irrelevant regions and improves the stability and consistency of multi-task parallel output.

[0022] 4) To address the problem that assembly relationship evaluation results are difficult to translate into interpretable and comparable dimensional outputs, this invention incorporates camera calibration parameters, a unified reference coordinate system, and the conversion link from pixel to workpiece coordinates into the processing flow. This enables assembly offsets to be expressed in the form of physical quantities and can be further linked with preset tolerance ranges to output assembly status, thereby supporting production line judgment, alarms, and process closed-loop adjustments.

[0023] 5) To address the common problem of cross-view matching error and computational overhead in multi-camera systems, this invention utilizes geometric priors obtained through viewpoint alignment or coordinate mapping to construct corresponding region constraints, enabling cross-view interactions to occur within the geometrically corresponding range. This not only suppresses the propagation of mismatches caused by texture repetition and cross-material boundaries, but also provides controllable computational load and latency boundaries for online deployment. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0025] Figure 1 This is a flowchart of the multi-lens visual detection image processing method in the embodiment.

[0026] Figure 2 This is a framework diagram of the multi-lens visual inspection image processing system in the embodiment. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0029] To avoid ambiguity, synchronous acquisition in this paper refers to at least two image acquisition devices completing exposure and outputting images within the same synchronous trigger cycle, with the trigger signal provided by a unified synchronous trigger module; the synchronization error does not exceed a preset synchronization error threshold to reduce the risk of inter-frame misalignment under motion conditions. Different perspectives in this paper refer to different observation directions formed by the extrinsic parameter arrangement of different image acquisition devices, or different observation directions formed by changing the optical path through a prism assembly; different perspectives have a preset geometric relationship and can be mapped through camera calibration parameters. Composite workpieces in this paper refer to workpieces containing at least two different material components that form an assembly interface or bonding interface.

[0030] The trigger signal output by the synchronization trigger module is used simultaneously for the exposure start alignment of each image acquisition device. The synchronization error threshold is set to 1.0 ms by default as an implementation parameter, and the adjustable range is 0.1 to 5.0 ms. The threshold setting is based on the tolerance of the production line movement speed and the target assembly tolerance to inter-frame misalignment. The synchronization error can be obtained by the processing end by statistically analyzing the timestamp difference of the same frame number. If it exceeds the threshold, the multi-view image group corresponding to that frame number is treated as invalid to avoid introducing misalignment features into subsequent fusion.

[0031] Example 1 like Figure 2 As shown, this application proposes a multi-lens visual inspection image processing system, comprising the following components: A multi-lens acquisition unit, comprising at least two image acquisition devices and a synchronization triggering module; The processing unit is communicatively connected to the multi-lens acquisition unit. The processing unit includes a processor and a memory. The memory stores program instructions, and the processor executes the program instructions to realize the fusion feature extraction network, the region-aware task modulation module, and the task decoding network. It also includes a prism assembly disposed in the optical path of at least one image acquisition device. The prism assembly is used to change the incident light path to form an oblique viewing angle imaging, and the installation attitude parameters of the prism assembly are recorded as part of the camera calibration parameters.

[0032] In this embodiment, at least one image acquisition device integrates a prism assembly in its optical path to change the imaging optical path and obtain a tilted viewing angle. The mounting posture and equivalent optical axis direction of the prism assembly are recorded as part of the calibration parameters and participate in the viewing angle alignment and coordinate mapping calculations. Further, the mounting posture parameters of the prism assembly are used to characterize the equivalent viewing angle change after optical path deflection. The posture change can be characterized by the relative rotation angle and relative translation, and are calculated together during calibration. The allowable assembly deviation of the rotation angle is set to 0.50 degrees by default, with an adjustable range of 0.10 to 5.00 degrees. Recalibration is recommended if this range is exceeded to avoid increasing the viewing angle alignment error. When the prism assembly is disassembled or replaced, the processing end prioritizes calling the most recent calibration parameters and performs a rapid consistency check. The reprojection error threshold is used to determine whether full calibration is required. By incorporating the prism viewing angle into a unified calibration model, a specific observation direction can be obtained without changing the camera's mounting position, thereby improving the visibility and detectability of assembly boundaries, bonding interfaces, or reflective areas.

[0033] It also includes a light source control unit, which is used to set parameters for illumination intensity, illumination angle or spectral channel under the control of the synchronous trigger module, and to store the parameters in association with camera calibration parameters for the processing unit to call.

[0034] In this embodiment, the light source control unit is used to set parameters for illumination intensity, illumination angle, or spectral channels. These settings can be used for machine adjustment during model changes or for synchronous illumination within the same trigger cycle. When using multi-channel illumination, each channel can be lit simultaneously or according to a preset strategy within the same exposure window, without increasing the number of additional acquisition frames, thus avoiding the cycle delay caused by multi-frame time-division imaging. In this embodiment, the illumination intensity can be expressed as a relative percentage parameter and stored in association with the workpiece model, with a default value of 60% and an adjustable range of 10% to 100%. The illumination angle is set to 45 degrees by default and has an adjustable range of 10% to 80 degrees. The spectral channels can be enabled or disabled according to the material combination and are kept within the same trigger cycle without increasing the number of additional acquisition frames. To ensure consistency between different batches, the processing end records the corresponding frame number and illumination setting simultaneously when calling illumination parameters. If an illumination intensity drift of more than 5% is detected, a prompt for resetting or recalibrating is made to avoid brightness changes affecting the defect mask threshold and the stability of assembly status judgment. Illumination parameters and calibration parameters are stored in association to quickly call the corresponding imaging configuration under different workpiece models or different material combinations.

[0035] Example 2 Based on Example 1, such as Figure 1 As shown, this application also provides a multi-lens visual inspection image processing method, including the following steps: Step S1: At least two images of the composite workpiece from different perspectives are acquired by at least two image acquisition devices within the same trigger cycle. The composite workpiece contains at least two different material components. In this embodiment, the multi-lens acquisition unit includes at least two cameras and a synchronous trigger module. The synchronous trigger module outputs a unified trigger signal to each camera and can simultaneously output a light source synchronization control signal consistent with the trigger signal. The camera can use a global shutter or an equivalent synchronous exposure method. Specifically, the exposure time is set to 200.0 μs by default as an implementation parameter, and the adjustable range is 10.0–2000.0 μs to balance the imaging signal-to-noise ratio and motion blur suppression under motion conditions. When motion blur is reduced by increasing illuminance or using stroboscopic illumination, the trigger edge of the stroboscopic illumination is consistent with that of the synchronous trigger module to ensure that the illumination conditions of the multi-view images within the same trigger cycle can be reproduced, avoiding brightness drift from different perspectives that could affect subsequent alignment and feature interaction. When the workpiece is in motion, motion blur is reduced by shortening the exposure time, increasing illuminance, or using stroboscopic illumination. To ensure the fusion of multi-view images, the acquisition unit also records the timestamp or frame number for each trigger and pairs them into multi-view image groups at the processing end according to the same frame number. For example, the frame number can be generated incrementally by the synchronization trigger module and broadcast within the multi-lens acquisition unit. The timestamp is recorded by the processing unit or the local clock of the acquisition unit, and the resolution is used as an implementation parameter with a default value of no less than 0.1 ms and an adjustable range of 0.01 to 1.0 ms. When only frame number pairing is used, the processing end can still periodically sample and check the arrival delay difference of the same frame number to detect cross-view mismatch caused by communication jitter and trigger the discarding of the group of images to ensure the consistency of the fusion input.

[0036] In this embodiment, the processing unit pre-acquires and stores camera calibration parameters. These parameters include at least the intrinsic parameters of each camera, distortion parameters, and extrinsic parameter relationships between cameras or between the camera and the workpiece reference coordinate system. Furthermore, the acquisition of camera calibration parameters can be performed after model changeover and adjustment, or after reinstallation of the camera or prism assembly. The calibration results are archived in the processing unit according to workpiece model and assembly station for easy retrieval. Besides being used for viewpoint alignment or coordinate mapping in step S2, the calibration parameters are also used for dimensional conversion of subsequent assembly offsets and definition of coordinate directions. The origin and axis of the workpiece reference coordinate system can be determined by the tooling reference features and fixed during calibration, ensuring comparability of output results from different batches of workpieces. Calibration can be completed using a calibration board or tooling reference features. Similarly, the calibration process can first estimate the intrinsic parameters and distortion parameters of each image acquisition device, and then combine the tooling reference features to obtain the extrinsic parameter relationships between cameras or from the camera to the workpiece reference coordinate system. The calibration quality can be self-checked through reprojection error statistics. The reprojection error threshold is set to 0.5 pixels by default, with an adjustable range of 0.2 to 2.0 pixels. If the threshold is exceeded, a prompt will be made to re-acquire calibration data to avoid introducing systematic geometric errors into the assembly offset output. After calibration, the processing end can map images from each viewpoint to a unified reference coordinate system to achieve spatial alignment of cross-viewpoint features. In this embodiment, the unified reference coordinate system can be either the pixel coordinate system of one viewpoint or the workpiece reference coordinate system. The choice between the two depends on whether the assembly offset needs to be directly output as a physical quantity. When the output is a physical quantity, the conversion ratio from pixel to workpiece coordinates can be obtained from calibration and expressed in mm. The conversion ratio is recalculated when the lens magnification or working distance changes to ensure the stability of the output dimensions. For assembly offsets expressed as physical quantities, the processing end establishes a conversion relationship between pixels and workpiece coordinates based on calibration parameters, thereby converting the pixel domain offset output by the network into physical quantity offsets such as millimeters or angles, or into qualified / unqualified classification results of the assembly status.

[0037] Step S2: Perform viewpoint alignment or coordinate mapping on images from different viewpoints based on preset camera calibration parameters; In this embodiment, the detection object boundaries of each material component are obtained through preset regions, template positioning, or learned partitioning: First, multiple ROIs of material regions are defined in a unified reference coordinate system based on tooling positioning and geometric priors; second, by locating key reference features, the CAD / template regions are projected onto the image to obtain each material region; third, a material partitioning network is used to output a material partitioning mask, and the defect detection task is limited to the corresponding material partition. The detection area of ​​the assembly relationship evaluation task can be further limited to a preset neighborhood of material interfaces, mating boundaries, or assembly features to improve the stability of assembly offset evaluation.

[0038] Step S3: Input the aligned multi-view images into the fusion feature extraction network to obtain a shared feature map; Step S4: Input the shared feature map into the region perception task modulation module, generate spatial weight maps for at least two types of preset detection tasks, and weight the shared feature map accordingly to obtain the corresponding task feature map. The preset detection tasks include defect detection tasks for components of various materials and assembly relationship evaluation tasks between components of different materials. Step S5: Input the feature maps of each task into the corresponding task decoding network, and output the defect detection results and assembly offset or assembly status results.

[0039] The fusion feature extraction network includes a shared encoder and a cross-view fusion module. The shared encoder extracts initial feature maps from images from each viewpoint, while the cross-view fusion module performs information exchange and aggregation on each initial feature map based on an interactive attention mechanism to generate a shared feature map.

[0040] The interactive attention mechanism includes: generating query features from an initial feature map of any viewpoint, and generating key and value features from an initial feature map of another viewpoint; calculating the correlation between the query features and key features and obtaining attention weights; using the attention weights to weightedly aggregate the value features to obtain cross-viewpoint enhanced features, and fusing them with the initial features of any viewpoint to update the viewpoint features, which are then used to merge the updated features from each viewpoint to form a shared feature map.

[0041] In the interactive attention mechanism, the correlation between query features and key features is expressed using a scaled dot product with temperature control to adjust the sharpness of the attention distribution. The correlation is superimposed on a geometrically corresponding region mask obtained by viewpoint alignment or coordinate mapping and then exponentially normalized to obtain attention weights. The attention weights are weighted and converged on the value features to obtain cross-viewpoint enhanced features, and residual fusion is performed with the initial features on the query side before layer normalization is performed to update the viewpoint features. The updated features of each viewpoint are backfilled into spatial feature maps and then mean fusion is performed to obtain a shared feature map. The mask is used to limit the normalization of attention weights to the geometrically corresponding region. In this embodiment, the shared encoder outputs an initial feature map of the same scale for each viewpoint image. The cross-viewpoint fusion module performs interactive attention on the features of each viewpoint: a query representation is generated for any target viewpoint feature, and key and value representations are generated for another viewpoint feature. Attention weights are generated based on the correlation between the query and the key, and the value representations are weighted and converged to obtain cross-viewpoint enhanced features. Subsequently, the cross-viewpoint enhanced features are fused with the target viewpoint features to update the target viewpoint representation. After repeating this process for each viewpoint, the updated multi-viewpoint features are merged through convolution after concatenation or weighted summation to obtain a shared feature map. To control real-time performance, attention calculation can be limited to the aligned corresponding regions or the sparsely sampled feature location set.

[0042] In one implementation, the correlation is calculated in the following manner: In the interactive attention mechanism, the correlation between query features and key features is used to characterize the degree of matching between the query position and the corresponding position, thereby forming the attention weight; With the first The first perspective as the source of the query, the second When two perspectives are used as the source of keys and values, the initial feature maps of the two perspectives are flattened into a sequence according to their spatial location and then linearly projected: , (1) In equation (1), Indicates by the first The feature sequence obtained by flattening the initial feature map from each viewpoint Indicates by the first The feature sequence obtained by flattening the initial feature map from each viewpoint This represents the projection matrix used to generate query features. This represents the projection matrix used to generate key features. This represents the projection matrix used to generate the value features. Indicates the query feature sequence. Represents the key feature sequence, Represents a sequence of characteristic values; The relevance is calculated using a scaled dot product and a temperature coefficient is introduced to stabilize training. For example, the temperature coefficient, as an implementation parameter, is set to 1.0 by default, with an adjustable range of 0.3 to 3.0. A smaller temperature coefficient will make the attention distribution too sharp, thus making it more sensitive to noise and occlusion, while a larger temperature coefficient will make the distribution too flat, thus weakening the cross-view enhancement effect. The dimension of the attention feature used in scaling is determined by the number of channels of the query feature and the key feature. The number of channels is set to 128 by default, with an adjustable range of 32 to 256 to adapt to different computing power and real-time requirements. , (2) In equation (2), Indicates by the first The first perspective query and the first The correlation matrix is ​​calculated from each perspective key. express and The number of channels in the feature dimension, Indicates the first To the temperature coefficient, This represents the transpose operator; if cosine similarity is used, correlation can be replaced with pairs of... and Perform the dot product after vector normalization. Still used to control the sharpness of the distribution; By combining viewpoint alignment or coordinate mapping, masking can be used to limit attention to propagate only within geometrically correspondent regions. Furthermore, the geometrically correspondent regions can be generated from the mapping relationship obtained in step S2. The processing end first calculates the mapping center position of the query position from another viewpoint, and then takes a neighborhood with a radius of 4 pixels as the reference as the correspondent set. The radius, as an implementation parameter, is adjustable from 2 to 12 pixels to balance alignment error and computational complexity. The masking suppression constant for non-corresponding positions is set to -100.0 by default, with an adjustable range of -20.0 to -1000.0, to ensure that the weights of non-corresponding positions are close to 0 after normalization and to prevent numerical overflow. , (3) In equation (3), Indicates the query location index is AND key position index is The masking value at that time, This indicates the mapping relationship constraint obtained in step S2 and the query position. The set of geometrically corresponding key positions This represents a negative infinity constant used to suppress non-corresponding positions; Attention weights are obtained by normalizing the relevance and masking layers. Similarly, the normalization calculation can first subtract the maximum value of the relevance at each query position to improve numerical stability. This stabilization process does not change the relative magnitude of the attention weights. When the geometrically corresponding set is empty or the number of corresponding positions is less than 1, the attention weight at that query position degenerates into an update method that only retains the initial features from the query side to avoid feature pollution caused by invalid matching. , (4) In equation (4), Represents the attention weight matrix. This represents an operator that performs exponential normalization along the key position dimension for each query position; Attention weights are used to weight and converge the value features to obtain cross-view augmented features, and residuals and layer normalization are performed with the query view features to facilitate convergence. , (5) In equation (5), Indicates by the first The cross-view enhanced feature sequence obtained by attention-weighting the viewpoint value features. Indicates the updated number A sequence of features from a single perspective. Representation layer normalization operator; When merging multiple views, the updated features from each view can be backfilled into a spatial feature map and then fused using mean fusion to control computation and latency. , (6) In equation (6), Indicates shared feature maps, This indicates the number of perspectives involved in the integration. Indicates the first Feature sequences updated from each perspective This represents an operator that restores a sequence to a two-dimensional spatial feature map. The view index is a value ranging from 1 to 1. The size of the global relevance matrix increases quadratically with the number of spatial locations; masking limits the attentionable locations to a limited number. During the process, the effective computational cost of correlation and weighted convergence decreases with the size of the corresponding set, adapting to real-time requirements. Optionally, the updated features of each viewpoint before mean fusion need to maintain consistency in spatial resolution. If the spatial size differs due to input resolution or encoder step size settings, the processing end can resample all viewpoint features to the target size of the shared feature map. The resampling method can be bilinear interpolation, and as an implementation parameter, it can be switched to nearest neighbor interpolation to reduce latency. The number of viewpoints for mean fusion is determined by the number of image acquisition devices participating in the fusion. When the image of a certain viewpoint is missing, making the feature of that viewpoint unusable, the shared feature map corresponding to the triggering period is calculated according to the mean of the remaining viewpoints and marked with invalid viewpoints for subsequent result interpretation.

[0043] As can be seen, within the same triggering cycle, after the viewpoint alignment or coordinate mapping of the multi-view images in step S2, the shared encoder extracts the initial feature maps of each viewpoint respectively, and flattens the initial feature maps into feature sequences according to their spatial positions; then, according to equation (1), the query-side feature sequence is projected, and the other viewpoint feature sequence is projected with key and value. Based on the projected query features and key features, the correlation matrix is ​​calculated according to equation (2). The correlation adopts the scaled dot product and introduces a temperature coefficient to adjust the sharpness of the attention distribution; when cosine similarity is used, the correlation calculation is normalized to the query features and key features according to the substitution description of equation (2) before the dot product is calculated. Combining the mapping relationship or epipolar constraint obtained in step S2, the mask is constructed according to equation (3) and superimposed on the correlation matrix so that the attention is only distributed in the geometrically corresponding region; the superimposed result is exponentially normalized according to equation (4) to obtain the attention weight matrix. Attention weights are used to perform weighted aggregation of value features, and cross-view enhanced feature sequences are obtained according to Equation (5). After residual fusion with the original feature sequences of the query side, layer normalization is performed to obtain the updated view feature sequences. The updated feature sequences of the multi-view perspectives are backfilled into spatial feature maps according to Equation (6) and mean fusion is performed. The shared feature map is output for subsequent region perception task modulation module to call. At the same time, the shrinkage of the attentionable region brought about by the masking is used to reduce the effective computation of correlation calculation and weighted aggregation.

[0044] Specifically, in the above implementation, cross-view attention is used to establish a matching relationship between the features of one viewpoint as a query and the features of another viewpoint as a key and value. The correlation calculation adopts dot product and introduces scaling and temperature control to keep the matching distribution stable in different batches and different scenes, avoiding excessive weight concentration or flattening. The attention weights are obtained by normalization operators and are only assigned to geometrically corresponding regions under mask constraints, thereby incorporating the priors formed by viewpoint alignment, epipolar constraints, or mapping tables into feature interaction and reducing mismatches caused by cross-material reflection, occlusion, or texture repetition. The weights are weighted and aggregated to form a cross-view enhancement result, which is then residually fused with the original query features and normalized to make the magnitude of cross-view information injection controllable and the training convergence process smoother. Multi-view merging can use lightweight methods such as mean to reduce latency, and the sparse attention brought by masking is also conducive to controlling the amount of computation, maintaining the fusion effect while taking into account the processing speed of online detection. The region-aware task modulation module includes a weight prediction sub-network. The weight prediction sub-network takes a shared feature map as input and outputs a spatial weight map for each preset detection task and for each spatial location of the shared feature map. The weight values ​​of the spatial weight map are constrained within a preset range.

[0045] The generation of task feature maps includes: for the k-th preset detection task, the shared feature map and the spatial weight map corresponding to the task are weighted element-wise to obtain the k-th task feature map, and the element-wise weighting includes element-wise multiplication or residual fusion with the shared feature map after element-wise multiplication.

[0046] In this embodiment, the region-aware task modulation module includes a weight prediction sub-network. This sub-network takes a shared feature map as input and outputs a spatial weight map corresponding to each task. The values ​​of the spatial weight map are constrained within a preset range using a preset activation function and normalization strategy to avoid training instability or inference oscillations caused by unbounded weights. In this embodiment, the lower bound constraint value of the spatial weight map is set to 0.05 by default and adjustable from 0.01 to 0.20, while the upper bound constraint value is set to 1.00 by default and adjustable from 0.80 to 1.50. The lower bound is not set to 0 to avoid gradient vanishing and feature interruption caused by complete gating closure, and the upper bound is not set too large to avoid excessive amplification of local responses leading to inference oscillations. When Softplus is used to replace the range constraint function, the lower bound constraint remains unchanged while the upper bound is allowed to adapt to changes in response amplitude caused by differences in the contrast of defects in different material components. The generation of the task feature map includes element-wise weighting: the shared feature map is multiplied element-wise with the task space weight map to obtain the gated features; in another embodiment, the gated features are residually fused with the shared feature map to retain the basic representation that is common to multiple tasks and reduce mutual interference between tasks.

[0047] During the task feature map generation process, the weight response output by the weight prediction sub-network is constrained to form a spatial weight map. The spatial weight map is multiplied element-wise with the shared feature map to obtain the task feature map, or the task feature map is obtained by residual fusion after element-wise multiplication with the shared feature map, so as to retain the general representation when the gating is weak. During the training phase, spatial weight map gating is used as a structural means to alleviate gradient conflicts in multiple tasks, and the sparsity constraint and complementary constraint of the spatial weight map are introduced into the joint training objective. At the same time, the loss weight of each task is adjusted according to the loss sliding statistics, so that different tasks form differentiated attention on the same shared representation and reduce mutual constraints. In one implementation, in the element-wise weighted approach, the generation of the task feature map occurs after step S4 and before step S5: the shared feature map provides a general representation, and the spatial weight map provides task-specific gating, so that the same shared representation presents different spatial attention distributions on different tasks; this process can be represented by a coherent link of weight map generation and range constraints—point-wise gating—optional residual fusion. In the case of the When generating a spatial weight map for a preset detection task, the unconstrained weight response map can be output by the weight prediction subnetwork, and a gated weight map can be obtained by applying range constraints: , (7) In equation (7), Indicates the first Unconstrained weighted response graphs corresponding to each preset detection task Indicates that for the first Weight prediction subnetwork mapping for a preset detection task, Indicates shared feature maps, Indicates the first Spatial weight map corresponding to each preset detection task express The lower bound constraint value, express The upper bound constraint value, This represents the Sigmoid range constraint function; if it is necessary to restrict the weights to be positive and allow the upper bound to be adaptive, then... Replace with Softplus and retain only the lower bound constraint. This replacement does not change the technical direction of restricting the weight values ​​of the spatial weight graph. when When used for element-wise multiplication gating, a single-channel gating method can be employed and broadcast along the channel dimension to reduce computational overhead; alternatively, a combination of gating and multiplication can be used. Channel-sensitive gating with the same number of channels improves the ability to depict differences in material textures; the corresponding element-wise multiplication form is: , (8) In equation (8), Indicates the first Feature maps of element-wise multiplication tasks for a pre-defined detection task. This represents the element-wise multiplication operator. Indicates shared feature maps, Indicates the first Spatial weight map corresponding to each preset detection task; When performing residual fusion with the shared feature map after element-wise multiplication, the gating result can be added back to the shared feature map. This allows the task feature map to retain general information channels even when the gating value approaches a low value, enabling stable output for assembly relationship evaluation and defect detection in areas with insufficient local texture. , (9) In equation (9), Indicates the first Residual fusion task feature maps for each preset detection task Indicates the first The residual fusion coefficients of a preset detection task Indicates shared feature maps, Indicates the first Spatial weight map corresponding to each preset detection task Represents the element-wise multiplication operator; When the value is positive and less than 1, the disturbance of the gated component to the shared information is limited, which helps to maintain the stability of the assembly state discrimination threshold under the conditions of multi-material reflection and occlusion. Specifically, the residual fusion coefficient is set to 0.50 by default as an implementation parameter, and the adjustable range is 0.10 to 1.00. A smaller coefficient can reduce the disturbance of the gated component to the shared information and improve the stability of the assembly state threshold, while a larger coefficient can enhance the task differentiation focus but may amplify local noise. In scenarios where the assembly relationship evaluation task is more sensitive to geometric boundaries, the residual fusion coefficient can be set to a value lower than that of the defect detection task to reduce overfitting updates of the shared representation.

[0048] When weighted graph gating is used as a structural means to alleviate gradient conflicts in multi-task operations, gating and residuals have a computable scaling effect on the inverse signal of the shared branch; with the first The gradient of the task loss with respect to the task feature map is denoted as: , (10) In equation (10), Indicates the first The gradient of a preset detection task to its task feature map. Indicates the first The loss function for a preset detection task. Indicates the first Given a set of task feature maps for a predefined detection task, the gradients of the shared feature maps satisfy the following conditions under both element-wise multiplication gating and residual gating: , (11) in, Indicates the first The gradient of a pre-defined detection task on a shared feature map. Indicates the first The gradient of a preset detection task to its task feature map. Indicates the first Spatial weight map corresponding to each preset detection task Indicates and A uniformly shaped full-1 tensor Indicates the first The residual fusion coefficients of a preset detection task This represents the element-wise multiplication operator. Indicates the first The task feature maps for each preset detection task. This represents the feature map of the element-wise multiplication task. Represents the feature map of the residual fusion task; This format allows the region of interest to be synchronously represented as the region with the shared updated representation during backpropagation, structurally reducing the gradient pull between the defect segmentation task and the assembly relationship assessment task on irrelevant regions. To make the weight map more closely reflect key areas of interest and reduce multi-task conflicts, sparse and complementary constraints can be added to the training objective; sparse constraints are used to encourage weights to concentrate on defect- or assembly-related areas. , (12) In equation (12), This represents the sparse regularization term of the spatial weight graph. Indicates the preset number of detection tasks. Represents the number of pixels at a spatial location. This represents calculating the absolute value based on spatial location and summing them up. Norm, Indicates the first The spatial weight map corresponds to each preset detection task; furthermore, the sparse regularization term weight coefficient is set to 0.01 by default as an implementation parameter, and the adjustable range is 0.0001 to 0.10. It is used to control the weight shrinkage intensity. If the value is too small, the weight map will tend to diffuse and weaken the gating effect. If the value is too large, the weight map may become too sparse, thus missing low contrast defects. The complementary regularization term weight coefficient is set to 0.01 by default, and the adjustable range is 0.0001 to 0.10. It is used to reduce the repeated attention of different tasks at the same location. The complementary constraint intensity is not taken to extreme values ​​to avoid the fragmentation of key area information caused by forced separation.

[0049] Complementary constraints are used to reduce redundant attention from different tasks at the same location: , (13) in, This represents the complementary regularization term of the spatial weight graph. Represents the number of pixels at a spatial location. Indicates spatial location index, Indicates the preset number of detection tasks. This represents the index of the first task, and its value ranges from 1 to... , Indicates the index of the second task and its value range is to , Indicates the first A preset detection task is located at The weight values, Indicates the first A preset detection task is located at The weight values; When incorporating the above regularization into a multi-task objective, adaptive task loss weights can be introduced to accommodate the differences in the proportion of defect samples for parts made of different materials: , (14) In equation (14), Indicates the goal of multi-task joint training. Indicates the first The loss weight coefficients for each preset detection task. Indicates the first The loss function for a preset detection task. This represents the weight coefficient of the sparse regularization term. This represents the weight coefficient of the complementary regularization term. This represents the sparse regularization term of the spatial weight graph. Represents the complementary regularization term of the spatial weight graph; for An update method based on loss sliding statistics can be adopted to make the error reduction rates of assembly relationship assessment and defect detection tasks more similar across different workpiece batches, thereby reducing the phenomenon of a single task dominating shared representations in the later stages of training. For example, the window length of the loss sliding statistics is set to 100 by default, with an adjustable range of 20 to 500. The statistical update cycle is set to 10 by default, with an adjustable range of 1 to 50. The updated task loss weight coefficients can be trimmed to 0.20 to 5.00 to avoid excessive weights leading to a single task dominating shared representations or excessive weights causing task learning stagnation. When the proportion of defect samples from different material parts varies significantly with batches, the window length can be increased first to smooth fluctuations and reduce frequent weight oscillations.

[0050] After the shared feature map is input into the region-aware task modulation module in step S4, the weight prediction subnetwork outputs an unconstrained weight response map for each preset detection task, and obtains the spatial weight map corresponding to the task through range constraints according to equation (7), so that the weight values ​​of the spatial weight map are limited to the preset numerical range. The spatial weight map and the shared feature map are gated element-wise in spatial location, and the task feature map is obtained by element-wise multiplication according to equation (8); when the residual fusion method is used, the gated features are added to the shared feature map according to equation (9) to form the residual fusion task feature map, so that the general representation channel is still retained when the gate approaches a low value. In the multi-task training stage, the gradient of the task loss on the task feature map is defined according to equation (10), and in the two cases of element-wise multiplication gate and residual gate, it is reflected as the structural scaling of the gradient of the shared feature map according to equation (11), so that the update of the shared representation is more focused on the region of interest of each task. To guide the spatial weight map to focus on defect or assembly-related regions and reduce inter-task conflicts, a sparse constraint term is added to the training objective according to equation (12), and a complementary constraint term is added according to equation (13). The joint loss is constructed according to equation (14), and the task loss weights are adjusted according to the sliding statistics of the loss during training, so that the error reduction speed of different tasks under the change of sample ratio is more similar. The output task feature map enters the corresponding task decoding network to generate defect detection results and assembly offset or assembly status results.

[0051] Specifically, the above implementation revolves around the generation link from shared features to task features. The spatial weight map acts as a gating mechanism, enabling different detection tasks to form different spatial attention distributions on the same shared representation. After being generated by the prediction sub-network, the weight map is constrained to a given interval by range constraints, avoiding excessive weights that could lead to amplified local responses and cause assembly state threshold drift. Pointwise multiplication selectively passes shared information according to location, which is suitable for enhancing the texture and geometric cues of defect areas or key assembly areas. Residual fusion adds the gating results back to the shared representation, retaining the general information channel even when the gating is too low or the texture is not obvious, making the output more stable in response to changes in occlusion, reflection, and cross-material boundaries. The gating structure synchronously scales the update range of the shared branches during backpropagation, making the task gradient more concentrated in their respective areas of interest and reducing mutual constraints among multiple tasks in irrelevant areas. With the help of sparsity and complementary constraints, attention can be narrowed from large background areas to key areas, and the repeated attention of different tasks in the same location can be reduced. After introducing adaptive task loss weights, the convergence speed of different tasks is more similar under changes in sample ratio or difficulty, which is beneficial to the long-term usability of the shared representation. The defect detection result is a pixel-level defect segmentation mask or defect category label, and the assembly offset is a physical quantity offset calculated based on camera calibration parameters, or the assembly status result is a classification label indicating whether the assembly is qualified or not.

[0052] In this embodiment, the output of the defect detection task includes a pixel-level defect mask, defect category, and its confidence level; the defect mask is used to mark the location and extent of defects in each material region. The output of the assembly relationship evaluation task includes assembly offset or assembly status: the assembly offset includes at least a two-dimensional displacement component along a preset reference direction, or assembly parameters such as gap and skew angle; optionally, the preset reference direction is established with reference to the reference feature of the first material component, and the assembly offset is defined as the displacement of the assembly feature of the second material component relative to the reference feature of the first material component, with the positive and negative directions of the displacement defined by the axis of the workpiece reference coordinate system; when the output includes assembly parameters such as gap or skew angle, the parameters are still derived from the same reference feature system and maintain unit consistency, where displacement is expressed in mm and angle in degrees, facilitating direct comparison with process tolerances. The assembly offset is defined as the displacement of the assembly feature of the second material component relative to the reference feature of the first material component; the reference direction is defined by the workpiece reference edge / reference hole or the axis of the assembly feature, and a correspondence with the camera reference coordinate system is established during calibration. The assembly status classification label is determined based on whether the offset falls within a preset tolerance range. Specifically, the preset tolerance range is derived from process specifications or product drawings and stored in the processing unit as implementation parameters. The default configurable displacement tolerance is 0.20 mm with an adjustable range of 0.05 to 2.00 mm, and the default angular tolerance is 1.00 degrees with an adjustable range of 0.10 to 5.00 degrees. The tolerance is not set to 0 to avoid frequent false alarms caused by sensor noise and calibration residuals. At the same time, the upper limit of the tolerance is not set too high to prevent assembly abnormalities from being masked.

[0053] Viewpoint alignment or coordinate mapping includes at least one of the following: planar mapping based on homography matrix, epipolar correction based on binocular extrinsic parameters, or a pixel-to-workpiece coordinate mapping table obtained from calibration. Furthermore, when the key detection area of ​​a composite workpiece can be approximated as a plane, planar mapping based on homography matrix is ​​preferred to obtain lower computational overhead; when two viewpoints form an effective baseline and it is necessary to reduce cross-viewpoint matching ambiguity, epipolar correction based on binocular extrinsic parameters is preferred; when the workpiece model is fixed and the tooling positioning has high repeatability, the pixel-to-workpiece coordinate mapping table can be generated offline and called during inference. The spatial sampling interval of the mapping table is set to 1.0 pixel by default as an implementation parameter, with an adjustable range of 0.5–4.0 pixels, to strike a balance between accuracy and storage overhead.

[0054] In this embodiment, the fusion feature extraction network, region-aware task modulation module, and task decoding network can obtain parameters through offline training and be deployed for inference on the production line. The training data includes multi-view synchronous image groups and corresponding annotations, with annotations including at least defect area annotations and assembly offset / assembly status annotations. During training, the corresponding losses for defect detection tasks and assembly evaluation tasks are calculated separately and jointly optimized. During deployment, the processing unit reads multi-view image groups of the same trigger cycle according to frame sequence number, performs alignment mapping, feature fusion, task modulation, and decoding output, and uses the results for rejection / alarm or for process closed-loop parameter tuning. Similarly, when a missing view or timeout occurs in the multi-view image group corresponding to a certain frame sequence number, the processing end can mark the frame sequence number as invalid and skip the output of assembly offset or assembly status results to avoid misjudgment based on incomplete geometric information. The timeout threshold is set to 5.0 ms by default as an implementation parameter, with an adjustable range of 1.0 to 50.0 ms. The processing end can cache the two most recent valid image groups for continuous diagnosis. If the cache is insufficient, the defect results are not retained to avoid misusing expired results for the current workpiece.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0056] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A multi-lens visual inspection image processing method, characterized in that, include: Step S1: At least two images of the composite workpiece from different perspectives are acquired by at least two image acquisition devices within the same trigger cycle. The composite workpiece contains at least two different material components. Step S2: Perform viewpoint alignment or coordinate mapping on the images from different viewpoints based on preset camera calibration parameters; Step S3: Input the aligned multi-view images into the fusion feature extraction network to obtain a shared feature map; Step S4: Input the shared feature map into the region perception task modulation module, generate spatial weight maps for at least two types of preset detection tasks, and weight the shared feature map accordingly to obtain the corresponding task feature map. The preset detection tasks include defect detection tasks for components of various materials and assembly relationship evaluation tasks between components of different materials. Step S5: Input the feature maps of each task into the corresponding task decoding network, and output the defect detection results and assembly offset or assembly status results.

2. The multi-lens visual inspection image processing method according to claim 1, characterized in that, The fusion feature extraction network includes a shared encoder and a cross-view fusion module. The shared encoder extracts initial feature maps for each view image, and the cross-view fusion module performs information interaction and aggregation on each initial feature map based on an interactive attention mechanism to generate the shared feature map.

3. The multi-lens visual inspection image processing method according to claim 2, characterized in that, The interactive attention mechanism includes: generating query features from an initial feature map of any viewpoint, and generating key features and value features from an initial feature map of another viewpoint; calculating the correlation between the query features and key features and obtaining attention weights; using the attention weights to weightedly aggregate the value features to obtain cross-viewpoint enhanced features, and fusing them with the initial features of any viewpoint to update the viewpoint features, which are used to merge the updated features from each viewpoint to form the shared feature map. In the interactive attention mechanism, the correlation between query features and key features is scaled dot product and temperature control is introduced to adjust the sharpness of the attention distribution. The correlation is superimposed on the geometrically corresponding region mask obtained by viewpoint alignment or coordinate mapping and then exponentially normalized to obtain the attention weight. The attention weight is weighted and converged on the value features to obtain cross-viewpoint enhanced features, and residual fusion is performed with the initial features on the query side before layer normalization is performed to update the viewpoint features. The updated features of each viewpoint are backfilled into spatial feature maps and then mean fusion is performed to obtain a shared feature map. The mask is used to limit the normalization of the attention weight to the geometrically corresponding region.

4. The multi-lens visual inspection image processing method according to claim 1, characterized in that, The region-aware task modulation module includes a weight prediction sub-network. The weight prediction sub-network takes a shared feature map as input and outputs a spatial weight map for each preset detection task and for each spatial location of the shared feature map. The weight values ​​of the spatial weight map are constrained within a preset numerical range.

5. The multi-lens visual inspection image processing method according to claim 4, characterized in that, The generation of task feature maps includes: for the k-th preset detection task, the shared feature map and the spatial weight map corresponding to the task are weighted element-wise to obtain the k-th task feature map, and the element-wise weighting includes element-wise multiplication or residual fusion with the shared feature map after element-wise multiplication; During the task feature map generation process, the weight response output by the weight prediction sub-network is constrained to form a spatial weight map. The spatial weight map is multiplied element-wise with the shared feature map to obtain the task feature map, or the task feature map is obtained by residual fusion with the shared feature map after element-wise multiplication, so as to retain the general representation when the gating is weak.

6. The multi-lens visual inspection image processing method according to claim 1, characterized in that, The defect detection result is a pixel-level defect segmentation mask or a defect category label, the assembly offset is a physical quantity offset calculated based on the camera calibration parameters, or the assembly status result is a classification label indicating whether the assembly is qualified or not.

7. The multi-lens visual inspection image processing method according to claim 1, characterized in that, The viewpoint alignment or coordinate mapping includes at least one of the following: planar mapping based on the homography matrix, epipolar correction based on binocular extrinsic parameters, or a mapping table based on pixel-to-workpiece coordinates obtained from calibration.

8. A multi-lens visual inspection image processing system, based on the multi-lens visual inspection image processing method according to any one of claims 1 to 7, characterized in that, include: A multi-lens acquisition unit, comprising at least two image acquisition devices and a synchronization triggering module; The processing unit is communicatively connected to the multi-lens acquisition unit. The processing unit includes a processor and a memory. The memory stores program instructions, and the processor executes the program instructions to realize the fusion feature extraction network, the region-aware task modulation module, and the task decoding network.

9. The multi-lens visual inspection image processing system according to claim 8, characterized in that, At least one image acquisition device has a prism assembly in its optical path. The prism assembly is used to change the path of the incident light to form an image with a tilted viewing angle, and the mounting attitude parameters of the prism assembly are recorded as part of the camera calibration parameters.

10. A multi-lens visual inspection image processing system according to claim 8, characterized in that, It also includes a light source control unit, which is used to set parameters for illumination intensity, illumination angle or spectral channel under the control of the synchronization trigger module, and to store the parameters in association with the camera calibration parameters for the processing unit to call.

Images