Cloud-edge-end three-level AI visual detection method for industrial inspection scene

Abstract

This invention discloses a three-level AI visual inspection method for industrial inspection scenarios, belonging to the field of AI visual inspection technology. It includes: acquiring surface images of industrial equipment on the edge and dividing them into blocks; calculating the phase correlation value between the image blocks and the defect template to obtain a first confidence level; extracting local contrast-weighted histograms and comparing them with a texture library to obtain a second confidence level; uploading low-confidence areas to the cloud; performing multi-scale directional filtering on image segments to separate the defect layer; extracting shape context features and matching them with a defect knowledge graph to obtain a third confidence level; and generating a decision based on the three levels of confidence: high confidence levels directly generate work orders, medium confidence levels are reviewed by a high-precision algorithm, and low confidence levels are transferred to manual review. Through cascaded filtering and feature encoding compression in edge-cloud collaboration, high-precision graded defect detection with low bandwidth usage is achieved.

Description

Technical Field

[0001] This invention relates to the field of AI visual inspection technology, specifically a cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios. Background Technology

[0002] In industrial inspection scenarios, machine vision technology can be used to automatically detect surface defects on equipment, effectively replacing manual visual inspection. Existing technologies typically employ two architectures for defect identification: one uploads all high-resolution images to a cloud server for centralized analysis by a cloud-deployed deep learning model; the other runs a lightweight model on edge nodes for real-time inference, only sending back image areas with low confidence to the cloud. Centralized cloud analysis relies on stable and high-bandwidth network connections, and the transmission latency and storage costs of massive image data constitute system bottlenecks. Edge-side lightweight models, limited by computational power, struggle to cover rare defect types with long-tail distributions. When faced with complex textured backgrounds or weak defect signals, their detection sensitivity and false alarm rate fail to meet industrial-grade requirements. The existing two-tier processing architecture fails to balance bandwidth usage, response latency, and detection accuracy. The edge only handles image acquisition and simple preprocessing, lacking an effective initial screening mechanism to compress the amount of uploaded data, resulting in useless images occupying a significant amount of transmission resources. Existing technologies, when transmitting candidate regions between the edge and the endpoint, employ rectangular cropping or full-resolution image patches, failing to compactly encode and represent the region contours, resulting in high communication overhead. In the cloud analysis stage, existing technologies directly perform convolutional neural network inference on image regions, neglecting the textural characteristics of industrial defects at different scales and orientations. This causes the feature responses of weak defects to be submerged by background texture, affecting the ability to detect early, subtle defects. When conflicting detection results at different levels or when confidence levels are in an ambiguous range, existing technologies lack an adaptive decision-making mechanism based on multi-level confidence levels, leading to excessive consumption of high-precision algorithm resources or the underreporting of genuine defects. Summary of the Invention

[0003] This invention provides a three-tiered AI visual inspection method for industrial inspection scenarios, addressing the problems in existing two-tier architectures: the lack of effective initial screening at the edge, leading to useless images consuming significant transmission bandwidth; the lack of compact encoding of region contours resulting in high communication overhead; and the failure of cloud analysis to utilize the multi-scale directional texture characteristics of defects, causing weak defect features to be submerged by the background. By implementing phase-correlation-based fast removal and gradient-direction chain code sequence encoding at the edge, compact description and directional transmission of suspicious regions are achieved. By implementing structural tensor-driven anisotropic diffusion enhancement at the edge, the interference of background texture on defect feature extraction is suppressed while preserving edge noise. By implementing multi-scale directional filtering and separating the defect layer at the cloud, shape context features are extracted and structurally matched with the knowledge graph, enhancing the detection capability of weak defects in multi-directional backgrounds. By fusing three levels of confidence and triggering hierarchical decisions based on different threshold ranges—automatic work orders, high-precision review, and manual auditing—adaptive defect detection from rapid initial screening to accurate judgment is achieved under low bandwidth conditions.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a three-level AI visual inspection method for industrial inspection scenarios, which achieves efficient and accurate defect identification by performing visual inspection processing of different complexities at the edge, cloud, and edge levels. The method includes the following steps: The system acquires images of the industrial equipment surface at the edge, segments the images into blocks, calculates the phase correlation value between each image block and a preset defect template, and obtains a first confidence level. The edge encodes image blocks with a first confidence level higher than a first threshold into chain code sequences along the spatial gradient direction, generates candidate region descriptors, and uploads them to the edge. Thus, the edge uses lightweight template matching to quickly filter a large number of defect-free regions, and only uploads the structured descriptions of suspected defects, reducing the amount of data transmission.

[0005] After receiving the candidate region descriptor, the edge performs anisotropic diffusion filtering on the corresponding original image region to obtain the enhanced region image. The local contrast-weighted histogram of the region image is extracted and compared with the structural similarity of a typical texture library to obtain the second confidence score. The edge uploads the regions with the second confidence score below the second threshold along with the original image fragments to the cloud. The edge further eliminates false alarms caused by the edge through texture enhancement and comparison, making the data uploaded to the cloud more targeted.

[0006] The cloud-based system performs multi-scale directional filtering on uploaded image segments to separate the defect layer. It then extracts the shape context features of the defect layer and matches them with an offline-built defect knowledge graph to obtain a third confidence level. The cloud then generates a comprehensive confidence level based on a weighted sum of the first, second, and third confidence levels. Regions with a comprehensive confidence level higher than the alarm threshold directly generate maintenance work orders. Regions with a comprehensive confidence level between the alarm threshold and the review threshold are re-analyzed using a high-precision visual algorithm. Regions with a comprehensive confidence level below the review threshold are transferred to a human expert review queue. Thus, the cloud utilizes the knowledge graph to achieve refined classification and decision-making, and handles cases in a tiered manner based on comprehensive confidence level, balancing automated processing with human review.

[0007] As a preferred embodiment of the present invention, the calculation of the phase correlation value between each image block and the preset defect template includes: dividing the acquired surface image into a rectangular grid of fixed size, with each grid serving as an independent image block; applying a Hanning window to each image block to suppress spectral leakage, and performing a two-dimensional fast Fourier transform to obtain a complex spectrum; calculating the amplitude spectrum and phase spectrum of the complex spectrum, and normalizing the phase spectrum based on the principal component direction to obtain a normalized phase matrix; performing element-wise phase correlation calculation between the normalized phase matrix and the phase matrix of the preset defect template, and outputting the phase correlation value. This method utilizes the characteristic that phase information is insensitive to texture translation and brightness changes, thereby improving the robustness of the initial screening at the end.

[0008] Preferably, the anisotropic diffusion filtering of the corresponding original image region includes: constructing a structure tensor for each pixel, calculating two eigenvalues ​​and a corresponding eigenvector of the structure tensor, wherein the eigenvector corresponding to the larger eigenvalue indicates the texture direction; defining the diffusion tensor as a directional diagonal matrix, setting the diffusion coefficient along the texture direction as a first preset value, and setting the diffusion coefficient perpendicular to the texture direction as a second preset value, wherein the second preset value is less than the first preset value; updating the original image region using an iterative difference format, calculating the mean of the region's gray-level gradient magnitude after each iteration, terminating the iteration when the rate of change of the mean is lower than a third preset value, and outputting the enhanced region image. This filtering method suppresses background texture while maintaining the defect contour, making subsequent structural similarity comparisons more reliable.

[0009] Furthermore, the multi-scale directional filtering of the uploaded image segments includes: constructing a set of two-dimensional Gabor filters with different scales and directions, where the center frequencies of the filters are uniformly distributed along logarithmic coordinates, and the direction angles are sampled at equal intervals within the range of 0 to 180 degrees; convolving the image segment with each filter to obtain a complex response, and taking the square of the response amplitude as the energy map at that scale direction; performing local maximum suppression on all energy maps, retaining the scale and direction combination with the strongest response, and using the phase information of the corresponding direction as the texture direction code; calculating the Hamming distance between the texture direction code and the direction code in the defect knowledge graph to obtain the direction matching degree, which is used as a component of the third confidence score. Through multi-scale directional analysis, the directional texture features of defects can be extracted, improving the accuracy of matching with the knowledge graph.

[0010] As another preferred technical solution of the present invention, the step of generating a comprehensive confidence score based on the weighted sum of a first confidence score, a second confidence score, and a third confidence score includes: setting a first weight, a second weight, and a third weight, wherein the first weight is proportional to the illumination intensity during edge image acquisition, the second weight is proportional to the area of ​​the edge candidate region, and the third weight is proportional to the coverage time of the cloud-based defect knowledge graph; calculating the sum of the first weight, the second weight, and the third weight, and dividing each weight by the sum to obtain a normalized first weight, a normalized second weight, and a normalized third weight; calculating the product of the first confidence score and the normalized first weight, the product of the second confidence score and the normalized second weight, and the product of the third confidence score and the normalized third weight, and adding the three products to obtain the comprehensive confidence score; when the comprehensive confidence score reaches the alarm threshold, extracting the centroid coordinates and geometric dimensions of the defect region, and generating a maintenance work order containing location and size information. This weighting strategy allows the contribution of the confidence score at each stage to be dynamically adjusted according to the actual working conditions, making the decision result closer to the on-site conditions.

[0011] Preferably, the end-side encoding of image blocks with a first confidence level higher than a first threshold into a chain code sequence along the spatial gradient direction includes: calculating the gradient magnitude and gradient direction of pixels within each image block, retaining pixels with gradient magnitude greater than a fourth preset value; starting from the pixel with the largest gradient magnitude, recording the relative directions of adjacent retained pixels sequentially according to the 8-neighborhood direction chain code rule to form an initial chain code; performing redundancy removal processing on the initial chain code, merging consecutive identical direction codes into a single direction code to obtain a compressed chain code as the candidate region descriptor.

[0012] As a further improvement to multi-scale directional filtering, after the directional angles are sampled at equal intervals within the range of 0 to 180 degrees, the following steps are performed: based on the spatial location of the region where the second confidence level uploaded from the side is lower than the second threshold, the common defect texture direction distribution of the device part where the region is located is statistically analyzed; the statistically obtained texture direction distribution is used as prior information to reweight the equally spaced sampling directions from 0 to 180 degrees, increasing the filter weight of common defect directions and decreasing the filter weight of non-defect directions; the multi-scale directional filtering is then re-executed using the weighted filter set. Introducing the prior direction distribution makes the cloud response to typical defects more sensitive, reducing the risk of missed detections.

[0013] Preferably, the phase matrix of the preset defect template is obtained through an offline calibration process: sample images containing typical defects are acquired; each sample image is divided into blocks according to the fixed size, retaining only the image blocks containing defects; a two-dimensional fast Fourier transform is performed on the retained image blocks to extract the phase spectrum; all phase spectra are aligned along the same principal component direction; the arithmetic mean of the aligned phase spectra is calculated to obtain the average phase matrix, which is then used as the phase matrix of the preset defect template. Using the average phase matrix as a template allows for the fusion of common features from various defect samples, improving the generalization ability of the initial screening at the edge.

[0014] Furthermore, the iterative difference format is used to update the original image region. After each iteration, the mean value of the gray-level gradient magnitude of the region is calculated. The iteration is terminated when the rate of change of the mean value is lower than a third preset value. Specifically, this includes: after each diffusion iteration, calculating the arithmetic mean of the gray-level gradient magnitudes of all pixels in the current iteration image; subtracting the arithmetic mean of the current iteration from the arithmetic mean of the previous iteration to obtain the difference; then dividing the difference by the arithmetic mean of the previous iteration to obtain the rate of change; stopping the iteration when the rate of change is less than the third preset value, and outputting the current iteration image as the enhanced region image. Using the rate of change of the mean gradient magnitude as the stopping criterion can adaptively control the diffusion degree and avoid excessive smoothing that leads to the loss of defect details.

[0015] Another preferred embodiment of the present invention is that the region with a comprehensive confidence level between the alarm threshold and the review threshold is re-analyzed by a high-precision visual algorithm, including: cropping the corresponding region from the complete original image stored in the cloud, performing color space conversion on the region, and separating the luminance component and chrominance component; calculating the local binary pattern texture spectrum based on the luminance component, and calculating the saturation histogram based on the chrominance component; concatenating the local binary pattern texture spectrum and the saturation histogram into a feature vector, and performing cosine similarity calculation with the feature vector of the corresponding defect type in the defect knowledge graph; if the cosine similarity exceeds a fifth preset value, it is determined as a confirmed defect and a work order is generated; otherwise, it is marked as a suspected defect and transferred to the human expert review queue. The fusion of luminance texture and chrominance statistical features for secondary analysis can effectively distinguish between real defects and complex background interference, reducing the generation of false work orders.

[0016] The technical effects and advantages provided by the present invention in the above technical solution are as follows: The edge processing employs a combination of phase-correlation-based fast alignment and chain code sequence encoding to process acquired images. Images are divided into fixed-size independent grid blocks. After applying a Hanning window to each block to suppress spectral leakage, a two-dimensional fast Fourier transform is performed, separating the obtained complex spectrum into amplitude and phase spectra. The phase spectrum is normalized based on the principal component directions to obtain a rotation-invariant phase matrix representation. This matrix is ​​then compared with the offline-calibrated defect template phase matrix for element-wise phase correlation calculation, outputting a first confidence score reflecting the degree of structural similarity. Image blocks exceeding the first threshold are identified as suspected defect candidate regions. The gradient magnitude and gradient direction of pixels within the region are calculated. Starting from the pixel with the largest gradient magnitude, the relative directions of adjacent retained pixels are traced according to the 8-neighborhood direction chain code rule to form an initial chain code. After redundancy removal and compression, a compact candidate region descriptor is generated. This descriptor uses only direction sequences to express the structural information of the region contour, reducing the data volume by more than an order of magnitude compared to transmitting rectangularly cropped image segments, thus compressing the communication load from the edge to the edge from the source. Meanwhile, the fast rejection mechanism on the edge performs filtering of a large number of normal image blocks locally. The data transmitted from the edge only targets a very small proportion of suspicious areas, avoiding the bandwidth bottleneck caused by uploading the full high-resolution original image. This enables the system to achieve millisecond-level response on the inspection terminal side, supporting concurrent detection of a large number of devices. When the cloud performs multi-scale directional filtering on the image segments uploaded from the edge, it constructs a set of two-dimensional Gabor filters with a center frequency uniformly distributed along the logarithmic coordinate and directional angles sampled at equal intervals within the range of 0 to 180 degrees. Each filter is convolved with the image segment to generate a complex response. The square of the response amplitude is taken to form an energy map at the corresponding scale and direction. Local maximum suppression is performed on all energy maps, retaining the scale and direction combination with the strongest response, and the phase information of the corresponding direction is encoded as a texture direction code. This operation separates the defect layer that dominates the response from the image segment and suppresses secondary response components from background texture and noise. The direction code obtained by filtering is matched with the direction codes of typical defects in the offline defect knowledge graph using Hamming distance. The direction matching degree between the filter and the structured defect template is used as the core component of the third confidence. The defect knowledge graph covers common defect morphologies, texture orientations, size proportions, and evolution trends of various equipment components. The Hamming distance of the directional code is insensitive to pixel-level brightness changes, focusing only on the topological orientation consistency of the defect structure. This allows for stable feature comparison results for weak defects even in complex multi-directional backgrounds, and the detection capability is unaffected by illumination fluctuations and contrast attenuation. When a region with a second confidence level below the threshold possesses prior defect direction distribution information for a specific equipment part, the directional sampling weights of the Gabor filter are redistributed. This results in higher gain for filters targeting common defect directions and suppression of filters targeting non-defect directions, enhancing selectivity for typical defect direction textures from the filtering source. This highlights the feature response of weak defects in multi-directional backgrounds. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of a three-level AI visual inspection method for industrial inspection scenarios, focusing on cloud, edge, and terminal. Figure 2 This is a flowchart of the anisotropic diffusion filtering enhancement process for image regions; Figure 3 This is a schematic diagram of the first confidence level of phase correlation of the end-side image block; Figure 4 It is a curve showing the change of the mean gray-level gradient magnitude of each region of the image with the number of iterations; Figure 5 This is a schematic diagram illustrating the relationship between overall confidence level and alarm review threshold; Figure 6 This is a comparison curve of the cosine similarity of the verification regions. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1This invention provides a cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios, including the following steps: At the device side, an image of the industrial equipment surface is acquired and segmented into blocks. The phase correlation value between each image block and a preset defect template is calculated to obtain a first confidence level. At the device side, image blocks with a first confidence level higher than a first threshold are encoded into chain code sequences along the spatial gradient direction to generate candidate region descriptors and uploaded to the edge side. After receiving the candidate region descriptors, the edge side performs anisotropic diffusion filtering on the corresponding original image region to obtain an enhanced region image. Then, the local contrast-weighted histogram of the region image is extracted and structurally similar to a typical texture library. The first image segment is compared to the second image segment to obtain the second confidence level. The second image segment with the second confidence level lower than the second threshold is uploaded to the cloud along with the original image segment. The cloud performs multi-scale directional filtering on the uploaded image segment to separate the defect layer, extracts the shape context features of the defect layer, and matches them with the offline constructed defect knowledge graph to obtain the third confidence level. The cloud generates a comprehensive confidence level based on the weighted sum of the first confidence level, the second confidence level, and the third confidence level. The comprehensive confidence level is higher than the alarm threshold and a maintenance work order is directly generated for the region. The comprehensive confidence level is between the alarm threshold and the review threshold and is re-analyzed by a high-precision visual algorithm. The region is lower than the review threshold and is transferred to the human expert review queue.

[0021] Example 1:

[0022] In practice, the surface images of industrial equipment acquired from the end-view are divided into non-overlapping rectangular grids according to a pre-defined fixed size, with each rectangular grid serving as an independent image block. The fixed size is determined based on the industrial camera resolution and typical defect scale; for example, the image is divided into 64-pixel × 64-pixel image blocks, with no overlapping areas between adjacent image blocks.

[0023] For each independent image patch, a two-dimensional Hanning window is applied to suppress spectral leakage caused by patch truncation. The two-dimensional Hanning window is generated as follows: first, a sequence of one-dimensional Hanning windows is generated, where the window length is the width of the independent image patch. The first-dimensional window sequence contains... The value of each element is ,in , The width of each independent image patch is given in pixels. A two-dimensional Hanning window matrix is ​​then obtained by multiplying the one-dimensional window by itself. The independent image patch is then multiplied pixel-by-pixel by the two-dimensional Hanning window matrix to obtain the windowed image patch.

[0024] A two-dimensional fast Fourier transform (FFT) is performed on the windowed image patch to obtain the complex spectrum. The FFT outputs a complex matrix of the same size as the windowed image patch, with each frequency point... The complex value at the location is denoted as Calculate the amplitude spectrum and phase spectrum from the complex spectrum, respectively. for Phase spectrum for ,in and These are the real and imaginary parts of the complex value, respectively.

[0025] Next, the phase spectrum is normalized and aligned based on the principal component directions. The principal component directions are determined as follows: the phase angle distribution at each frequency position in the phase spectrum is calculated, and principal component analysis is used to obtain the direction of maximum variance, which corresponds to the main orientation of the texture in the spatial domain. All phase values ​​in the phase spectrum are then... Subtracting the phase offset corresponding to the principal component directions aligns the principal component directions with the zero-phase reference direction, yields the normalized phase matrix. .

[0026] Element-by-element phase correlation is calculated using the following formula:

[0027] in, This represents the calculated phase correlation value; and These represent the number of sampling points in the two dimensions of the image patch in the frequency domain, respectively, and are the same as the width and height of the windowed image patch; The frequency index in the normalized phase matrix is... The element value; This represents the index of the same frequency in the phase matrix of the preset defect template. The element value; Given a cosine function; summate across all frequency indices. Phase correlation value. The range of values ​​is The closer the value is The more similar the phase structure of the image patch is to the preset defect template, the higher the confidence level.

[0028] The phase matrix of the aforementioned preset defect template is obtained through an offline calibration process. During offline calibration, multiple sample images containing typical defects on the surface of industrial equipment are first acquired. Typical defects include cracks, corrosion, scratches, etc. For each sample image, a rectangular grid is divided according to the same fixed size as in the online processing stage, resulting in sample image blocks. Manually or based on existing annotation information, only sample image blocks whose central region contains more than a preset proportion of defect pixels are retained; specifically, sample image blocks whose defect area accounts for more than 30% of the total area of ​​the sample image block are considered for retention.

[0029] For each retained sample image patch, the same two-dimensional Hanning window as in the online processing stage is applied, and a two-dimensional fast Fourier transform is performed to extract the phase spectrum from the resulting complex spectrum. Since the principal directions of the defect texture may differ between different sample image patches, all extracted phase spectra are aligned along the same principal component direction. The alignment method for the principal component direction is consistent with the alignment method of the normalized phase matrix in the online stage; that is, using the standard reference direction as a reference, the principal component direction of each phase spectrum is rotated to this standard reference direction to obtain the aligned phase spectrum.

[0030] Calculate the aligned phase spectrum at each frequency position. The arithmetic mean at points 1 to 2. Let there be a total of 10 points. The aligned phase spectrum, the... The aligned phase spectrum at the frequency index The phase value at that point is Then the average phase matrix The element value is:

[0031] The obtained average phase matrix is ​​the phase matrix of the preset defect template, which is stored in the non-volatile memory on the end side as a reference template for online calculation of phase correlation values.

[0032] In some implementations, before calculating the element-by-element difference between the normalized phase matrix and the phase matrix of the preset defect template, phase unwrapping processing can be further performed to eliminate errors introduced by phase jumps and ensure that the calculation results reflect the true similarity of the phase structure. The phase correlation value is directly used as the first confidence output, which is then compared with the first threshold to determine whether the image patch needs to generate candidate region descriptors and be uploaded to the edge.

[0033] See Figure 3 In the figure, the horizontal axis represents the image patch number of the industrial equipment surface acquired from the end side, ranging from approximately 0 to 520. The vertical axis represents the first confidence value calculated between the corresponding image patch and three preset defect templates (crack template, corrosion template, and scratch template). The curves use solid lines, dashed lines, and dotted lines to represent the changing trends of the first confidence value for the crack template, corrosion template, and scratch template, respectively. The figure also indicates the first threshold with a red dotted line, and the threshold value is approximately 0.65.

[0034] As can be seen from the curve changes, the first confidence level corresponding to the crack template is generally high, with values ​​mostly concentrated between 0.75 and 0.95, and a few peaks fluctuating close to 1, indicating that the phase structure similarity between the image patch and the crack template is high, and it has strong defect matching characteristics. The first confidence level curve of the erosion template is located in the middle segment, with values ​​mostly fluctuating between 0.6 and 0.85. The confidence levels of some image patches are close to or slightly higher than the first threshold, indicating that the texture of some image patches has some similarity to the erosion template, but the overall similarity is lower than that of the crack template. The first confidence level of the scratch template is generally low, concentrated between 0.45 and 0.65, and the confidence levels of most image patches do not exceed the first threshold, indicating that the phase structure matching between the image patch and the scratch template is weak.

[0035] The first threshold line serves as a filter; image blocks with a confidence level higher than this threshold are identified as potential defect candidates. Spatial gradient direction encoding is then performed to generate candidate region descriptors, which are uploaded to the edge detection system. Image blocks with a confidence level lower than the threshold are filtered out at the edge detection stage and not uploaded. Actual curves show that the vast majority of image blocks in the crack template have a confidence level higher than the threshold, indicating that crack defect detection performs well at the edge detection stage. In the corrosion template, approximately some image blocks have confidence levels close to the threshold, suggesting that these areas require close attention. In the scratch template, most image blocks have confidence levels lower than the threshold and are filtered out at the edge detection stage; only a few image blocks exceeding the threshold are uploaded as candidates to the edge detection system.

[0036] Example 2:

[0037] In specific implementation, please refer to Figure 2 After receiving the candidate region descriptor from the edge-side receiver, the corresponding original image region is located in the original image data cached on the edge-side based on the spatial location information contained in the candidate region descriptor. The edge-side then performs anisotropic diffusion filtering on this original image region to enhance the texture structure within the region and suppress noise.

[0038] For each pixel in the original image region, a structure tensor is constructed. The structure tensor is a 2×2 symmetric positive semi-definite matrix, obtained by calculating the gray-level gradient information in the pixel's neighborhood. The coordinates of the pixel in the original image region are used as... Centered on the pixel, a local neighborhood window is selected, and the size of the neighborhood window is set to [value]. A rectangular region of pixels, where This represents the spatial scale parameter, with a value of 3 pixels. Within the neighborhood window, the grayscale gradient along the horizontal direction is calculated for each pixel. and grayscale gradient along the vertical direction Horizontal grayscale gradient The vertical grayscale gradient is obtained by convolving a neighborhood window with the horizontal Sobel operator. Obtained by convolution of a neighborhood window with the vertical Sobel operator. Structure tensor Represented as:

[0039] in, This indicates that the summation is performed on all pixels within the neighborhood window. This represents the scalar value of the horizontal grayscale gradient of a pixel within the neighborhood window. This represents the scalar value of the vertical grayscale gradient of the same pixel within the neighborhood window. (For the structure tensor) Perform eigenvalue decomposition to obtain the first eigenvalue. Second eigenvalue and the corresponding first feature vector Second eigenvector First eigenvalue Second eigenvalue satisfy The larger eigenvalue is the first eigenvalue. The corresponding first feature vector The indicator is the texture direction, i.e., the dominant texture direction of the local area where the pixel is located; the smaller feature value is the second feature value. The corresponding second feature vector Indicates the direction perpendicular to the texture direction.

[0040] Based on the first feature vector Second eigenvector Define the diffusion tensor Diffusion tensor It is a directional 2×2 diagonal matrix, with the first eigenvector... Second eigenvector In the local coordinate system formed, the diffusion tensor The diagonal elements are the diffusion coefficients along the texture direction. and diffusion coefficient perpendicular to the texture direction Diffusion coefficient along the texture direction Set to the first preset value, with a value of 1.0; diffusion coefficient perpendicular to the texture direction. The second preset value is set to 0.1. This second preset value of 0.1 is less than the first preset value of 1.0, making the diffusion along the texture direction stronger than the diffusion perpendicular to the texture direction. This allows for smoothing along the texture direction while preserving the sharpness of the texture edges. Diffusion Tensor The representation in the world coordinate system is obtained through similarity diagonalization: ,in For the first eigenvector Second eigenvector A rotation matrix formed by column combination. The elements on the diagonal are in order. and diagonal matrix, Representing the rotation matrix The transpose of .

[0041] An iterative difference scheme is used to update the original image region. The number of iterations is set to an integer variable. express, Starting from 0, the number... The image region after the next iteration is denoted as ,in hour This represents the original image region. Each iteration uses an explicit difference scheme for updating, with the update equation being: ,in This represents the time step, with a value of 0.15. Indicates the first After the second iteration, the image region is in coordinates The gray-level gradient vector at that location; Denotes the divergence operator, Represents the diffusion tensor With gray gradient vector Matrix product; This represents the grayscale increment in each iteration. Time step. The value of 0.15 is chosen to ensure the numerical stability of the explicit difference scheme while also considering the convergence speed. With a spatial step size of 1 pixel, this value satisfies the stability condition. .

[0042] The mean value of the gray-level gradient magnitude of the region is calculated after each iteration. Specifically, the calculation method is as follows: In the first iteration... After the next iteration, for the image region For all pixels in the array, calculate the grayscale gradient magnitude for each pixel. The grayscale gradient magnitude is defined as follows: ,in and These are the grayscale gradient values ​​of a pixel along the horizontal and vertical directions, respectively. The arithmetic mean of the grayscale gradient magnitudes of all pixels within the image region is then calculated to obtain the first... Mean value of the region gray-level gradient after the second iteration The mean of the gray-level gradient magnitude of the region in this iteration. Average gray-level gradient magnitude of the region compared to the previous iteration Compare the results and calculate the rate of change. The rate of change is calculated as follows: Calculate... and absolute value of the difference Divide the absolute value of the difference by the mean magnitude of the gray-level gradient of the region from the previous iteration. The rate of change was obtained. When the rate of change The iteration terminates when the rate of change falls below a third preset value. The third preset value is 0.005, meaning iteration stops when the rate of change is less than 0.5%. The reason for setting the third preset value to 0.005 is that it indicates the image region texture enhancement process has converged, and further iterations will no longer significantly improve the texture enhancement effect. The current iteration image at the time of termination is used as an example. The enhanced region image is output as the final image. This enhanced region image is then used to extract local contrast-weighted histograms and compare their structural similarity with a typical texture library. The typical texture library is constructed as follows: First, a large number of images of defect-free industrial equipment surfaces under different lighting conditions and on different equipment parts are collected as a sample set for constructing the texture library. Then, each sample image is divided into blocks according to the same fixed size as in the online processing stage, resulting in several normal texture image blocks. For each normal texture image block, its local contrast-weighted histogram is extracted as a texture feature description. The local contrast-weighted histogram is extracted by calculating the grayscale contrast between each pixel within the image block and its neighboring pixels. Using this contrast value as a weight, the grayscale values ​​of the pixels are weighted and statistically analyzed to generate a 128-dimensional weighted histogram vector. The 128-dimensional weighted histogram vectors of all normal texture image blocks are then aggregated to form a vector set. K-means clustering was performed on the vector set, with the number of clusters K set to 64. Through iterative optimization, the sum of the squared Euclidean distances from each vector to the center of its cluster was minimized, resulting in 64 cluster center vectors. These 64 cluster center vectors were used as typical texture feature vectors in a typical texture library and stored in a non-volatile memory on the side for subsequent structural similarity comparison with the local contrast-weighted histogram of the image to be detected. The similarity (e.g., cosine similarity or histogram cross-validation) between the 128-dimensional local contrast-weighted histogram of the region to be detected and the 64 typical texture feature vectors was calculated, and the maximum value was taken as the second confidence score.

[0043] See Figure 4 In the figure, the horizontal axis represents the number of iterations of the anisotropic diffusion filter, ranging from 0 to 700, and the vertical axis represents the mean gray-level gradient magnitude of the region, ranging from 0 to 50. Three curves are plotted in the figure, corresponding to three different original image regions processed by the side, labeled as region A (blue solid line), region B (orange dashed line), and region C (green dotted line).

[0044] The curves show an overall trend of starting with a high initial mean grayscale gradient magnitude, which rapidly decreases with increasing iterations, exhibiting a clear exponential decay. Specifically, the initial value for region A is approximately 45, for region B approximately 30, and for region C approximately 18. The order of the initial mean grayscale gradient magnitudes for these three regions is consistent, reflecting the differences in initial texture intensity across different regions.

[0045] Within the first 100 iterations, all three curves decreased rapidly at a high rate, indicating that anisotropic diffusion filtering was effective in smoothing texture and suppressing noise in the image region in the initial stages. As the number of iterations approached 100, the curves flattened out and gradually converged to stable values. Regions A, B, and C converged to average grayscale gradient moduli of approximately 25, 15, and 8, respectively, indicating that the texture enhancement process tended to stabilize within this iteration range, and further iterations had limited effect on improving the texture enhancement effect.

[0046] The iteration termination condition corresponds to the preset threshold (0.005) based on the mean change rate of the gray-level gradient magnitude described in Example 2. That is, when the mean change rate of the gray-level gradient magnitude is less than 0.5% for two consecutive iterations, the texture enhancement process is determined to have converged, and the iteration stops. The stable phase of the three curves in the figure reflects this convergence state.

[0047] Example 3:

[0048] In practice, the cloud acquires the original image fragment uploaded from the edge, and this original image fragment corresponds to the region where the second confidence level is lower than the second threshold. The cloud performs multi-scale directional filtering on the original image fragment to separate the defect layer and extract texture direction features.

[0049] Construct a set of two-dimensional Gabor filters with different scales and orientations. Each two-dimensional Gabor filter is constructed by multiplying a Gaussian envelope by a complex exponential plane wave. The kernel function of the two-dimensional Gabor filter is defined in the spatial domain as having an elliptical Gaussian envelope whose direction coincides with the propagation direction of the complex exponential wave. The scale of the two-dimensional Gabor filter is controlled by a center frequency, which is uniformly distributed along a logarithmic coordinate system. Let the total number of scales be... , The value is 5, and the center frequencies of the 5 scales are respectively , , , , The frequency unit is period per pixel. The reason for the uniform distribution of the center frequency along the logarithmic coordinate is that the width of the defect texture in the image spans multiple octaves, and the logarithmic distribution can cover different texture scales from coarse to fine with uniform octave intervals. The directional angle of the two-dimensional Gabor filter is sampled at equal intervals within the range of 0 to 180 degrees, and the total number of samples is set to... , The value is 12, and the corresponding direction angles are respectively , , , , The reason for setting the total number of directional angle samples to 12 is that the angular resolution requirement for the directional distribution of industrial defect textures is about 15 degrees. 12 directions can achieve a balance between directional sensitivity and computational overhead, and cover symmetrical directions within a 180-degree range.

[0050] After sampling at equal intervals for the orientation angle, an orientation weighting step is performed. The cloud determines the equipment part type of the region based on the spatial location of the region uploaded from the edge where the second confidence level is lower than the second threshold. The method for determining the equipment part type is as follows: the cloud maintains an equipment structure-part mapping table, recording the spatial range of each component of industrial equipment in the inspection image. The equipment part to which the part belongs is determined based on the spatial coordinates of the uploaded region, such as bearing housing, flange, weld, etc. The cloud also maintains an equipment part-defect orientation distribution mapping table, which is constructed by offline statistical analysis of historical defect sample data, recording the statistical probability distribution of common texture orientation angles when defects occur in each equipment part. For a given equipment part, the common defect texture orientation distribution is represented as a vector. ,in Indicates the first The weighted prior values ​​corresponding to each sampling direction angle , Let be the total number of direction samples. Using the statistically obtained texture direction distribution as prior information, the direction angles sampled at equal intervals are reweighted: for the ... There are directional angles, and the corresponding two-dimensional Gabor filter kernel function is denoted as . ,in For scale indexing, , Given spatial coordinates, construct weighting coefficients. , ,in As the amplification factor, The value is 2.0. The mean of all prior weight values Weighting coefficients A value greater than 1 indicates that the filter weights are increased for directions corresponding to common defects, while a value less than 1 indicates that the filter weights are decreased for directions other than defects. Amplification factor The value of 2.0 is chosen to approximately double the filter gain in common defect directions while keeping the gain in non-defect directions essentially unchanged, thus enhancing sensitivity to typical defects. The kernel function of the two-dimensional Gabor filter is modified using weighting coefficients to obtain the weighted filter kernel function. The weighted filter set is used to re-perform subsequent multi-scale directional filtering.

[0051] The original image segments are convolved using a weighted set of filters. For each scale... and direction The combination of these factors will result in a weighted two-dimensional Gabor filter kernel function. With the original image fragment Perform two-dimensional convolution to obtain complex responses. Complex response It includes both real and imaginary parts. The square of the complex response amplitude is taken as the energy map at that scale. The calculation method is as follows ,in This indicates the operation of taking the real part. This indicates the operation of taking the imaginary part. The energy map represents the local texture response intensity of the original image fragment at a specific scale and direction.

[0052] Local maxima suppression is performed on all energy maps. For each spatial location... At all scales and direction Searching for energy values ​​within the constructed three-dimensional space. The largest combination. The specific method of local maximum suppression is as follows: Using each spatial location as the center, comparisons are made within adjacent scale and orientation layers. If the energy value of a combination is a maximum in its neighborhood, that combination is retained; otherwise, the suppression is zero. After local maximum suppression, at most one strongest scale and orientation combination is retained for each spatial location. After retaining the strongest scale and orientation combination, the phase information of the corresponding orientation is extracted as a texture orientation code. The complex response corresponding to the strongest combination is then used... Extracting phase angle ,in and These represent the scale index and orientation index selected after local maximum suppression, respectively. The phase angle... Quantization to binary code, the quantization method is to quantize... The phase range is evenly divided into 8 intervals, each interval corresponding to a 3-bit binary code, thus obtaining the texture orientation code for that spatial location. For all spatial locations in the image segment that retain the strongest combination, their binary texture orientation codes are extracted to form a texture orientation code map.

[0053] The Hamming distance between the texture orientation code and the orientation codes in the defect knowledge graph is used to calculate the orientation matching degree. The defect knowledge graph stores template orientation codes for each known defect type. The orientation code for a specific defect type is a binary sequence with the same length as the texture orientation code. The orientation matching degree is calculated using the following formula:

[0054] in, Indicates the direction matching degree, with a value range of 100%. A higher value indicates a better match of directional features; This represents the total number of binary bits of the direction code involved in the comparison, and its value is the product of the total number of valid pixels in the texture direction code map and the number of binary bits per pixel. This represents the first step in the texture direction code sequence extracted from the original image fragment. The binary value of a bit. The direction code sequence corresponding to the defect type in the defect knowledge graph is represented by the first... The binary value of a bit; The XOR operation represents the operation where the two bits are the same and the result is 0, and the result is 1 if they are different; the SUM operator iterates through all bits. One binary bit. Direction matching degree. As a component of the third confidence score, it is combined with the matching degree based on other parts such as shape context features to form the complete third confidence score.

[0055] The contours of the defect region are extracted from the defect layer separated by multi-scale directional filtering. Adaptive thresholding is performed on the strongest energy map retained after local maximum suppression. Using the global mean of the energy map as the initial threshold, the average gray levels of the foreground and background are iteratively calculated and the threshold is updated until convergence. Pixels with energy higher than the final threshold are designated as foreground (defect), and the remaining pixels are designated as background, resulting in a binary defect layer. The defect layer is a binary image, where foreground pixels correspond to defect locations. Canny edge detection is performed on the defect layer to obtain a set of contour points for the defect region. The contour point set is uniformly sampled with a sampling interval of 2 pixels to obtain a set of contour sampling points. The total number of sampling points is denoted as [missing information]. For each sampling point , Calculate its shape context features. The shape context features describe the sampling points. Compared to the rest The spatial distribution of the sampling points. Specifically, with A polar coordinate system is established with the origin, dividing the polar coordinate space into 5 intervals along the logarithmic distance direction and 12 intervals along the angular direction, for a total of 60 intervals. For each of the remaining sampling points... , Calculate its relative to distance and angle ,according to and Voting was conducted within the respective intervals to obtain... 60-dimensional shape context histogram After calculating the shape context histogram for all sampling points, the shape context feature set of the defect region is obtained. In the defect knowledge graph, each defect type node stores a set of typical shape context features corresponding to that defect type. This set of typical features is obtained through hierarchical clustering of the shape context features of a large number of historically confirmed samples of that defect type, denoted as . ,in This represents the typical number of sampling points. The shape context feature set of the defect region... Typical shape context feature sets corresponding to the defect types in the knowledge graph Perform a match. For Each histogram in Calculate its relationship with Each histogram Chi-square distance:

[0056] in, This is a histogram interval index, with values ​​ranging from 1 to 60. The minimum constant takes the value of Prevent division by zero. (The result is...) The cost matrix is ​​obtained. The Hungarian algorithm is used to solve for the optimal matching of the cost matrix, yielding the minimum matching cost. Convert the matching cost to shape context matching degree:

[0057] in, This is a normalization constant, and its value is the standard deviation of all historical matching costs. The range of values ​​is A value closer to 1 indicates a better shape match. Overall directional matching degree. Shape context matching degree The third confidence level is obtained:

[0058] in, and These are the weighting coefficients. , The value is 0.4. The value is 0.6.

[0059] In some implementations, texture orientation codes are extracted only from edge regions within the original image segment whose gradient magnitude is greater than a certain threshold, in order to reduce the interference of noise in flat regions on the orientation codes. In this case, only pixels with responses greater than the threshold are included in the calculation. The value decreases accordingly, and Hamming distance calculation is only performed within the effective area.

[0060] Example 4:

[0061] In practice, after the cloud completes the calculation of the third confidence level, it obtains the first confidence level generated by the edge, the second confidence level generated by the edge, and the third confidence level generated by the cloud itself. The comprehensive confidence level is generated by the weighted sum of the three confidence levels and their corresponding weights.

[0062] Define a first weight, a second weight, and a third weight. The first weight is proportional to the illumination intensity during edge image acquisition. The illumination intensity during edge image acquisition is measured by the illumination sensor mounted on the edge device while acquiring images of the industrial equipment surface, and the unit of measurement is lux. The illumination intensity value is denoted as... The first weight is denoted as Then the expression is:

[0063] The calculation yielded the following result. The denominator constant of 500 is set based on the following: when the illuminance is 500 lux, this lighting condition is considered a standard industrial inspection lighting environment. At this time, the first weight value is 0.5, which keeps the weight at a moderate level under typical lighting conditions, facilitating balance with other weights. When the illuminance increases, The increase in this value indicates that, under sufficient illumination, the confidence contribution of the end-side phase correlation value should be strengthened, and its first confidence level has a correspondingly greater influence in the overall decision; when the illumination intensity decreases... The corresponding decrease indicates that the reliability of end-side detection results decreases under insufficient lighting conditions, and its contribution should be weakened. First weight. The value range is between 0 and 1.

[0064] The second weight is proportional to the area of ​​the edge candidate region. The area of ​​the edge candidate region refers to the total number of pixels in the original image region corresponding to the candidate region descriptor uploaded from the edge, denoted as . The second weight is denoted as , by expression The calculation shows that 6400 pixels corresponds to a typical defect area of ​​80 pixels × 80 pixels. This value is determined based on the typical size distribution of common defects on the surface of industrial equipment, such as cracks and corrosion spots. When the area of ​​the candidate edge region is 6400 pixels, the second weight... It equals 1; the larger the area, the more... The larger the area, the higher the probability that the large area contains a real defect, and the greater the contribution of the second confidence level on the edge to the overall decision; the smaller the area, the higher the probability that the large area contains a real defect. The smaller.

[0065] The third weight is proportional to the coverage time of the cloud-based defect knowledge graph. The coverage time of the cloud-based defect knowledge graph refers to the number of days between the earliest recorded sample's timestamp and the current system time for the device part and defect type of the currently uploaded segment, denoted as... The third weight is denoted as , by expression The calculation shows that 365 days corresponds to a one-year accumulation period, and this value is determined based on the update frequency of industrial inspection data and the sufficiency of defect type samples accumulating over time. When the knowledge graph coverage time is 365 days, the third weight... Equals 1; the longer the coverage time, The larger the value, the richer the experience in matching the corresponding defect patterns in the knowledge graph, and the more reliable the contribution of the cloud-based third-party confidence to the overall decision; the shorter the coverage time, the better. The smaller.

[0066] Then calculate the sum of the first weight, the second weight, and the third weight. Divide each weight by the sum to obtain the normalized weights: , , The normalized weights satisfy... And each normalization weight is between 0 and 1.

[0067] Calculate the first confidence level with normalized first weight The product, second confidence level With normalized second weight The product of the third confidence level With normalized third weight The product of the three products is used to obtain the overall confidence level. The formula for calculating the overall confidence level is:

[0068] at this time The value range is [0,1]. Overall confidence level After calculation, the overall confidence level is compared with the preset alarm threshold. The alarm threshold is preset to 0.85. This setting was determined through backtesting of historically labeled defect samples, so that areas with an overall confidence level higher than 0.85 have a very high probability of defect confirmation and can directly trigger maintenance actions.

[0069] When the overall confidence level When the alarm threshold of 0.85 is reached, the defect region is extracted from the defect layer separated during the multi-scale directional filtering process in the cloud. The defect layer is a binary image, where foreground pixels correspond to defect locations. All connected components are searched within the defect layer, and the connected component with the largest area is selected as the main defect region. The centroid coordinates of the main defect region are calculated, and the x-coordinate of the centroid coordinates is... The result is obtained by the arithmetic mean of the x-coordinates of all foreground pixels within the defect area, and the y-coordinate... The result is obtained by the arithmetic mean of the ordinates of all foreground pixels within the defect area. The minimum bounding rectangle of the main defect area is calculated, and its width and height are used as the geometric dimensions of the defect area. The centroid coordinates and geometric dimensions, along with the equipment part identification and inspection timestamp, are assembled into a maintenance work order record and pushed to the maintenance management system via a cloud interface. The maintenance work order contains the spatial coordinates and geometric extent of the defect location, allowing maintenance personnel to directly locate and handle it.

[0070] In some implementations, when there are multiple connected regions with similar areas in the defect layer, the centroid coordinates and geometric dimensions of each connected region whose area exceeds a preset area threshold are extracted, multiple maintenance work orders are generated, and the relative position index of the defect region is marked in each work order.

[0071] When the overall confidence level Between the alarm threshold of 0.85 and the review threshold, the review threshold is set to 0.55, and this area is re-analyzed by a high-precision visual algorithm. When the overall confidence level is lower than the review threshold of 0.55, all original image data of the corresponding area and intermediate results at all levels are transferred to the human expert review queue for final judgment by humans.

[0072] See Figure 5 In the graph, the horizontal axis represents the detection area number, ranging from 0 to 510, and the vertical axis represents the overall confidence level, ranging from 0 to 1. The graph uses a continuous black line to represent the overall confidence level value calculated for each detection area. The red dashed line represents the preset alarm threshold of 0.85, and the blue dotted line represents the preset review threshold of 0.55.

[0073] As can be seen from the curve trend, the overall confidence level fluctuates greatly among the detection areas, with the values ​​mainly concentrated between 0.3 and 1.0. Most of the overall confidence level values ​​are within the range between the review threshold of 0.55 and the alarm threshold of 0.85. The overall confidence level in a few areas exceeds the alarm threshold, indicating a higher probability of defect confirmation.

[0074] According to Example 4, the overall confidence level is calculated by weighting the first confidence level from the edge side, the second confidence level from the side edge, and the third confidence level from the cloud side, with corresponding weights. The overall confidence level of each detection area reflects the comprehensive judgment result of the presence of defects in that area. In the figure, when the overall confidence level is greater than 0.85, the system will directly trigger the generation of a maintenance work order; when the overall confidence level is between 0.55 and 0.85, the system will use a high-precision visual algorithm for review and analysis; areas below 0.55 will be transferred to the human expert review queue.

[0075] Example 5:

[0076] In practice, after the first confidence level is calculated on the edge, spatial gradient direction encoding is performed on image blocks with a first confidence level higher than the first threshold to generate candidate region descriptors.

[0077] Calculate the gradient magnitude and direction for each pixel within the image patch. For a pixel at coordinate position... The gradient magnitude of the pixel. and gradient direction The calculation method is as follows: first calculate the horizontal grayscale difference. and vertical grayscale difference ,in Indicates the image patch in coordinates The grayscale value at the location; then the gradient magnitude. gradient direction , This is the arctangent function in the four quadrants, and its return value range is... After calculating the gradient magnitude of all pixels within the image block, pixels with a gradient magnitude greater than a fourth preset value are retained. The fourth preset value is 0.8 times the average gradient magnitude of all pixels within the image block. The reason for using 0.8 times the average grayscale gradient magnitude as the fourth preset value is to retain foreground texture pixels with strong gradients while filtering out noise pixels in flat areas with weak gradients. This 0.8 factor strikes a balance between suppressing background noise and retaining weak defect edges.

[0078] From the retained pixels, the pixel with the largest gradient magnitude is selected as the starting point for chain code encoding. If multiple pixels have the same maximum gradient magnitude, the one with the smallest lexicographical order is selected. For the selected starting point, the next adjacent retained pixel is searched within its 8-neighborhood according to the 8-neighborhood directional chain code rule. The 8-neighborhood directional chain code rule defines the 0-degree direction as the right adjacent pixel, the 45-degree direction as the top-right adjacent pixel, the 90-degree direction as the top-right adjacent pixel, the 135-degree direction as the top-left adjacent pixel, the 180-degree direction as the left-right adjacent pixel, the 225-degree direction as the bottom-left adjacent pixel, the 270-degree direction as the bottom-right adjacent pixel, and the 315-degree direction as the bottom-right adjacent pixel. The direction code values ​​are represented by integers from 0 to 7. The search method is as follows: within the 8-neighborhood of the current pixel, each adjacent position is checked in ascending order of direction code value. The first retained pixel checked is selected as the next encoding node, and the direction code value from the current pixel to the next encoding node is recorded. The encoded pixels are marked as visited. Then, using the newly found adjacent reserved pixels as the current point, the search continues within the 8-neighborhood for the next unvisited reserved pixel. This search and direction code recording operation is repeated until there are no unvisited reserved pixels within the 8-neighborhood of the current pixel. All the direction code values ​​recorded in sequence constitute the initial chain code.

[0079] Redundancy removal is performed on the initial chain code. The redundancy removal process involves scanning the direction code sequence in the initial chain code and merging consecutively repeating identical direction codes into a single direction code. For example, if the direction code sequence is "0,0,1,1,1,2", the merged compressed chain code will be "0,1,2". The occurrence of consecutive identical direction codes corresponds to the edges of straight line segments extending in the same direction in the image. The merging operation shortens the chain code length without losing direction change information. The resulting compressed chain code is then used as a candidate region descriptor and packaged for submission to the edge detection system. The candidate region descriptor also includes the top-left corner coordinates of the image patch in the original surface image and the width and height information of the image patch, allowing the edge detection system to locate the corresponding original image region.

[0080] In some implementations, when the connected path formed by the retained pixels has branches, after the search for the chain code encoding starting point is completed, it is checked whether there are any unvisited retained pixels. If so, a new chain code encoding starting point is selected for each isolated connected branch, and multiple compressed chain codes are generated. All compressed chain codes are concatenated and uploaded as candidate region descriptors for the image block.

[0081] Regions with overall confidence levels between the alarm threshold and the review threshold are re-analyzed using high-precision vision algorithms. The corresponding region is cropped from the complete original image stored in the cloud. The complete original image stored in the cloud is an image of the industrial equipment surface that has not undergone segmentation and is synchronously uploaded after being acquired at the edge. Based on the spatial coordinates of the corresponding region, a sub-image of the corresponding pixel range is extracted from the complete original image.

[0082] The extracted sub-image undergoes color space conversion. The sub-image is converted from the RGB color space to the HSV color space. In the HSV color space, the H channel represents the hue component, the S channel represents the saturation component, and the V channel represents the lightness component. The lightness component is extracted from the three HSV channels and denoted as... The saturation component in the saturation component is separated and denoted as... .

[0083] Based on luminance component Calculate the local binary pattern texture spectrum. For each pixel in the luminance component, take a region centered at that pixel with a radius of... On the circumference of a pixel Equally spaced neighborhood sampling points, The value is 1 pixel. The value is taken from 8 sampling points. The brightness values ​​of each neighboring sampling point are compared with the brightness value of the center pixel. If the brightness value of a neighboring sampling point is greater than or equal to the brightness value of the center pixel, the corresponding binary bit of that sampling point is set to 1; otherwise, it is set to 0. This is done in a fixed order. A binary digit is combined to form a... The binary number is converted to decimal to obtain the local binary mode value of that pixel. After calculating the local binary mode values ​​for all pixels in the luminance component, a histogram of the local binary mode values ​​is plotted. The histogram contains a total of... For each interval, the frequency of the corresponding local binary pattern value in the luminance component is statistically analyzed. This 256-dimensional histogram vector is used as the local binary pattern texture spectral feature vector, denoted as... .

[0084] Based on the saturation component of the color component Calculate the saturation histogram. Determine the range of values ​​for the saturation components. The saturation component is divided into 32 equal intervals, each with a width of 1 / 32. The number of pixels whose saturation values ​​fall within each interval is counted, forming a 32-dimensional saturation histogram vector, denoted as [vector]. .

[0085] Local binary pattern texture spectrum feature vector and saturation histogram vector The concatenation is performed to obtain the concatenated feature vector. The total dimension of the concatenated feature vector is 256 + 32 = 288.

[0086] The cosine similarity between the concatenated feature vector and the feature vector of the corresponding defect type in the defect knowledge graph is calculated. Each defect type node in the defect knowledge graph stores a typical feature vector corresponding to that defect type. This typical feature vector is also 288-dimensional and is obtained by the arithmetic mean of the concatenated feature vectors of a large number of historically confirmed samples of that defect type, denoted as . .

[0087] The formula for calculating cosine similarity is:

[0088] in, This represents the calculated cosine similarity value, with a range of values ​​of 1000. A higher value indicates a stronger directional consistency between the two eigenvectors; This represents the total dimension of the concatenated feature vectors. ; This represents the first concatenated feature vector extracted from the current processing region. Element values ​​in each dimension; The typical feature vector of the corresponding defect type in the defect knowledge graph. The element values ​​of each dimension; the summation operation in the numerator calculates the inner product of two vectors; the denominator... To concatenate feature vectors for the current region Norm, For the typical feature vectors of the defect knowledge graph The norm, the product of the two, serves as a normalization factor, transforming the inner product into the form of cosine distance.

[0089] Cosine similarity Compare with the fifth preset value. The fifth preset value is set at 0.82. The basis for setting the fifth preset value to 0.82 is: when backtesting on historically labeled samples, when the cosine similarity is higher than 0.82, the visual features of the current region are highly consistent with the corresponding defect type in the knowledge graph, the precision of defect confirmation reaches the preset acceptable level for engineering, and the recall rate is controlled within a reasonable range.

[0090] When the cosine similarity exceeds the fifth preset value of 0.82, the current area is determined to be a confirmed defect, and a maintenance work order containing the defect location, geometric dimensions, and equipment part identification is generated and pushed to the maintenance management system via the cloud interface. When the cosine similarity does not exceed the fifth preset value of 0.82, it is marked as a suspected defect, and the original image data of the current area, confidence values ​​at all levels, and intermediate analysis results are packaged and added to the human expert review queue. Human experts make the final judgment through the review terminal.

[0091] See Figure 6 In the figure, the horizontal axis represents the verification area number, ranging from 0 to 500, and the vertical axis represents the cosine similarity value, ranging from 0 to 1. The figure shows the cosine similarity curves corresponding to three types of defects: crack cosine similarity (blue solid line), corrosion cosine similarity (orange dashed line), and scratch cosine similarity (green dotted line). The fifth preset value of 0.82 set in Example 5 is marked with a red dotted line.

[0092] As can be seen from the figure, the crack cosine similarity curves are generally distributed above 0.8, and the similarity of multiple regions reaches 1.0, which is significantly higher than the fifth preset value of 0.82. This indicates that the visual features of the crack category identified in the verification area are highly consistent with the typical features in the defect knowledge graph, which meets the judgment criteria of confirming a defect by using a cosine similarity higher than 0.82 in Example 5.

[0093] The corrosion cosine similarity curve fluctuated between 0.6 and 0.8. Although some areas were close to or slightly higher than the preset value of 0.8, most areas did not exceed the threshold of 0.82, which means that the visual features of corrosion defects and the knowledge graph matched relatively poorly, and there was some uncertainty in recognition.

[0094] The overall distribution of the scratch cosine similarity curve is low, with most data points located in the range of 0.4 to 0.7, which is below the 0.82 judgment threshold. This indicates that the visual features of scratch-type defects have a low matching degree with the typical features of the knowledge graph, and further manual expert review is required to confirm the defect status.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios, characterized in that, Includes the following steps: The surface image of the industrial equipment is acquired at the end, and the image is divided into blocks. The phase correlation value between each image block and the preset defect template is calculated to obtain the first confidence level. The edge encodes image blocks with a first confidence level higher than the first threshold into chain code sequences along the spatial gradient direction, generates candidate region descriptors, and uploads them to the edge. After receiving the candidate region descriptor, the corresponding original image region is subjected to anisotropic diffusion filtering to obtain the enhanced region image. Then, the local contrast-weighted histogram of the region image is extracted and compared with the structural similarity of a typical texture library to obtain the second confidence score. The regions with a second confidence level lower than the second threshold, along with the original image fragments, are uploaded to the cloud. The cloud performs multi-scale directional filtering on the uploaded image segments to separate the defect layer, extracts the shape context features of the defect layer, and matches them with the defect knowledge graph constructed offline to obtain the third confidence level; The cloud generates a comprehensive confidence score based on the weighted sum of the first, second, and third confidence scores. Areas with a comprehensive confidence score higher than the alarm threshold are directly generated as maintenance work orders. Areas with a comprehensive confidence score between the alarm threshold and the review threshold are re-analyzed by a high-precision visual algorithm. Areas with a comprehensive confidence score lower than the review threshold are transferred to the human expert review queue.

2. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The calculation of the phase correlation value between each image block and the preset defect template includes: The acquired surface image is divided into a fixed-size rectangular grid, with each grid serving as an independent image block; A Hanning window is applied to each image patch to suppress spectral leakage, and then a two-dimensional fast Fourier transform is performed to obtain the complex spectrum; Calculate the amplitude spectrum and phase spectrum of the complex spectrum, and normalize the phase spectrum with the principal component direction as the reference to obtain the normalized phase matrix; The phase correlation value is calculated element-wise between the normalized phase matrix and the phase matrix of the preset defect template, and the phase correlation value is output.

3. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The anisotropic diffusion filtering of the corresponding original image region includes: Construct a structure tensor for each pixel, calculate the two eigenvalues ​​and the corresponding eigenvector of the structure tensor, where the eigenvector corresponding to the larger eigenvalue indicates the texture direction; Define the diffusion tensor as a directional diagonal matrix, set the diffusion coefficient along the texture direction as a first preset value, set the diffusion coefficient perpendicular to the texture direction as a second preset value, and set the second preset value as less than the first preset value; The original image region is updated using an iterative difference format. After each iteration, the mean value of the gray-level gradient magnitude of the region is calculated. The iteration is terminated when the rate of change of the mean value is lower than a third preset value, and the enhanced region image is output.

4. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The multi-scale directional filtering of the uploaded image segments includes: Construct a set of two-dimensional Gabor filters with different scales and orientations. The center frequency of the filters is uniformly distributed along the logarithmic coordinates, and the orientation angle is sampled at equal intervals within the range of 0 to 180 degrees. Each filter is convolved with the image segment to obtain a complex response, and then the square of the response amplitude is taken as the energy map at that scale direction; Local maxima suppression is performed on all energy maps, preserving the scale and direction combination with the strongest response, and the phase information of the corresponding direction is used as the texture direction code; The Hamming distance between the texture direction code and the direction code in the defect knowledge graph is calculated to obtain the direction matching degree, which is used as a component of the third confidence degree.

5. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The step of generating a comprehensive confidence score based on a weighted sum of the first confidence score, the second confidence score, and the third confidence score includes: Set a first weight, a second weight, and a third weight, where the first weight is proportional to the illumination intensity during edge image acquisition, the second weight is proportional to the area of ​​the edge candidate region, and the third weight is proportional to the coverage time of the cloud-based defect knowledge graph. Calculate the sum of the first weight, the second weight, and the third weight, and divide each weight by the sum to obtain the normalized first weight, the normalized second weight, and the normalized third weight; calculate the product of the first confidence level and the normalized first weight, the product of the second confidence level and the normalized second weight, and the product of the third confidence level and the normalized third weight, and add the three products to obtain the overall confidence level; When the overall confidence level reaches the alarm threshold, the centroid coordinates and geometric dimensions of the defect area are extracted, and a maintenance work order containing location and size information is generated.

6. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The endpoint encodes image blocks with a first confidence level higher than a first threshold into chain code sequences along the spatial gradient direction, including: Calculate the gradient magnitude and gradient direction of each pixel in each image block, and retain pixels with a gradient magnitude greater than a fourth preset value; Starting from the pixel with the largest gradient magnitude, the relative directions of adjacent retained pixels are recorded in sequence according to the 8-neighborhood direction chain code rule to form the initial chain code; The initial chain code is deredundant, and consecutive identical direction codes are merged into a single direction code to obtain a compressed chain code, which serves as the candidate region descriptor.

7. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 4, characterized in that, After the orientation angle is sampled at equal intervals within the range of 0 to 180 degrees, the following steps are also performed: Based on the spatial location of the region where the second confidence level is lower than the second threshold uploaded from the side, the distribution of common defect texture directions of the device part where the region is located is statistically analyzed; Using the statistically obtained texture direction distribution as prior information, the equally spaced sampling directions from 0 to 180 degrees are reweighted, increasing the filter weight for common defect directions and decreasing the filter weight for non-defect directions. The multi-scale directional filtering is re-executed using the weighted filter set.

8. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 2, characterized in that, The phase matrix of the preset defect template is obtained through an offline calibration process: Collect sample images containing typical defects, divide each sample image into blocks according to the fixed size, and retain only the image blocks containing defects; Perform a two-dimensional fast Fourier transform on the retained image patch and extract the phase spectrum, then align all phase spectra along the same principal component direction; The arithmetic mean of the aligned phase spectrum is calculated to obtain the average phase matrix, which is then used as the phase matrix of the preset defect template.

9. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 3, characterized in that, The method involves updating the original image region using an iterative difference format. After each iteration, the mean value of the gray-level gradient magnitude of the region is calculated. The iteration terminates when the rate of change of the mean value is lower than a third preset value. Specifically, this includes: After each diffusion iteration, the arithmetic mean of the gray-level gradient magnitudes of all pixels in the current iteration image is calculated. Subtract the arithmetic mean of the current iteration from the arithmetic mean of the previous iteration to obtain the difference, and then divide the difference by the arithmetic mean of the previous iteration to obtain the rate of change; The iteration stops when the rate of change is less than a third preset value, and the current iterated image is output as the enhanced region image.

10. The cloud-edge-device three-level AI visual inspection method for industrial inspection scenarios according to claim 1, characterized in that, The region where the overall confidence level falls between the alarm threshold and the review threshold is re-analyzed by a high-precision visual algorithm, including: The corresponding area is cropped from the complete original image stored in the cloud, and the color space of the area is converted to separate the luminance component and chrominance component; Local binary pattern texture spectrum is calculated based on luminance component, and saturation histogram is calculated based on chrominance component. The local binary pattern texture spectrum and saturation histogram are concatenated to form a feature vector, and the cosine similarity is calculated with the feature vector of the corresponding defect type in the defect knowledge graph. If the cosine similarity exceeds the fifth preset value, it is determined to be a confirmed defect and a work order is generated; otherwise, it is marked as a suspected defect and transferred to the human expert review queue.

Images