Fully automatic engine visual inspection equipment and method

Abstract

The present application provides a fully automatic engine visual inspection device and method, which can collect multi-angle polarization images of the target area of ​​the engine; determine the polarization degree and main polarization direction of each pixel in the multi-angle polarization image; convert the polarization degree of all pixel points into a spectrum distribution diagram, and then perform decoupling analysis on the spectrum distribution diagram to obtain the spectrum disturbance energy of the spectrum distribution diagram; generate a crack response heat map of the target area through the main polarization direction and the spectrum disturbance energy combined with a convolutional neural network algorithm, and then generate a structural perception tensor of crack defects in different crack areas based on the response intensity distribution of all pixels in the crack response heat map; automatically locate and detect crack defects on the target area of ​​the engine based on all structural perception tensors combined with crack perception thresholds. Using the solution of the present application, the contrast of microcracks can be improved based on the reflective characteristics of the material surface.

Description

Technical Field

[0001] The present application relates to the field of visual inspection technology, and more specifically, to a fully automatic engine visual inspection device and method. Background Art

[0002] Visual inspection is a key method based on image acquisition and image processing for non-contact, automated identification and analysis of the surface status, structural features or defect information of the object being tested. It is widely used in industrial manufacturing, medical diagnosis, security monitoring and other fields. At the same time, visual inspection is also highly scalable and can be embedded in automated production lines to achieve multifunctional integrated processing such as defect location, classification, measurement, and early warning, promoting the upgrade of traditional inspection methods to integrated intelligent perception and decision-making.

[0003] Fully automatic engine visual inspection refers to an intelligent inspection system that uses advanced machine vision technology to achieve high-precision quality control of the entire engine production process. It uses industrial cameras, optical lenses and image acquisition cards as its hardware foundation, and is equipped with deep learning algorithms and image processing software to build a complete inspection system from data acquisition to defect judgment. Fully automatic engine visual inspection not only greatly improves inspection efficiency and accuracy, but also reduces labor costs and subjective errors, effectively ensuring the quality of engine products. Various components of the engine (such as the housing, cylinder head, and pipeline connection areas) are very prone to microcrack defects when operating at high temperature / high pressure. However, the surface of the engine housing forms a complex background due to oil and carbon deposits. On this basis, traditional vision-based algorithms such as fixed threshold segmentation and edge detection extract image features based on preset rules, which are difficult to adapt to the background noise generated by the complex background, and thus cannot effectively enhance the contrast of microcracks. Therefore, how to improve the contrast of microcracks based on the reflective characteristics of the material surface has become a difficult problem facing the industry. Summary of the Invention

[0004] The present application provides a fully automatic engine visual inspection device and method, which can improve the contrast of microcracks based on the reflective properties of the material surface.

[0005] In a first aspect, the present application provides a fully automatic engine visual inspection method, comprising the following steps:

[0006] Automatically collect a multi-angle polarization image of the target area of ​​the engine, wherein the multi-angle polarization image includes multiple polarization directions;

[0007] Determining the polarization degree and main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions;

[0008] Converting the polarization degree of all pixels into a spectrum distribution diagram of the material surface under polarized reflection, and then performing a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram;

[0009] Based on a convolutional neural network algorithm, a response simulation of crack defects in a target area is performed in combination with all main polarization directions and the spectrum disturbance energy to obtain a crack response heat map, and then a structural perception tensor of different crack areas in the crack response heat map is generated through the response intensity distribution of all pixels in the crack response heat map;

[0010] Crack defects in the target area of ​​the engine are automatically located and detected based on all structure perception tensors combined with crack perception thresholds.

[0011] In some embodiments, determining the polarization degree and the main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions specifically includes:

[0012] Determine the polarization intensity difference of each pixel in different orthogonal directions based on the orthogonal relationship between different polarization directions;

[0013] The polarization degree and main polarization direction of each pixel point in the multi-angle polarization image are determined by all polarized light intensity differences.

[0014] In some embodiments, converting the polarization degrees of all pixels into a spectrum distribution diagram of the material surface under polarized reflection specifically includes:

[0015] Construct a two-dimensional polarization distribution map through the polarization degrees of all pixels;

[0016] Performing spectrum transformation on the two-dimensional polarization distribution diagram to obtain a frequency domain complex response diagram;

[0017] A frequency spectrum distribution diagram of the material surface under polarized reflection is determined according to the frequency domain complex response diagram.

[0018] In some embodiments, performing a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram specifically includes:

[0019] extracting a high-frequency mask area from the frequency spectrum distribution map based on the high-frequency response characteristics of the crack defect on the material surface;

[0020] extracting amplitude response information of the frequency spectrum distribution diagram in the high-frequency mask area;

[0021] The spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram is determined according to the amplitude response information.

[0022] In some embodiments, the crack response heat map is obtained by simulating the response of the crack defect in the target area based on the convolutional neural network algorithm in combination with all main polarization directions and the spectrum disturbance energy, specifically including:

[0023] Construct all main polarization directions into a directional characteristic map;

[0024] Constructing a multi-channel fusion tensor by combining the directional feature map with the spectrum disturbance energy;

[0025] The multi-channel fusion tensor is input into the convolutional neural network algorithm to simulate the crack defects in the target area, thereby obtaining a crack response heat map.

[0026] In some embodiments, generating structural perception tensors of different crack regions in the crack response heat map through the response intensity distribution of all pixels in the crack response heat map specifically includes:

[0027] Determine the response intensity distribution of all pixels in the crack response heat map;

[0028] Performing stress fitting based on the response intensity distribution to obtain multiple crack stress trajectories;

[0029] The structural perception tensors of different crack regions in the crack response heat map are determined according to the stress trajectories of the cracks.

[0030] In some embodiments, automatically locating and detecting crack defects on a target area of ​​an engine based on all structure perception tensors combined with a crack perception threshold specifically includes:

[0031] Determine the perception tension value of the crack defect in the target area of ​​the engine through all structural perception tensors;

[0032] Preset the crack perception threshold of the target area;

[0033] Crack defects on a target area of ​​an engine are automatically located and detected based on the crack perception threshold and the perceived tension value.

[0034] In some embodiments, the target area includes the surface of the engine casing, the joint of the cylinder head, and the transition area between the heat sink and the frame.

[0035] In some embodiments, the polarization direction refers to the angle formed by the electric vector of the light wave and the perpendicular direction of the light propagation direction when the polarized light source illuminates the target area, including 0°, 45°, 90° and 135°.

[0036] In a second aspect, the present application provides a fully automatic engine visual inspection device, comprising:

[0037] An acquisition module, configured to automatically acquire a multi-angle polarization image of a target area of ​​an engine, wherein the multi-angle polarization image includes multiple polarization directions;

[0038] a processing module, configured to determine the degree of polarization and the main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions;

[0039] The processing module is further configured to convert the polarization degrees of all pixel points into a spectrum distribution diagram of the material surface under polarized reflection, and then perform a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram;

[0040] The processing module is further configured to simulate the response of crack defects in the target area based on a convolutional neural network algorithm in combination with all main polarization directions and the spectrum disturbance energy to obtain a crack response heat map, and then generate structural perception tensors of different crack areas in the crack response heat map through the response intensity distribution of all pixels in the crack response heat map;

[0041] The execution module is used to automatically locate and detect crack defects on the target area of ​​the engine based on all structural perception tensors combined with crack perception thresholds.

[0042] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0043] In the fully automatic engine visual inspection equipment and method provided in the present application, a multi-angle polarization image of the target area of ​​the engine is automatically collected, and the multi-angle polarization image includes multiple polarization directions; the polarization degree and main polarization direction of each pixel point in the multi-angle polarization image are determined based on the orthogonal relationship between different polarization directions; the polarization degree of all pixel points is converted into a spectral distribution diagram of the material surface under polarized reflection, and then the spectral distribution diagram is decoupled and analyzed based on the high-frequency response characteristics of the crack defects on the material surface to obtain the spectral disturbance energy of the crack defects in the preset high-frequency mask area of ​​the spectral distribution diagram; based on the convolutional neural network algorithm, the response simulation of the crack defects in the target area is combined with all main polarization directions and the spectral disturbance energy to obtain a crack response heat map, and then the structural perception tensors of different crack areas in the crack response heat map are generated through the response intensity distribution of all pixel points in the crack response heat map; the crack defects on the target area of ​​the engine are automatically located and detected according to all structural perception tensors combined with the crack perception threshold.

[0044] It can be seen that in the present application, the polarization degree of each pixel in the multi-angle polarization imaging reflects the reflection difference of the material surface to light of different polarizations; the present application maps the subtle surface structure differences in the spatial domain to the frequency domain by obtaining a spectral distribution diagram after performing a two-dimensional Fourier transform on the polarization degree image. Sharp edges and small disturbances such as cracks will show significant energy in the high-frequency region; by presetting the high-frequency mask area, only the amplitude spectrum (spectral disturbance energy) of this frequency band is retained, thereby effectively suppressing the overall low-frequency illumination, shadow and background texture interference, and only highlighting the high-frequency features brought by microcracks, thereby achieving contrast enhancement of cracks in the frequency domain. The spectral disturbance energy not only quantifies the degree of existence of cracks, but also numerically amplifies the difference between microcracks and the surrounding background, providing clearer and more stable input features for subsequent deep learning or threshold segmentation. This enables fine cracks that are difficult to detect in the traditional spatial domain to be reliably identified in visual inspection; subsequently, the present application simulates the response of crack defects in the target area by combining all main polarization directions and the spectral disturbance energy through a convolutional neural network algorithm to obtain a crack response heat map, and thus constructs a multidimensional tensor through the crack response heat map, which contains information such as the pixel coordinates, response amplitude, polarization direction distribution and local texture gradient of the area (structure perception tensor), thereby integrating the sensitivity of the spectral energy to high-frequency disturbances, so that the neural network and subsequent algorithms can distinguish between true crack paths and isolated noise or speckle interference in multidimensional space; through this tensorization feature, the significance of the crack contrast is improved, and the robustness to noise and complex background is enhanced; in summary, this scheme can improve the contrast of microcracks based on the reflective characteristics of the material surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a fully automatic engine visual inspection method according to some embodiments of the present application;

[0046] Figure 2 is a schematic diagram of a process for determining a spectrum distribution diagram according to some embodiments of the present application;

[0047] Figure 3 is a schematic diagram of a process for realizing crack defect judgment according to some embodiments of the present application;

[0048] Figure 4 is a schematic structural diagram of a fully automatic engine visual inspection device according to some embodiments of the present application;

[0049] Figure 5 This is a diagram of the internal structure of a computer device for implementing a fully automatic engine visual inspection method according to some embodiments of the present application. DETAILED DESCRIPTION

[0050] In order to better understand the technical solution in this embodiment, the technical solution in this embodiment will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] refer to Figure 1 , which is a flow chart of a fully automatic engine visual inspection method according to some embodiments of the present application. The fully automatic engine visual inspection method mainly includes the following steps:

[0052] In step 101 , a multi-angle polarization image of a target area of ​​an engine is automatically collected, wherein the multi-angle polarization image includes multiple polarization directions.

[0053] In specific implementation, the polarization imaging acquisition system can be used to emit light wave electric vectors to sequentially polarize and synchronously image local areas of the engine. The system sequentially controls the linearly polarized light source to illuminate the target area under multiple preset polarization directions, and the industrial grayscale camera collects the corresponding reflected image under each polarization direction; the image data collected under each polarization direction are aligned and fused in the pixel space to construct a single multi-angle polarization image. The multi-angle polarization image includes multiple polarization directions, and each pixel point of the multi-angle polarization image contains the grayscale value of the spatial position under each polarization direction.

[0054] It should be noted that the polarization imaging acquisition system described in this application refers to a hardware device that can obtain surface reflection images of the target area under different polarization directions, specifically including: a four-channel linear polarization light source, an industrial grayscale camera equipped with a switchable polarization filter, a synchronous trigger controller and a positioning bracket; the multi-angle polarization image refers to an image sequence acquired in sequence under multiple linear polarization directions at a fixed observation position; the target area includes the engine casing surface, the cylinder head seam, and the transition area between the heat sink and the skeleton; the multi-angle polarization image refers to a multi-angle image formed by irradiating the target area in multiple polarization directions at a fixed observation position and collecting reflection images, and then fusing multiple polarization direction images at the pixel layer based on the principle of spatial position alignment; the polarization direction refers to the angle formed by the electric vector of the light wave and the perpendicular direction of the light propagation direction when the polarized light source irradiates the target area, including 0°, 45°, 90° and 135°, which will not be repeated here.

[0055] In step 102, the polarization degree and main polarization direction of each pixel in the multi-angle polarization image are determined based on the orthogonal relationship between different polarization directions.

[0056] In some embodiments, determining the polarization degree and the main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions can be achieved by using the following steps:

[0057] Determine the polarization intensity difference of each pixel in different orthogonal directions based on the orthogonal relationship between different polarization directions;

[0058] The polarization degree and main polarization direction of each pixel point in the multi-angle polarization image are determined by all polarized light intensity differences.

[0059] In specific implementation, the polarization intensity difference of each pixel point in different orthogonal directions is determined based on the orthogonal relationship between different polarization directions. The following method is adopted, namely: first, a pixel point is selected, and the grayscale values ​​of the pixel point at the same position in the multi-angle polarization image at polarization angles of 0 degrees and 90 degrees are used as the light intensity values ​​of the pixel point in the first group of orthogonal directions; at the same time, the grayscale values ​​at polarization angles of 45 degrees and 135 degrees are used as the light intensity values ​​of the pixel point in the second group of orthogonal directions; then, for each pixel point, its light intensity difference in the first group of orthogonal directions is calculated, that is, the difference between the grayscale values ​​of the pixel point in the 0 degree and 90 degree images, as the polarization intensity difference of the pixel point in the first orthogonal direction; then, its light intensity difference in the second group of orthogonal directions is calculated, that is, the difference between the grayscale values ​​of the pixel point in the 45 degree and 135 degree images, as the polarization intensity difference of the pixel point in the second orthogonal direction, and the above steps are repeated to determine the polarization intensity differences corresponding to the remaining pixel points, thereby obtaining the polarization intensity differences of each pixel point in different orthogonal directions.

[0060] It should be noted that the polarization intensity difference refers to the quantitative difference between the grayscale values ​​in the polarization direction corresponding to the same pixel point under different orthogonal polarization directions, which is used to characterize the degree of change in the response of the pixel point to the reflected light in a specific polarization direction; the polarization intensity difference is usually calculated based on the image grayscale values ​​at two sets of orthogonal polarization angles of 0 degrees and 90 degrees, and 45 degrees and 135 degrees, reflecting the anisotropic optical properties of the target area under orthogonal polarization conditions.

[0061] In specific implementation, the polarization degree and main polarization direction of each pixel point in the multi-angle polarization image can be determined by all polarization light intensity differences. That is, first, a pixel point is selected, and then the sum of the square values ​​of the polarization light intensity differences of the pixel point in two sets of orthogonal polarization directions and the sum of the grayscale values ​​of the pixel point in all polarization directions are determined. Then, the square root of the sum of the square values ​​is compared with the sum of the grayscale values ​​as the polarization degree of the pixel point; subsequently, the inverse tangent function value of the ratio of the polarization light intensity differences of the pixel point in the two sets of orthogonal polarization directions is determined as the main polarization direction at the pixel point, and the above steps are repeated to determine the polarization degree and main polarization direction of the remaining pixel points.

[0062] It should be noted that the polarization degree refers to the normalized ratio between the vector amplitude of the polarization light intensity difference reflected by the same pixel point under different polarization directions and the total reflection intensity of the pixel point under all polarization directions, which is used to quantify the strength of the polarization characteristics of the light wave at the pixel point; in addition, the main polarization direction refers to the relative relationship angle between the polarization light intensity differences corresponding to the same pixel point under different orthogonal polarization angles, which is used to characterize the dominant orientation of the polarization direction in the light reflected by the pixel point.

[0063] In step 103, the polarization degree of all pixel points is converted into a spectrum distribution diagram of the material surface under polarized reflection, and then the spectrum distribution diagram is decoupled and analyzed based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram.

[0064] In some embodiments, reference Figure 2 As shown in FIG, this figure is a schematic diagram of the process of determining the spectrum distribution diagram shown in some embodiments of the present application. The polarization degree of all pixel points is converted into the spectrum distribution diagram of the material surface under polarized reflection by the following steps:

[0065] First, in 1031, a two-dimensional polarization distribution map is constructed using the polarization degrees of all pixels;

[0066] Then, in 1032, spectrum transformation processing is performed on the two-dimensional polarization distribution diagram to obtain a frequency domain complex response diagram;

[0067] Finally, in 1033 , a frequency spectrum distribution diagram of the material surface under polarized reflection is determined according to the frequency domain complex response diagram.

[0068] In specific implementation, constructing a two-dimensional polarization distribution map using the polarization degrees of all pixel points can be achieved in the following manner: first, the polarization degree corresponding to each pixel point in the multi-angle polarization image is selected in turn; then, each polarization degree is arranged according to the spatial position of the corresponding pixel point in the multi-angle polarization image, and a two-dimensional matrix image of the same size as the multi-angle polarization image is obtained as the two-dimensional polarization distribution map.

[0069] It should be noted that the two-dimensional polarization distribution diagram refers to a two-dimensional matrix image constructed by arranging the polarization values ​​of all pixel points in the multi-angle polarization image point by point according to their original spatial positions, which is used to characterize the spatial distribution characteristics of the polarization consistency corresponding to each pixel point in the entire image.

[0070] In a specific implementation, the two-dimensional polarization distribution diagram is subjected to spectrum transformation processing to obtain a frequency domain complex response diagram, which can be achieved in the following manner, namely: first, the two-dimensional polarization distribution diagram is used as input data, and fast Fourier transform processing is performed in sequence in the row direction and the column direction, specifically including: using the polarization degree of each pixel point in the two-dimensional polarization distribution diagram as the spatial domain input intensity, mapping it into corresponding frequency components under the horizontal frequency and vertical frequency coordinates, respectively, and then rearranging the frequency components in the complex transformation coefficient matrix according to the frequency arrangement rule in such a manner that the low-frequency components are toward the center and the high-frequency components are toward the edge, thereby constructing a frequency domain complex response diagram, wherein the frequency domain complex response diagram contains the complex amplitude value corresponding to each frequency point.

[0071] It should be noted that the frequency domain complex response diagram described in this application refers to the complex value distribution image formed in the frequency space after the two-dimensional polarization distribution diagram is subjected to spectral transformation processing, and each frequency point corresponds to a complex frequency response, which is used to characterize the energy distribution and directional characteristics of the spatial domain polarization characteristics in different frequency dimensions; in addition, the complex amplitude value refers to the modulus value of the complex response value corresponding to each frequency point in the frequency domain complex response diagram, which is used to quantify the response intensity of the frequency component in the spectrum.

[0072] In a specific implementation, determining the spectrum distribution diagram of the material surface under polarized reflection based on the frequency domain complex response diagram can be achieved in the following manner, namely: first, traversing each frequency coordinate point in the frequency domain complex response diagram, extracting the modulus of its corresponding complex amplitude value as the spectrum energy value of the frequency point; then, based on the two-dimensional arrangement structure of the frequency points in the frequency domain complex response diagram, reconstructing the modulus values ​​corresponding to all frequency points into a two-dimensional amplitude distribution diagram in the original coordinate order; finally, using the two-dimensional amplitude distribution diagram as the spectrum distribution diagram of the material surface under polarized reflection conditions.

[0073] It should be noted that the spectrum distribution diagram refers to a two-dimensional amplitude image obtained by extracting the modulus of the complex amplitude values ​​corresponding to all frequency points in the frequency domain complex response diagram under polarized reflection conditions, and reconstructing it according to its frequency coordinate structure, which is used to characterize the energy distribution characteristics of the material surface at different frequency components.

[0074] In some embodiments, decoupling analysis is performed on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram by using the following steps:

[0075] extracting a high-frequency mask area from the frequency spectrum distribution map based on the high-frequency response characteristics of the crack defect on the material surface;

[0076] extracting amplitude response information of the frequency spectrum distribution diagram in the high-frequency mask area;

[0077] The spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram is determined according to the amplitude response information.

[0078] It should be noted that the high-frequency response characteristics of the surface crack defects of the material, that is, the high-frequency energy enhancement phenomenon exhibited by the crack area in the frequency domain, are specifically manifested as the presence of a significant amplitude mutation or energy concentration distribution at the high-frequency frequency point corresponding to the polarized reflection image after spectral transformation; the high-frequency response characteristics, after being mapped by a unified frequency division standard, can be used as the basis for spectrum mask selection to screen potential crack areas in the frequency domain; wherein, the division standard of high-frequency frequency points can be based on frequency distribution statistics (such as energy threshold ratio), which is not limited in this application; therefore, as a preferred embodiment, a high-frequency mask area is extracted from the spectrum distribution diagram based on the high-frequency response characteristics of the surface crack defects of the material, that is: a high-frequency frequency threshold is set, and then, the area composed of all frequency points in the spectrum distribution diagram that are greater than or equal to the high-frequency frequency threshold is used as the high-frequency mask area, thereby realizing the extraction of the high-frequency mask area from the spectrum distribution diagram. In other embodiments, other methods can also be used to achieve this, which is not limited here.

[0079] It should be noted that the high-frequency mask area refers to a frequency space sub-area composed of a set of frequency points that meet the high-frequency frequency point judgment conditions in the spectrum distribution diagram, which is used to characterize the spectrum response segment of the material surface that exhibits high energy concentration or amplitude mutation in the frequency domain.

[0080] In a specific implementation, the amplitude response information of the spectrum distribution diagram in the high-frequency mask area can be extracted in the following manner, namely: all frequency points corresponding to the spectrum distribution diagram in the high-frequency mask area are determined; then, for each frequency point, the corresponding amplitude response value is extracted, that is, the modulus length of the complex response of the frequency point; and then, a set of all the extracted amplitude response values ​​is used as the amplitude response information set of the high-frequency mask area.

[0081] It should be noted that the amplitude response information set refers to a data set composed of the modulus lengths of the complex amplitude values ​​corresponding to all frequency points extracted in the high-frequency mask area, which is used to characterize the energy response characteristics of the material surface in the high-frequency band; this set can reflect the local energy enhancement phenomenon of the material structure in the high-frequency frequency component, and is the basic data basis for crack defect spectrum disturbance identification, energy analysis and subsequent feature quantitative modeling.

[0082] In a specific implementation, determining the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram based on the amplitude response information can be achieved in the following manner, namely: first, all amplitude response values ​​in the amplitude response information are obtained, and then, the sum of all amplitude response values ​​is divided by the total number of all amplitude response values ​​as the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram. In other embodiments, other methods can also be used for implementation, which are not limited here.

[0083] It should be noted that the spectral disturbance energy refers to the mean-valued energy measurement value that characterizes the spectral response intensity of the crack defect, calculated based on the amplitude response information of each frequency point within the preset high-frequency mask area; it essentially reflects the degree of energy disturbance caused by structural abnormalities (such as cracks) on the material surface in the high-frequency band, and is usually obtained by statistically averaging the amplitude response values ​​of all frequency points in the area to quantify the overall high-frequency disturbance effect caused by the crack in the frequency domain.

[0084] In step 104, a response simulation of crack defects in the target area is performed based on a convolutional neural network algorithm combined with all main polarization directions and the spectral disturbance energy to obtain a crack response heat map, and then the structural perception tensors of different crack areas in the crack response heat map are generated through the response intensity distribution of all pixels in the crack response heat map.

[0085] In some embodiments, a crack response heat map is obtained by simulating the response of crack defects in a target area based on a convolutional neural network algorithm in combination with all main polarization directions and the spectrum disturbance energy. The following steps can be used:

[0086] Construct all main polarization directions into a directional characteristic map;

[0087] Constructing a multi-channel fusion tensor by combining the directional feature map with the spectrum disturbance energy;

[0088] The multi-channel fusion tensor is input into the convolutional neural network algorithm to simulate the crack defects in the target area, thereby obtaining a crack response heat map.

[0089] In specific implementation, constructing all main polarization directions into a directional characteristic map can be achieved in the following manner, namely: arranging all main polarization directions according to the coordinates of the corresponding pixel points to form a two-dimensional directional data matrix as the directional characteristic map. In other embodiments, other methods can also be used for implementation, which are not limited here. It should be noted that the directional characteristic map refers to a two-dimensional directional data matrix formed after spatially reconstructing all main polarization directions according to their corresponding image pixel coordinates, which is used to characterize the distribution characteristics of the main polarization response directions on the material surface at different spatial positions.

[0090] In specific implementation, the multi-channel fusion tensor is constructed by combining the directional feature map with the spectral perturbation energy. The following method is adopted: first, the spectral perturbation energy is used as the global response value, and is spatially expanded according to the size of the directional feature map, and the spectral perturbation energy is filled into a two-dimensional matrix with the same row and column dimensions as the directional feature map; then, the two-dimensional matrix is ​​combined with the directional feature map pixel by pixel in the channel dimension, that is, the main polarization direction corresponding to each pixel point and the spectral perturbation energy are used as vector elements to form a feature vector, and then the feature vectors corresponding to all pixels are used as a multi-channel fusion tensor. As a preferred embodiment, the polarization degree value corresponding to each pixel point can also be used as a vector element of the feature vector to enhance the multi-channel fusion tensor's ability to characterize changes in the optical properties of the material surface, and further improve the response recognition ability of the convolutional neural network in complex crack boundaries or weak reflection areas. It should be noted that the multi-channel fusion tensor refers to a three-dimensional data structure formed by combining the multi-dimensional feature vectors of each pixel point in the spatial dimension based on multiple material surface optical property information such as the directional feature map and the spectral perturbation energy.

[0091] In a specific implementation, the multi-channel fusion tensor is input into the convolutional neural network algorithm to simulate the crack defects in the target area, and then the crack response heat map is obtained. The following method is used to achieve this: first, according to the manifestation of the crack characteristics in the image (such as strip-like, bifurcated, subtle high-frequency edges, etc.), a network structure suitable for local texture modeling and spatial context perception is selected as the basic architecture. Preferably, a lightweight convolutional neural network model with a multi-scale receptive field can be adopted, such as introducing deep separable convolution (DSC) and atrous convolution (AC); then, a basic hierarchical structure of the network is constructed, specifically including: convolution layer, activation layer and downsampling layer. Secondly, the multi-channel fusion tensor is input as input data into the convolutional neural network model. In a convolutional neural network (CNN), the main polarization direction, spectral perturbation energy, and optional polarization degree features are extracted layer by layer through convolution kernels at each level to extract a deep feature map containing spatial distribution information and crack response patterns. Subsequently, an upsampling module or deconvolution layer is configured at the output end of the convolutional neural network to upsample and restore the extracted high-dimensional features, and finally a response heat map with the same size as the original image is output as the crack response heat map of the target area.

[0092] It should be noted that the crack response heat map refers to a two-dimensional image generated after feature extraction and spatial information fusion of multi-channel fusion tensors based on the convolutional neural network algorithm. It is expressed as a pixel matrix consistent with the size of the target area, wherein each pixel point intuitively reflects the response intensity of the crack defect at the corresponding position through changes in color depth or grayscale intensity. In addition, each pixel point of the crack response heat map corresponds to the response intensity value of the crack defect in the target area; wherein the response intensity value refers to the detection signal intensity of the crack defect represented by each pixel point in the crack response heat map at that position. The response intensity value is used to quantify the probability and severity of the existence of the crack defect at the location of the corresponding pixel point. The larger the response intensity value, the more obvious the characteristics of the crack defect at that location and the higher the crack risk.

[0093] In some embodiments, generating structure-aware tensors of different crack regions in the crack response heat map based on the response intensity distribution of all pixels in the crack response heat map can be achieved by using the following steps:

[0094] Determine the response intensity distribution of all pixels in the crack response heat map;

[0095] Performing stress fitting based on the response intensity distribution to obtain multiple crack stress trajectories;

[0096] The structural perception tensors of different crack regions in the crack response heat map are determined according to the stress trajectories of the cracks.

[0097] In specific implementation, the response intensity distribution of all pixel points in the crack response heat map can be determined in the following manner, namely: select a pixel point, determine the response intensity difference between the pixel point and the corresponding left adjacent pixel point and right adjacent pixel point, and determine the response intensity difference between the pixel point and the corresponding upper adjacent pixel point and lower adjacent pixel point; then, use all the response intensity differences corresponding to the pixel point as components of the gradient vector, thereby determining the amplitude of the gradient vector as the response intensity gradient of the pixel point; repeat the above steps to determine the response intensity gradients of the remaining pixel points, and then use the set of all response intensity gradients as the response intensity distribution of all pixel points in the crack response heat map.

[0098] It should be noted that the response intensity distribution refers to the set of response intensity gradients of all pixels in the crack response heat map, which is used to characterize the spatial variation characteristics and local response variation rate of the crack defect in the target area.

[0099] In a specific implementation, stress fitting is performed based on the response intensity distribution to obtain multiple crack stress trajectories, which can be achieved in the following manner: first, a crack sensitivity threshold is set, and multiple crack starting points are extracted from all pixel points by combining the crack sensitivity threshold with the response intensity distribution; then, a crack starting point is selected, and multiple stress diffusion points corresponding to the crack starting point are obtained in the crack response heat map based on a preset continuity condition; finally, the crack starting point and all corresponding stress diffusion points are taken as a crack stress trajectory, and the above steps are repeated to determine the crack stress trajectories corresponding to the remaining crack starting points, thereby obtaining multiple crack stress trajectories; wherein, the crack sensitivity threshold can be set according to the distribution characteristics of the response intensity gradient in the crack response heat map, for example, it can be dynamically determined in the form of the mean weighted standard deviation of the response intensity gradient, or it can be set through training and learning with pre-labeled samples; in addition, the The continuity condition may include that the angle between the gradient directions of adjacent pixel points is less than a preset angle threshold, and the relative difference in the response intensity gradient does not exceed a set ratio; preferably, for a crack starting point in the crack response heat map, its response intensity gradient direction is horizontal (i.e. horizontal to the right), and when selecting the stress diffusion point, it specifically includes: first, searching for the response intensity gradient information of the adjacent pixel points on the right side of the crack starting point; if the angle between the gradient direction of the pixel point and the gradient direction of the starting point is less than a set angle threshold, and the difference in the amplitude of its response intensity gradient for the crack starting point does not exceed a set ratio, then the pixel point is used as the stress diffusion point of the crack starting point; secondly, continue to use the current expanded point as the new reference point, search forward according to the same continuity condition, expand point by point, and form a continuous stress diffusion path; if the current pixel point does not meet the above continuity condition in all adjacent pixels, then stop searching.

[0100] It should be noted that the crack stress trajectory refers to the path formed by connecting the crack starting point and its adjacent stress diffusion points based on the response intensity gradient distribution in the crack response heat map, which is used to reflect the expansion direction, morphological characteristics and stress distribution of the crack in the target area.

[0101] In a specific implementation, the structural perception tensors of different crack areas in the crack response heat map are determined according to each crack stress trajectory, which can be achieved in the following manner: first, a crack stress trajectory is selected, and the coordinates of all pixel points of the crack stress trajectory and the corresponding response intensity gradient amplitudes are extracted; then, the response intensity gradient amplitudes of all pixel points in the crack stress trajectory are encoded as local features to construct a one-dimensional structural perception tensor to represent the response intensity distribution characteristics of the crack area; then, the one-dimensional structural perception tensors corresponding to all pixel points in the crack stress trajectory are merged into a planar tensor area as the structural perception tensor of the crack area corresponding to the crack stress trajectory; the above steps are repeated to determine the structural perception tensors of the crack areas corresponding to the remaining crack stress trajectories.

[0102] It should be noted that the structural perception tensor refers to a one-dimensional tensor representation formed by encoding the response intensity gradient amplitude of all pixel points in the crack stress trajectory, which reflects the spatial distribution characteristics of the response intensity in the crack area. The structural perception tensor can be used to describe the local intensity changes and overall spatial structural characteristics of the crack, thereby assisting in the regional division of crack defects.

[0103] In step 105 , crack defects on a target area of ​​the engine are automatically located and detected based on all structure perception tensors combined with a crack perception threshold.

[0104] In some embodiments, automatically locating and detecting crack defects on a target area of ​​an engine based on all structure perception tensors combined with a crack perception threshold may be achieved by the following steps:

[0105] Determine the perception tension value of the crack defect in the target area of ​​the engine through all structural perception tensors;

[0106] Preset the crack perception threshold of the target area;

[0107] Crack defects on a target area of ​​an engine are automatically located and detected based on the crack perception threshold and the perceived tension value.

[0108] In specific implementation, the perception tension value of the crack defect in the target area of ​​the engine can be determined by all structural perception tensors in the following way, namely: first, traverse the structural perception tensors corresponding to all crack areas in the target area, and then extract the response intensity gradient amplitude contained in each structural perception tensor in turn; then, accumulate the response intensity gradient amplitude of each crack area to obtain the cumulative response value of the corresponding crack area; then, add the cumulative response values ​​of all crack areas to obtain the overall response intensity sum of the crack defects in the target area of ​​the engine as the perception tension value of the crack defects in the target area of ​​the engine.

[0109] It should be noted that the perception tension value refers to the cumulative response value of the response intensity gradient amplitude in the structural perception tensor corresponding to all crack areas in the target area of ​​the engine, which reflects the quantitative index of the overall response intensity of the crack defect in the target area. It is used to characterize the comprehensive severity and distribution density of the crack defect, and serves as the key judgment basis for automatic crack positioning detection.

[0110] It should be noted that the crack perception threshold means that when the structural perception tension value of the crack area exceeds the crack perception threshold, it can be considered that the microstructural disturbance of the crack area has reached the judgment standard for constituting an engineering crack defect; preferably, the crack tolerance limit of different areas can be determined based on the thermal-mechanical load distribution borne by the target area of ​​the engine during operation, combined with the finite element structure simulation results; then, based on the distribution range of the structural perception tension value extracted from a large number of crack sample images, statistical learning or threshold regression methods are used to extract the discrimination boundary value as the crack perception threshold in this application. In other embodiments, other methods can also be used for implementation, which is not limited here.

[0111] It should be noted that the extraction of the crack stress trajectory only characterizes the possible geometric path of the crack, that is, the process of the crack stress trajectory constituting a crack defect in the engineering sense needs to be further judged by the relationship between the perceived tension value and the judgment threshold; therefore, in specific implementation, the automatic positioning detection of crack defects on the target area of ​​the engine based on the crack perception threshold and the perceived tension value can be achieved in the following manner, namely: first, set the crack perception threshold, and then judge the crack defect in the target area by using the crack perception threshold, refer to Figure 3 As shown, this figure is a flow chart of realizing crack defect judgment shown in some embodiments of the present application, which specifically includes: when the perception tension value is greater than or equal to the crack perception threshold, it is determined that there is a crack defect in the target area, and when the perception tension value is less than the crack perception threshold, no processing is performed; secondly, when it is determined that there is a crack defect in the target area, the specific location and range information of the crack defect are extracted based on the spatial distribution characteristics of the structural perception tensor of each crack area, specifically including: by analyzing the local peak position of the response intensity gradient in the structural perception tensor, the crack starting point and extension direction are determined; and then the spatial contour of the crack is accurately depicted in combination with the morphological characteristics of the crack stress trajectory; for example, if the response intensity gradient of the crack stress trajectory in the specified local area is significantly increased, then the local area is determined to be an active growth area of ​​the crack, and is used as a key marking area for automatic positioning, thereby realizing automatic positioning detection of crack defects on the target area of ​​the engine. It should be noted that the morphological characteristics include length, curvature and branching.

[0112] In addition, in another aspect of the present application, in some embodiments, the present application provides a fully automatic engine visual inspection device, referring to Figure 4 , which is a schematic diagram of the structure of a fully automatic engine visual inspection device according to some embodiments of the present application. The fully automatic engine visual inspection device 200 includes: an acquisition module 201, a processing module 202, and an execution module 203, which are described as follows:

[0113] Acquisition module 201, in this application, acquisition module 201 is mainly used to automatically acquire multi-angle polarization images of the engine target area, wherein the multi-angle polarization images include multiple polarization directions;

[0114] Processing module 202, in this application, the processing module 202 is mainly used to determine the polarization degree and main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions;

[0115] In addition, the processing module 202 in the present application is further configured to convert the polarization degree of all pixel points into a spectrum distribution diagram of the material surface under polarized reflection, and then perform a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram;

[0116] In addition, the processing module 202 in the present application is further used to simulate the response of crack defects in the target area based on a convolutional neural network algorithm combined with all main polarization directions and the spectrum disturbance energy to obtain a crack response heat map, and then generate a structural perception tensor of different crack areas in the crack response heat map through the response intensity distribution of all pixels in the crack response heat map;

[0117] Execution module 203, in this application, execution module 203 is mainly used to automatically locate and detect crack defects on the target area of ​​the engine based on all structural perception tensors combined with crack perception thresholds.

[0118] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores code, and the processor is configured to obtain the code and execute the above-mentioned fully automatic engine visual inspection method.

[0119] In some embodiments, reference Figure 5 , which is an internal structure diagram of a computer device for implementing a fully automatic engine visual inspection method according to some embodiments of the present application. The fully automatic engine visual inspection method in the above embodiment can be Figure 5 The computer device 300 shown in FIG. 1 is implemented as shown in FIG. 1 , and the computer device 300 includes at least one processor 301 , a communication bus 302 , a memory 303 , and at least one communication interface 304 .

[0120] The processor 301 may be a general-purpose central processing unit (CPU), or an application-specific integrated circuit (ASIC) or one or more processors for controlling the execution of the fully automatic engine visual inspection method of the present application.

[0121] The communication bus 302 is used to transmit information between the above components.

[0122] Memory 303 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, an optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer. Memory 303 may exist independently and be connected to processor 301 via communication bus 302. Memory 303 may also be integrated with processor 301.

[0123] Memory 303 is used to store program code for implementing the present invention, and is controlled by processor 301 for execution. Processor 301 is configured to execute the program code stored in memory 303. The program code may include one or more software modules. The fully automatic engine visual inspection method in the above embodiment can be implemented using processor 301 and one or more software modules in the program code stored in memory 303.

[0124] The communication interface 304 uses any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0125] In a specific implementation, as an example, a computer device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0126] The aforementioned computer device may be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of this application do not limit the type of computer device.

[0127] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned fully automatic engine visual inspection method.

[0128] In summary, in the fully automatic engine visual inspection equipment and method disclosed in the embodiments of the present application, a multi-angle polarization image of the target area of ​​the engine is automatically collected, and the multi-angle polarization image includes multiple polarization directions; the polarization degree and main polarization direction of each pixel point in the multi-angle polarization image are determined based on the orthogonal relationship between different polarization directions; the polarization degree of all pixel points is converted into a spectral distribution diagram of the material surface under polarized reflection, and then the spectral distribution diagram is decoupled and analyzed based on the high-frequency response characteristics of the crack defects on the material surface to obtain the spectral disturbance energy of the crack defects in the preset high-frequency mask area of ​​the spectral distribution diagram; based on the convolutional neural network algorithm combined with all main polarization directions and the spectral disturbance energy, the response simulation of the crack defects in the target area is performed to obtain a crack response heat map, and then the structural perception tensors of different crack areas in the crack response heat map are generated through the response intensity distribution of all pixel points in the crack response heat map; the crack defects on the target area of ​​the engine are automatically located and detected according to all structural perception tensors combined with the crack perception threshold; the microcrack contrast can be improved based on the reflection characteristics of the material surface.

[0129] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0130] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if such changes and modifications fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such changes and modifications.

Claims

1. A fully automatic engine visual inspection method, characterized in that: The steps include: Automatically collect a multi-angle polarization image of the target area of ​​the engine, wherein the multi-angle polarization image includes multiple polarization directions; Determining the polarization degree and main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions; Converting the polarization degree of all pixels into a spectrum distribution diagram of the material surface under polarized reflection, and then performing a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram; Based on a convolutional neural network algorithm, a response simulation of crack defects in a target area is performed in combination with all main polarization directions and the spectrum disturbance energy to obtain a crack response heat map, and then a structural perception tensor of different crack areas in the crack response heat map is generated through the response intensity distribution of all pixels in the crack response heat map; Crack defects in the target area of ​​the engine are automatically located and detected based on all structure perception tensors combined with crack perception thresholds.

2. The method according to claim 1, wherein Determining the polarization degree and the main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions specifically includes: Determine the polarization intensity difference of each pixel in different orthogonal directions based on the orthogonal relationship between different polarization directions; The polarization degree and main polarization direction of each pixel point in the multi-angle polarization image are determined by all polarized light intensity differences.

3. The method according to claim 1, wherein Converting the polarization degree of all pixels into the spectrum distribution diagram of the material surface under polarized reflection specifically includes: Construct a two-dimensional polarization distribution map through the polarization degrees of all pixels; Performing spectrum transformation on the two-dimensional polarization distribution diagram to obtain a frequency domain complex response diagram; A frequency spectrum distribution diagram of the material surface under polarized reflection is determined according to the frequency domain complex response diagram.

4. The method according to claim 1, wherein The spectrum distribution diagram is decoupled and analyzed based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram, which specifically includes: extracting a high-frequency mask area from the frequency spectrum distribution map based on the high-frequency response characteristics of the crack defect on the material surface; extracting amplitude response information of the frequency spectrum distribution diagram in the high-frequency mask area; The spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram is determined according to the amplitude response information.

5. The method according to claim 1, wherein Based on the convolutional neural network algorithm, all main polarization directions and the spectrum disturbance energy are combined to simulate the response of crack defects in the target area, and the crack response heat map is obtained, which specifically includes: Construct all main polarization directions into a directional characteristic map; Constructing a multi-channel fusion tensor by combining the directional feature map with the spectrum disturbance energy; The multi-channel fusion tensor is input into the convolutional neural network algorithm to simulate the crack defects in the target area, thereby obtaining a crack response heat map.

6. The method according to claim 1, wherein Generating structural perception tensors of different crack regions in the crack response heat map through the response intensity distribution of all pixels in the crack response heat map specifically includes: Determine the response intensity distribution of all pixels in the crack response heat map; Performing stress fitting based on the response intensity distribution to obtain multiple crack stress trajectories; The structural perception tensors of different crack regions in the crack response heat map are determined according to the stress trajectories of the cracks.

7. The method according to claim 1, wherein Automatically locate and detect crack defects on the target area of ​​the engine based on all structural perception tensors combined with crack perception thresholds. Specifically, the following are performed: Determine the perception tension value of the crack defect in the target area of ​​the engine through all structural perception tensors; Preset the crack perception threshold of the target area; Crack defects on a target area of ​​an engine are automatically located and detected based on the crack perception threshold and the perceived tension value.

8. The method according to claim 1, wherein The target area includes the engine housing surface, the cylinder head joint area, and the transition area between the heat sink and the frame.

9. The method according to claim 1, wherein The polarization direction refers to the angle formed by the electric vector of the light wave and the perpendicular direction of the light propagation direction when the polarized light source illuminates the target area, including 0°, 45°, 90° and 135°.

10. A fully automatic engine visual inspection device, characterized in that: Includes: An acquisition module, configured to automatically acquire a multi-angle polarization image of a target area of ​​an engine, wherein the multi-angle polarization image includes multiple polarization directions; a processing module, configured to determine the degree of polarization and the main polarization direction of each pixel in the multi-angle polarization image based on the orthogonal relationship between different polarization directions; The processing module is further configured to convert the polarization degrees of all pixel points into a spectrum distribution diagram of the material surface under polarized reflection, and then perform a decoupling analysis on the spectrum distribution diagram based on the high-frequency response characteristics of the crack defect on the material surface to obtain the spectrum disturbance energy of the crack defect in the preset high-frequency mask area of ​​the spectrum distribution diagram; The processing module is further configured to simulate the response of crack defects in the target area based on a convolutional neural network algorithm in combination with all main polarization directions and the spectrum disturbance energy to obtain a crack response heat map, and then generate structural perception tensors of different crack areas in the crack response heat map through the response intensity distribution of all pixels in the crack response heat map; The execution module is used to automatically locate and detect crack defects on the target area of ​​the engine based on all structural perception tensors combined with crack perception thresholds.

Images