An engine leakage detection method and detection system based on AI image processing

Abstract

The application relates to the technical field of image processing, and provides an engine liquid leakage detection method and a detection system based on AI image processing, which comprises the following steps: step 1: obtaining surface image information of an engine under an idle speed condition to obtain a first contrast image; and obtaining surface image information of the engine under a running condition to obtain a second contrast image; step 2: respectively pre-processing and feature extraction are performed on the first contrast image and the second contrast image to obtain first image data and second image data; step 3: the first image data and the second image data are input into a SAM model to obtain a mask image of an initial judgment area of an engine surface; and step 4: morphological operation is performed on the mask image to generate a liquid leakage area; compared with a traditional ultrasonic flaw detector, the technical scheme disclosed by the application can determine whether the engine is damaged by using the image processing technology, and the detection efficiency is significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an engine leakage detection method and system based on AI image processing. BACKGROUND

[0002] The content of this part only provides background information related to the present application, which may not constitute prior art.

[0003] After the engine is assembled, it needs to be detected whether the engine has leakage. There are generally three kinds of liquids in the engine: fuel, oil and coolant. The three liquids exist in three circulation systems, and each circulation system needs to be carefully checked for leakage.

[0004] In the prior art, visual image analysis system or ultrasonic flaw detector is generally used for inspection. In practice, the visual image analysis system can only find larger damage, which generally includes cracks and sand holes. The pipeline, weld, sealing surface and fastening point are not clear in the picture, and neither the naked eye nor the visual processing algorithm can accurately judge whether there is damage. The ultrasonic flaw detector needs to detect the surface of the entire engine, which is prone to missing in practice, and the internal structure of the engine is complex, so the echo signal generated by the ultrasonic flaw detector cannot judge whether the hollow area is normal damage or normal hollow structure. Therefore, using the ultrasonic flaw detector to check whether the engine has leakage needs to spend a lot of time to detect the surface of the entire engine.

[0005] In summary: there is currently a lack of a fast and accurate method and system for checking whether the engine has leakage. SUMMARY

[0006] Therefore, the purpose of the present application is to provide an engine leakage detection method based on AI image processing, which can solve the technical problems raised in the background art.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] An engine leakage detection method based on AI image processing, comprising the following steps:

[0009] Step 1: obtaining surface image information of the engine under idle speed condition to obtain a first contrast image; obtaining surface image information of the engine under running condition to obtain a second contrast image;

[0010] Step 2: pre-processing and feature extraction are performed on the first contrast image and the second contrast image respectively to obtain first image data and second image data;

[0011] Step 3: input the first image data and the second image data into the SAM model to obtain a mask image of an initial determination area of the engine surface;

[0012] Step 4: performing morphological operation on the mask image to generate a leakage area;

[0013] Step 5: performing intensity determination on the leakage area to determine whether the engine has damage according to the intensity determination result.

[0014] Compared with the traditional ultrasonic flaw detector, the technical solution significantly improves the detection efficiency. In order to further enhance the detection accuracy, the scheme does not directly extract the damage features through image processing technology, but focuses on the detection of the leakage phenomenon of the engine during operation. Practice shows that the leakage phenomenon caused by damage during engine operation is more significant than the subtle damage, and the detection recognition degree is higher, thereby effectively improving the detection accuracy. In the leakage area positioning aspect, the scheme compares and analyzes two groups of image data collected under the engine idle mode and working mode to improve the detection accuracy. The comparison scheme can quickly eliminate the complex detail features of the engine surface, accurately extract the leakage area information, and greatly improve the accuracy of leakage area recognition.

[0015] In view of the problem that the color features of engine oil, fuel and coolant are low in recognition degree, and it is difficult to effectively extract features from images when there is a trace leakage, the following technical solution is proposed:

[0016] When obtaining the first comparison image and the second comparison image of the engine component, ultraviolet light is used as the illumination light source;

[0017] In the engine running state, fluorescent agent is added to the engine oil and coolant.

[0018] The scheme adds fluorescent agent to the two types of liquid without damage, so that the leaked liquid has fluorescent characteristics. Combined with ultraviolet light source irradiation, the fluorescent reaction of the leakage area can be clearly captured. Compared with the traditional detection method, the technology can quickly filter out irrelevant background features in the comparison image, accurately locate the potential leakage area caused by damage and other defects, and effectively improve the detection accuracy and efficiency in the trace leakage scenario. Moreover, the ultraviolet light source can clearly distinguish the fuel from the background information, so that the engine can also identify the trace fuel stains on the engine surface without adding fluorescent agent to the fuel, thereby increasing the recognition accuracy and avoiding the problem of carbon deposition in the engine caused by adding fluorescent agent to the engine during detection.

[0019] In some possible embodiments, the preprocessing in step 2 includes the following steps:

[0020] Step 21: taking the first control image or the second control image as a preprocessed image, performing Gaussian filtering on the preprocessed image to obtain a Gaussian filtered image;

[0021] Step 22: performing adaptive histogram equalization processing on the Gaussian filtered image to generate an equalized image;

[0022] Step 23: performing Gamma correction on the equalized image to generate the first image data or the second image data.

[0023] In the technical scheme provided in the present application, the image preprocessing process adopts the order of "Gaussian filtering→image equalization→image correction", which can effectively improve the image quality and reduce the difficulty of subsequent feature extraction. The specific principle is as follows: Gaussian filtering suppresses image noise through smoothing processing, reducing the interference of noise data on subsequent processing. Since image equalization may amplify residual noise when enhancing contrast, the Gaussian filtering in the preprocessing stage has filtered out most of the noise in advance, avoiding the unreasonable amplification of noise in the equalization process. After equalization processing, the image has reduced noise interference, and the contrast and details are optimized. At this time, when performing image correction, the algorithm does not need to additionally process the interference caused by noise, thereby significantly reducing the correction calculation amount. This processing order forms a logical chain of noise suppression-contrast enhancement-accurate correction, forming a layer-by-layer optimization preprocessing system, and finally realizing the systematic improvement of image quality, laying a good data foundation for subsequent feature extraction.

[0024] In some possible embodiments, step 21 includes the following steps:

[0025] Step 211: setting a Gaussian kernel size, and calculating a Gaussian function value G(x, y) of each preprocessed image in the Gaussian kernel;

[0026]

[0027] x, y represents the coordinates of a point in the Gaussian kernel relative to the center point of the Gaussian kernel, σ represents the standard deviation of the Gaussian distribution, k represents the radius of the Gaussian kernel, π represents the circular constant, and exp represents the natural exponential function;

[0028] Step 212: filtering each pixel point in the preprocessed image with the Gaussian function value G(x, y) to obtain a Gaussian filtered image;

[0029]

[0030] Wherein, I'(i,j) represents the pixel value of the Gaussian filter image at position (i,j), I(i+m,j+n) represents the pixel value of the preprocessed image at position (i+m,j+n), m and n represent the offset of the Gaussian kernel respectively, m∈[-k,k], n∈[-k,k], i and j represent the horizontal coordinate and vertical coordinate of the pixel point respectively.

[0031] In some possible embodiments, step 22 comprises the following steps:

[0032] Step 221: Count the number of pixel points n of each gray level in the Gaussian filter image j ;

[0033] Step 222: Calculate the CDF value of each gray level;

[0034]

[0035] r k represents the original gray level of the Gaussian filter image, N represents the total number of pixel points of the Gaussian filter image, k represents the upper limit of the gray level currently calculated, CDF(r k ) represents the proportion of pixel points in the image whose gray value is less than or equal to r k , used to describe the cumulative probability distribution of the gray level;

[0036] Step 223: Linearly map the CDF to the interval, and replace the gray value in the Gaussian image with the mapped value to generate an equalization image:

[0037] s k = round(CDF(r k )×(L-1));

[0038] round represents the rounding function, L represents the total number of gray levels, s k represents the mapped gray value.

[0039] In some possible embodiments, step 23 comprises the following steps:

[0040] Step 231: Normalize the gray value of the equalization image;

[0041] Step 232: Perform power transformation on the normalized gray value:

[0042]

[0043] Wherein, I gamma (x,y) is the pixel value of the pixel point (x,y) after power transformation, I norm (x,y) is the pixel value of the pixel point (x,y) of the equalization image.

[0044] Step 233: I gamma (x, y) is performed to complete the Gamma correction.

[0045] The present application realizes multi-dimensional image quality optimization through the serial pretreatment process of "Gaussian filtering → adaptive histogram equalization → Gamma correction": Gaussian filtering preferentially suppresses noise, avoiding amplification of noise interference in the subsequent contrast enhancement stage; adaptive histogram equalization improves image local details and global contrast on the basis of low noise, enhancing feature discrimination; Gamma correction further precisely regulates overall brightness distribution, improving image dynamic range. The three work together to form the progressive optimization logic of "noise reduction-sharpness enhancement-light adjustment", which not only significantly improves the image signal-to-noise ratio and feature distinguishability, but also reduces the computational redundancy of subsequent equalization and correction by pre-inhibiting noise, effectively reducing the complexity of subsequent algorithms and improving the overall processing efficiency.

[0046] Further, the training method of the SAM model in step 3 includes the following steps:

[0047] Step 31: A standard data set is prepared in advance; the standard data set includes a plurality of samples, and each sample includes first image data, second image data, and a segmentation mask;

[0048] Step 32: The SAM segmentation model is trained through transfer learning;

[0049] Step 33: The SAM segmentation model is iteratively trained with the standard data set until the desired effect is achieved.

[0050] In the technical solution provided by the present application, the SAM segmentation model is trained using transfer learning, so that the pre-training weights of the SAM image encoder can be directly reused, and the SAM segmentation module does not need to be trained from scratch, thereby increasing the segmentation efficiency of the SAM segmentation model.

[0051] When segmenting image data, the first image data and the second image data are used as comparison information to increase the accuracy of image data segmentation. However, when the first image data and the second image data are input into the SAM segmentation model, a large number of redundant features are generated, which in turn cannot accurately generate an image mask. Therefore, the present application provides the following technical solution;

[0052] Further, the SAM segmentation model includes:

[0053] An image encoder for inputting first image data and second image data to generate high-dimensional image embeddings;

[0054] A prompt encoder for inputting prompt data collected from the segmentation mask to generate prompt embeddings;

[0055] a convolutional network, performing convolution operation on the image embedding to generate a convolutional feature;

[0056] a mask encoder, inputting the convolutional feature and the prompt embedding, aligning the prompt embedding with the convolutional feature to generate an image mask.

[0057] In the technical solution provided in the present application, a convolutional network is added in the SAM segmentation model. The convolutional network performs convolution operation on the high-dimensional image embedding. After the convolution operation, the difference information between the first image data and the second image data can be captured. By controlling the receptive field of the convolutional network, the local feature changes between the first image data and the second image data are captured. On the one hand, the information redundancy of the image embedding is reduced, and on the other hand, the accuracy of feature extraction is increased.

[0058] Further, the generation process of the image embedding is as follows:

[0059] S1: The first image data and the second image data are respectively divided into blocks to obtain N feature blocks;

[0060] S2: Each feature block is linearly projected to generate a high-dimensional vector of each feature block;

[0061] S3: The high-dimensional vectors of each feature block are added block by block, and are encoded by a Transformer to generate a high-dimensional image embedding.

[0062] In the technical solution provided in the present application, the first image data and the second image data are respectively divided into blocks, so that the first image data and the second image data can be mixed. The high-dimensional vectors extracted therefrom contain the detail information of both. Therefore, a large amount of comparison information of the first image data and the second image data exists in the finally generated high-dimensional image embedding.

[0063] Further, the prompt data includes point prompts and box prompts. The extraction steps of the point prompts and the box prompts are as follows:

[0064] Z1: The first image data and the second image data are overlapped with each other. The pixel points with the same pixel value in the first image data and the second image data are regarded as background regions, and the regions with different pixel points are regarded as foreground regions;

[0065] Z2: k points are randomly selected from the foreground region to generate a positive point sampling set M GT m points are randomly selected from the background region to generate a negative point sampling set M HT A random offset is added to each positive point and each negative point;

[0066] Z3: The minimum and maximum coordinates of the foreground region are obtained to generate a minimum bounding box;

[0067] Z4: The positive point sampling set MGT and negative point sampling set M HT As a point prompt, the minimum bounding box is taken as a box prompt.

[0068] In the technical solution provided in the present application, the foreground region and the background region can be quickly generated through mutual comparison between the first image data and the second image data, so that the related features of the region to be segmented can be accurately extracted without labeling.

[0069] The generated mask image may have some noise signals, and therefore the present application provides the following technical solution:

[0070] Step 4 includes the following steps:

[0071] Step 41: performing an opening operation of first erosion and then dilation on the mask image to obtain mask information;

[0072] Step 42: performing a closing operation of first dilation and then erosion on the mask information to obtain mask features;

[0073] Step 43: extracting a connected region from the mask features, and taking the connected region as a liquid leakage region.

[0074] In the present application, the opening operation eliminates isolated small noise points (such as scattered dot-shaped foreground) in the mask through “first erosion and then dilation”, while retaining the main structure of the target object; the closing operation fills holes (such as fractures or missing of the target region) in the mask through “first dilation and then erosion”, thereby enhancing the integrity of the mask. This processing significantly improves the purity and continuity of the mask.

[0075] There may be some subtle differences between the first image data and the second image data, which may be identified as a liquid leakage region. Therefore, the present application provides the following technical solution:

[0076] Step 5 includes the following steps:

[0077] Step 51: performing area determination on the liquid leakage region, if the area is less than a preset threshold, then the liquid leakage region does not exist engine damage, otherwise, the engine damage exists;

[0078] Step 52: performing intensity threshold determination on the liquid leakage region, if the fluorescence intensity is less than a preset dynamic intensity threshold, then the liquid leakage region does not exist engine damage, otherwise, the engine damage exists.

[0079] In the technical solution provided in the present application, the liquid leakage region is comprehensively judged whether there is real damage according to the liquid leakage area and the fluorescence threshold. According to the fluorescence characteristics generated by the damage leakage, it is accurately judged whether there is a liquid leakage region.

[0080] An engine leakage detection system based on AI image processing adopts an engine leakage detection method based on AI image processing to determine whether the engine has damage.

[0081] The technical scheme of the embodiment of the present application has at least the following advantages and beneficial effects:

[0082] The technical scheme determines whether the engine has damage by using image processing technology, which significantly improves the detection efficiency compared with traditional ultrasonic flaw detection instruments. To further enhance the detection accuracy, the scheme does not directly extract damage features by image processing technology, but focuses on the detection of leakage phenomena during engine operation. Practice shows that the leakage phenomenon caused by damage during engine operation is more significant than subtle damage, and the detection recognition degree is higher, thereby effectively improving the detection accuracy. In terms of leakage area positioning, the scheme compares and analyzes two groups of image data collected under the engine idle mode and working mode. The comparison scheme can quickly eliminate the complex details of the engine surface, accurately extract the leakage area information, and greatly improve the accuracy of leakage area identification. BRIEF DESCRIPTION OF DRAWINGS

[0083] Fig. 1 is a gray scale diagram of engine oil under ultraviolet irradiation;

[0084] Fig. 2 is a gray scale diagram of cooling liquid under ultraviolet irradiation;

[0085] Fig. 3 is a gray scale diagram of fuel oil under ultraviolet irradiation;

[0086] Fig. 4 is a flowchart of the engine leakage detection method based on AI image processing;

[0087] Fig. 5 is a structural schematic diagram of the SAM model. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely in combination with specific embodiments. The same reference signs in the drawings represent the same components. It should be noted that the described embodiments are part of the embodiments of the present application, not all. Based on the described embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0089] Compared to the embodiments shown in the drawings, the feasible implementation solutions within the protection scope of the present application can have fewer components, have other components not shown in the drawings, have different components, have differently arranged components, or have differently connected components, etc. In addition, two or more components in the drawings can be implemented in a single component, or a single component shown in the drawings can be implemented as multiple separate components.

[0090] Unless otherwise defined, technical terms or scientific terms used herein should be understood as having the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms "first", "second", and similar terms used herein do not necessarily denote any order, quantity, or importance, but are used to distinguish different components. Similarly, "one" or "a" or similar terms do not necessarily denote a quantity of one. "Up", "down", and the like are used only to indicate relative positions, and when the absolute positions of the described objects are changed, the relative positions can also be changed accordingly.

[0091] Embodiment 1:

[0092] Reference Fig. 4 The present application needs to detect the damage on the surface of the engine, including cracks and sand holes, which are very small and cannot be accurately found by naked eyes and image recognition, and the dust on the surface of the engine will block these sand holes, resulting in the inability to accurately find these cracks.

[0093] The first embodiment of the present application discloses an engine leakage detection method based on AI image processing, comprising the following steps:

[0094] Step 1: Obtain the surface image information of the engine under idle condition to obtain the first contrast image; obtain the surface image information of the engine under running condition to obtain the second contrast image.

[0095] When obtaining the first contrast image and the second contrast image of the engine component, ultraviolet light is used as the illumination light source; and a trace amount of fluorescent agent is added to the engine oil and the cooling liquid under the running state of the engine.

[0096] Specifically, the wavelength of the ultraviolet light is 350-365 nm, the concentration of the fluorescent agent in the engine oil and the cooling liquid is 0.01-0.1 g / L, and no fluorescent agent is added to the fuel oil.

[0097] As Figs. 1-3 shown, the engine oil and the cooling liquid will be more obvious after adding the fluorescent agent, and the fuel oil can also have a good display effect under the irradiation of ultraviolet light with a wavelength of 350-365 nm.

[0098] The fluorescent agent needs to be shaken uniformly after being added into the engine oil and the cooling liquid, so that the engine oil and the cooling liquid will contain certain fluorescent substances after being leaked.

[0099] The first contrast image and the second contrast image are images of the same position, the same angle and the same focal length of the engine surface.

[0100] In practice, the first contrast image and the second contrast image can be obtained in the following way:

[0101] First, fix the camera on one side of the engine surface to be measured, and align the lens of the camera with the engine surface. Adjust the focal length of the camera so that the image is clear. Then, remove the remaining light source, use ultraviolet light with a wavelength of 350-365 nm as the light source, and the camera obtains a picture, which is the first contrast image.

[0102] Then, keep the engine stationary and the remaining parameters unchanged, start the engine, and run for 10 minutes under high speed conditions. Use ultraviolet light with a wavelength of 350-365 nm as the light source, and the camera obtains a second picture, which is the second contrast image.

[0103] Therefore, the first contrast image and the second contrast image are compared, which is the picture of the engine under high speed operation. In the case of no engine leakage, the second contrast image and the first contrast image are basically the same, and if there is leakage, a fluorescent region will appear in the second contrast image, which is the key to judging whether there is leakage.

[0104] And the image may be partially offset due to engine vibration, and simple manual comparison is easy to miss. Therefore, subsequent steps are needed to automatically determine whether there is damage on the engine.

[0105] Step 2: Preprocess and feature extract the first contrast image and the second contrast image to obtain first image data and second image data.

[0106] The first contrast image and the second contrast image have the same format and the same processing method, and both need to be preprocessed, the specific method is as follows:

[0107] The preprocessing in step 2 includes the following steps:

[0108] Step 21: Take the first contrast image or the second contrast image as the preprocessed image, and perform Gaussian filtering on the preprocessed image to obtain a Gaussian filtered image.

[0109] The pre-processed image is required to be Gaussian filtered, histogram equalized and Gamma corrected, which is the first or second control image. The first and second control images are pre-processed in the same way, and thus the pre-processing of the first control image is only introduced herein.

[0110] After obtaining the first and second control images, the first and second control images are required to be gray-scaled. The gray scaling is to convert the image from RGB format to gray scale format, and the specific conversion method is not described herein.

[0111] Specifically, step 21 includes the following steps:

[0112] Step 211: set the Gaussian kernel size, and calculate the Gaussian function value G(x, y) of each pre-processed image in the Gaussian kernel;

[0113]

[0114] x, y represent the coordinates of a point in the Gaussian kernel relative to the center point of the Gaussian kernel, σ represents the standard deviation of the Gaussian distribution, k represents the radius of the Gaussian kernel, π represents the circular constant, and exp represents the natural exponential function.

[0115] Step 212: filter each pixel point in the pre-processed image with the Gaussian function value G(x, y) to obtain a Gaussian filtered image;

[0116]

[0117] wherein I'(i, j) represents the pixel value of the Gaussian filtered image at position (i, j), I(i+m, j+n) represents the pixel value of the pre-processed image at position (i+m, j+n), m and n represent the offset of the Gaussian kernel, m ∈ [-k, k], n ∈ [-k, k], and i and j represent the horizontal and vertical coordinates of the pixel point, respectively.

[0118] The Gaussian filtering can remove the discrete noise points in the pre-processed image and reduce the influence of noise on subsequent feature extraction.

[0119] Step 22: performing adaptive histogram equalization on the Gaussian filtered image to generate an equalized image.

[0120] Step 22 includes the following steps:

[0121] Step 221: counting the number of pixel points n of each gray level in the Gaussian filtered image j .

[0122] The value range of the gray value is [0, 255], and a plurality of gray scales are set in the application, for example, the gray scale is set to 10, and the size of the value range of each gray scale is 25.5.

[0123] After setting the gray scale, the value range of each gray scale can be determined, then the gray value of each pixel point in the Gaussian filtering image is extracted, the number of pixel points in each gray scale is counted, and n j , j represents the index of the gray scale.

[0124] Step 222: Calculate the CDF value of each gray scale;

[0125]

[0126] r k represents the original gray scale of the Gaussian filtering image, N represents the total number of pixel points of the Gaussian filtering image, k represents the upper limit of the gray scale currently calculated, CDF(r k ) represents the proportion of pixel points with a gray value less than or equal to r k in the image, and is used to describe the cumulative probability distribution of the gray scale.

[0127] Step 223: Linearly map the CDF to the interval, and replace the gray value in the Gaussian image with the mapped value to generate an equalization image:

[0128] s k = round(CDF(r k ) * (L-1));

[0129] round represents the rounding function, L represents the total number of the total number of gray scales, s k represents the mapped gray value.

[0130] The global equalization strategy provided in the application can excessively enhance the contrast of the noise region, thereby stretching the region with uneven gray distribution in the image, increasing the local contrast, and making the local details clearer.

[0131] Step 23: Gamma correction is performed on the equalization image to generate first image data or second image data.

[0132] Step 23 includes the following steps:

[0133] Step 231: Normalize the gray value of the equalization image.

[0134] The gray value of the equalization image is gray value S k , and the value range of S k is [0, 255].

[0135] Step 232: Perform power transformation on the normalized gray value:

[0136]

[0137] wherein, I gamma (x,y) is the pixel value of the pixel point (x, y) in the equalized image, I norm (x,y) is the pixel value of the pixel point (x, y) in the equalized image, I

[0138] Step 233: performing inverse normalization on I gamma (x,y) to complete Gamma correction.

[0139] The key of Gamma correction is that after normalizing the pixel value, the pixel value is transformed by using a preset gamma value, and then inverse normalization is performed to restore the gray value. In the present application, γ = 0.5.

[0140] By using the above scheme, the first contrast image and the second contrast image can be preprocessed, and the preprocessed image has higher definition, more obvious features and less noise. The preprocessed image is the first image data and the second image data.

[0141] Subsequently, the first image data and the second image data need to be inputted to extract the mask information related to the liquid leakage. Specifically:

[0142] Step 3: inputting the first image data and the second image data into the SAM model to obtain a mask image of an initial determination area on the surface of the engine.

[0143] In step 3, the key part is the SAM model, which is an image processing model for automatic segmentation of images. Generally, a single picture is inputted, and the corresponding mask information is extracted according to the inputted single picture. The mask information is the feature information to be extracted.

[0144] In the present application, in order to increase the extraction capability of the feature area, two images, i.e. the first image data and the second image data, are inputted into the SAM model at one time.

[0145] The SAM segmentation model includes an image encoder, a prompt encoder, a convolutional network and a mask encoder. The image encoder is signal connected with the convolutional network, the convolutional network is connected with the mask encoder, and the prompt encoder is connected with the mask encoder.

[0146] The image encoder is used to input the first image data and the second image data to generate a high-dimensional image embedding.

[0147] The generation process of the image embedding is as follows:

[0148] S1: block the first image data and the second image data respectively to obtain N feature blocks.

[0149] The first image data and the second image data are input into the image encoder twice, and then divided to obtain N feature blocks. The first image data is divided into N / 2 feature blocks, and the second image data is divided into N / 2 feature blocks.

[0150] The size of the feature block can be set according to the requirement. In this application, the size of the feature block is 16x16 pixels.

[0151]

[0152] H and W can be divided by 16, and H and W represent the length and width of the first image data respectively.

[0153] S2: linearly project each feature block to generate a high-dimensional vector of each feature block.

[0154] The specific way of the high-dimensional vector is as follows:

[0155] S21: set the projection matrix W proj :

[0156] E represents the length of the feature block, and D represents the dimension of the high-dimensional vector.

[0157] S22: linearly project each feature block using the projection matrix W proj to calculate the high-dimensional vector;

[0158]

[0159] where I patched represents the image matrix after the feature block is unfolded, and X patch represents the high-dimensional vector after the feature block is projected.

[0160] S3: add the high-dimensional vectors of each feature block block by block, and perform Transformer encoding to generate a high-dimensional image embedding.

[0161] S3 includes the following steps:

[0162] S31: add the high-dimensional vector X patch of each feature block to the position encoding P to obtain the embedding information X in ; P represents the position encoding P of the feature block, and the position encoding is the serial number of the feature block.

[0163] S32: input X patch into the Transforme encoder to gradually extract the global semantic information in the input embedding matrix to generate the image embedding.

[0164] The hint encoder receives the hint data collected from the segmentation mask as input and generates a hint embedding.

[0165] The hint data includes point hints and box hints. The extraction steps are as follows:

[0166] Z1: Overlap the first image data and the second image data with each other, and regard the pixel points with the same pixel values in the first image data and the second image data as background regions, and regard the regions with different pixel points as foreground regions.

[0167] Z2: Randomly select k points from the foreground regions to generate a positive point sample set M GT Randomly select m points from the background regions to generate a negative point sample set M HT Add a random offset to each positive point and each negative point.

[0168] Z3: Obtain the minimum and maximum coordinates of the foreground regions to generate a minimum bounding box.

[0169] Z4: Take the positive point sample set M GT and the negative point sample set M HT as point hints, and take the minimum bounding box as a box hint.

[0170] The convolutional network performs convolution operation on the image embedding to generate convolutional features.

[0171] The convolutional network includes an input layer, a convolutional layer, an activation function layer, and a pooling layer.

[0172] The input layer receives the image embedding, and the data dimension is determined by the output of the image encoder.

[0173] The image embedding [B, C, H, W], where B represents the batch size, C represents the number of channels, and H and W represent the height and width respectively.

[0174] The convolutional layer includes three parts, namely front-end convolution, middle-end convolution, and back-end convolution.

[0175] The front-end convolution uses a small size convolution kernel for convolution processing. Such a setting can preserve local details of the image while performing preliminary feature extraction on the image embedding, so that the network begins to learn basic features such as edges and textures in the image.

[0176] The middle-end convolution uses a larger convolution kernel, for example, a 5x5 convolution kernel.

[0177] The back-end convolution uses a 1x1 convolution kernel, and the number of convolution kernels is determined according to the input requirements of the subsequent mask encoder, for example, set to 32.

[0178] Activation function layer: ReLU activation function is added after the front-end convolution, the middle convolution and the back-end convolution.

[0179] Pooling layer: average pooling is performed between the middle convolution and the back-end convolution.

[0180] Mask encoder: input convolutional features and prompt embeddings, align the prompt embeddings with the convolutional features to generate an image mask.

[0181] The training method of the SAM model in step 3 includes the following steps:

[0182] Step 31: a standard data set is prepared in advance; the standard data set includes a plurality of samples, and each sample includes first image data, second image data and a segmentation mask.

[0183] The segmentation mask is annotation data, which is a mask correctly segmented by humans.

[0184] Step 32: train the SAM segmentation model through transfer learning.

[0185] Step 33: iteratively train the SAM segmentation model with the standard data set until the desired effect is achieved.

[0186] Step 4: perform morphological operations on the mask image to generate a liquid leakage area.

[0187] Step 4 includes the following steps:

[0188] Step 41: perform an opening operation of first erosion and then dilation on the mask image to obtain mask information;

[0189]

[0190] Where A represents the mask image, B represents the structure element, represents the erosion operation, and represents the dilation operation.

[0191] The meaning of the erosion operation is that the structure element B slides on the mask image A, and if B is completely contained in the foreground region of A, the center pixel is retained as 1, otherwise as 0.

[0192] The meaning of the dilation operation is that the structure element B slides on the mask image A, and if B intersects with the image foreground region, the center pixel is set to 1.

[0193] Step 42: perform a closing operation of first dilation and then erosion on the mask information to obtain mask features.

[0194]

[0195] Where C represents the erosion operation, and B represents the structure element.

[0196] The structural element is a preset binary matrix, for example, a 3x3 binary matrix.

[0197] After the mask features are obtained, the mask features can be filtered by a Gaussian filter to remove possible noise signals.

[0198] Step 43: Extracting connected regions from the mask features, and taking the connected regions as the liquid leakage regions.

[0199] Step 43 includes the following steps:

[0200] Step 431: Labeling the mask image by connected regions, and identifying all continuous foreground regions.

[0201] Pixels with mutually adjacent mask features and the same attribute are classified into one connected region. By scanning each pixel in the mask image, if a pixel is a foreground pixel and has not been labeled, the pixel is taken as a starting point, a four-neighborhood search algorithm is used to spread to the adjacent pixels, and the foreground pixels connected to the adjacent pixels are labeled as the same region number. By traversing the entire mask image, all continuous foreground regions can be identified, which can correspond to the liquid leakage regions on the engine surface.

[0202] Step 432: Calculating the area, position and shape features of each foreground region, and screening out candidate regions meeting the preset conditions as the liquid leakage regions.

[0203] Step 5: Intensity determination of the liquid leakage regions, and judging whether the engine has damage according to the intensity determination result.

[0204] Step 5 includes the following steps:

[0205] Step 51: Area determination of the liquid leakage regions, if the area is less than a preset threshold, the liquid leakage regions do not have engine damage, otherwise, the liquid leakage regions have engine damage.

[0206] For example, if the resolution is 100 pixels / cm, 1 cm 2 corresponds to 100x100 pixels, if the liquid leakage area is less than a preset 0.5 cm 2 , it is indicated that there is no liquid leakage.

[0207] Step 52: Intensity threshold determination of the liquid leakage regions, if the fluorescence intensity is less than a preset dynamic intensity threshold, the liquid leakage regions do not have engine damage, otherwise, the liquid leakage regions have engine damage.

[0208] For example, the area of the liquid leakage region is 1 cm 2 , 1 cm 2The fluorescence intensity corresponding to the liquid leakage area of the engine is 10. If the fluorescence intensity is less than 10, it means that the liquid leakage area is not a real liquid leakage area. If it is greater than 10, it means that the liquid leakage area is a real liquid leakage area, and there is engine damage.

[0209] In a more specific embodiment, the engine surface can be detected for liquid leakage area at different angles. If the same area has leakage phenomenon in multi-angle detection, the area is marked as a liquid leakage area. Otherwise, it is not.

[0210] In a more specific embodiment, the engine surface can be detected for liquid leakage area at different angles. If the same area has leakage phenomenon in multi-angle detection, the area is marked as a liquid leakage area. Otherwise, it is not.

[0211] The above is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An engine leak detection method based on AI image processing, characterized in that, Includes the following steps: Step 1: Obtain surface image information of the engine under idling conditions to obtain a first comparison image; obtain surface image information of the engine under operating conditions to obtain a second comparison image; Step 2: Preprocess and extract features from the first and second comparison images respectively to obtain the first image data and the second image data; Step 3: Input the first image data and the second image data into the SAM model to obtain the mask image of the initial determination area on the engine surface; Step 4: Perform morphological operations on the mask image to generate the leakage area; Step 5: Determine the strength of the leak area and judge whether there is any damage to the engine based on the strength determination results; Step 4 includes the following steps: Step 41: Perform an opening operation on the mask image, first erosion and then dilation, to obtain the mask information; Step 42: Perform a dilation-erosion closing operation on the mask information to obtain the mask features; Step 43: Extract connected regions from the mask features and use the connected regions as leakage areas; Step 5 includes the following steps: Step 51: Determine the area of ​​the leak. If the area is less than a preset threshold, there is no engine damage in the leak area; otherwise, there is engine damage. Step 52: Determine the intensity threshold of the leakage area. If the fluorescence intensity is less than the preset dynamic intensity threshold, there is no engine damage in the leakage area; otherwise, there is engine damage.

2. The engine leak detection method based on AI image processing according to claim 1, characterized in that, The preprocessing in step 2 includes the following steps: Step 21: Use the first or second comparison image as a preprocessed image, and apply Gaussian filtering to the preprocessed image to obtain a Gaussian filtered image; Step 22: Perform adaptive histogram equalization on the Gaussian filtered image to generate an equalized image; Step 23: Perform Gamma correction on the equalized image to generate either the first image data or the second image data.

3. The engine leak detection method based on AI image processing according to claim 2, characterized in that, Step 21 includes the following steps: Step 211: Set the Gaussian kernel size and calculate the Gaussian function value G(x, y) for each preprocessed image within the Gaussian kernel. ； x and y represent the coordinates of a point within the Gaussian kernel relative to the center point of the Gaussian kernel, σ represents the standard deviation of the Gaussian distribution, k represents the radius of the Gaussian kernel, π represents pi, and exp represents the natural exponential function. Step 212: Filter each pixel in the preprocessed image using the Gaussian function value G(x, y) to obtain a Gaussian filtered image; ； in, Let represent the pixel value at position (i, j) in the Gaussian filtered image. Indicates the location of the preprocessed image. The pixel value at the location, m and n represent the offset of the Gaussian kernel, m∈[-k, k], n∈[-k, k], and i and j represent the x and y coordinates of the pixel, respectively.

4. The engine leak detection method based on AI image processing according to claim 2, characterized in that, Step 22 includes the following steps: Step 221: Count the number of pixels n at each gray level in the Gaussian filtered image. j ; Step 222: Calculate the CDF value for each gray level; ； r k The original gray levels of the Gaussian-filtered image are represented by , N represents the total number of pixels in the Gaussian-filtered image, and k represents the upper limit of the currently calculated gray levels. This indicates that the grayscale value in the image is less than or equal to r. k The percentage of pixels is used to describe the cumulative probability distribution of gray levels; Step 223: Linearly map the CDF to the interval, and replace the gray values ​​in the Gaussian image with the mapped values ​​to generate a balanced image: ； round represents the rounding function, L represents the total number of gray levels, and s k This represents the grayscale value after mapping.

5. The engine leak detection method based on AI image processing according to claim 2, characterized in that, Step 23 includes the following steps: Step 231: Normalize the gray values ​​of the equalized image; Step 232: Perform a power transformation on the normalized grayscale values: ； in, This represents the pixel value at (x, y) after the power transformation. To equalize the pixel values ​​at pixel (x, y) in the image; Step 233: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Gamma correction is completed by performing inverse normalization.

6. The engine leak detection method based on AI image processing according to claim 1, characterized in that, The training method for the SAM model in step 3 includes the following steps: Step 31: Prepare a standard dataset in advance; the standard dataset includes several samples, each sample including first image data, second image data and segmentation mask; Step 32: Train the SAM segmentation model through transfer learning; Step 33: Iteratively train the SAM segmentation model using a standard dataset until the expected results are achieved.

7. The engine leak detection method based on AI image processing according to claim 6, characterized in that, The SAM segmentation model includes: An image encoder is used to take first image data and second image data as input and generate a high-dimensional image embedding. The prompt encoder takes prompt data collected from the segmentation mask as input and generates a prompt embedding. Convolutional networks perform convolution operations on image embeddings to generate convolutional features; The mask encoder takes convolutional features and cue embeddings as input and aligns the cue embeddings with the convolutional features to generate an image mask.

8. The engine leak detection method based on AI image processing according to claim 1, characterized in that, When acquiring the first and second comparison images of the engine components, ultraviolet light was used as the illumination source. While the engine is running, add a trace amount of fluorescent agent to the engine oil and coolant.

9. An engine leak detection system based on image processing, characterized in that, The engine leakage detection method based on AI image processing, as described in any one of claims 1 to 8, is used to determine whether there is damage to the engine.

Images