Intelligent detection method and system for vehicle-mounted camera module

Abstract

The invention relates to the technical field of industrial internet platforms, and discloses an intelligent detection method and system for a vehicle-mounted camera module, and the method comprises the steps: capturing an image sequence through a vehicle-mounted camera, and obtaining a dust accumulation degree index; a Sobel operator is adopted to calculate the gray value gradient of the edge area of the protective cover; performing feature extraction on the gradient by adopting a convolutional neural network model, and determining a sealing integrity score; if the leakproofness integrity score is lower than a preset threshold value, sensor offset induced by vibration is analyzed, and the potential damage risk level is judged; obtaining an overall working state evaluation value according to the dust accumulation degree index, the sealing integrity score and the potential damage risk level; and performing fault prevention and real-time performance calibration according to the evaluation value. According to the method, the overall working state of the camera module in the high-dust and strong-vibration environment can be monitored and accurately judged in real time, and the image stability and equipment reliability of the camera are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial internet platform and intelligent detection technology, and particularly relates to an intelligent detection method and system for a vehicle camera module in a mine vehicle and the like in a harsh environment. BACKGROUND

[0002] The vehicle camera is a core component for a mine vehicle to perceive the environment and ensure safety in production. The harsh environment of high dust, strong vibration and low illumination in a mine poses high requirements on the performance and reliability of the camera module.

[0003] In the prior art, the detection of the vehicle camera mainly relies on manual inspection or simple performance testing, which is difficult to monitor the state of the camera in a complex working environment in real time and dynamically. For example, the traditional detection method may ignore the influence of continuous dust accumulation on the lens clarity or the potential damage of vibration to the stability of the sensor. In addition, the dust not only adheres to the surface of the lens, but also may penetrate into the inside of the protective cover, affecting the sealing property and further increasing the risk of damage to the internal sensor. Therefore, the prior art has the technical problem that the key components of the camera (such as lens cleanliness and protective cover sealing property) cannot be evaluated in real time and comprehensively, resulting in that the detection result is inconsistent with the actual working state and potential faults cannot be found in time. SUMMARY

[0004] The present application provides an intelligent detection method and system for a vehicle camera module to solve the problem that the detection method in the prior art cannot adapt to harsh environments such as high dust and strong vibration and cannot accurately judge the working state of the camera module in real time.

[0005] In a first aspect, the present application provides an intelligent detection method for a vehicle camera module, comprising: capturing an image sequence by a vehicle camera, and analyzing the image sequence by an image processing algorithm to obtain a lens surface dust accumulation degree index; calculating a protective cover edge region gray value gradient by a Sobel operator according to the dust accumulation degree index; extracting features of the gradient by a convolutional neural network model according to the gray value gradient to determine a sealing integrity score; extracting vibration-related displacement information from the image sequence according to the sealing integrity score, calculating a sensor offset amplitude by the displacement information, and judging a potential damage risk level; simulating the influence of environmental interference on image quality by an image processing algorithm according to the dust accumulation degree index, the sealing integrity score and the potential damage risk level to obtain an overall working state evaluation value; According to the overall working state evaluation value, a dynamic change trend is determined by comparing the difference between the sequence and a reference; If the overall working state evaluation value exceeds a set normal range, a historical image data sequence is obtained as a reference, a dynamic change trend is determined based on the difference between the current image sequence and the reference; According to the dynamic change trend, a cleanliness recovery simulation path is generated using an image processing algorithm, and a fault prevention adjustment parameter is obtained.

[0006] In an optional embodiment, the analysis of the image sequence to obtain a lens surface dust accumulation degree index includes: capturing an image sequence of a mine environment by a vehicle-mounted camera; analyzing pixel distribution changes between consecutive frames in the image sequence using an image processing algorithm to determine a dust accumulation area; and calculating a dust coverage ratio based on the dust accumulation area to obtain the dust accumulation degree index of the lens surface.

[0007] In an optional embodiment, the feature extraction of the gradient using a convolutional neural network model to determine a sealing integrity score includes: inputting the gray value gradient into a pre-trained convolutional neural network model to generate a gradient feature vector; inputting the gradient feature vector into a softmax classifier to output a probability distribution of the sealing belonging to a category, and calculating a sealing integrity score based on the probability distribution weighting.

[0008] In an optional embodiment, the extraction of vibration-related displacement information, the calculation of a sensor offset amplitude based on the displacement information, and the determination of a potential damage risk level include: if the sealing integrity score is lower than a preset threshold, extracting a vibration-induced displacement vector from the image sequence; calculating a sensor offset amplitude based on the displacement vector, and determining the potential damage risk level based on the correlation between the sensor offset amplitude and historical damage data.

[0009] In an optional embodiment, the obtaining of an overall working state evaluation value based on the dust accumulation degree index, the sealing integrity score, and the potential damage risk level includes: simulating the influence of environmental interference on image quality using an image processing algorithm; and obtaining the overall working state evaluation value based on the influence of the environmental interference, the dust accumulation degree index, and the sealing integrity score through fuzzy logic reasoning.

[0010] In an alternative embodiment, if the overall working state evaluation value exceeds the set normal range, historical image data sequences are obtained as a reference, differences between the current image sequences and the reference are compared, and a dynamic change trend is determined based on the differences, including: if the overall working state evaluation value exceeds the set normal range, historical image data sequences are obtained as a reference to obtain a reference image sequence; a convolutional neural network is used to extract feature vectors of the current image sequence and the obtained reference image sequence respectively; the Euclidean distance between the feature vectors is calculated to quantify the differences, and a dynamic change trend is determined based on the differences through time series analysis.

[0011] In an alternative embodiment, the method further comprises: using the fault prevention adjustment parameter to fuse the sealing integrity score and the sensor offset amplitude to perform real-time calibration on the vehicle-mounted camera module.

[0012] In a second aspect, the present application provides an intelligent detection system for a vehicle-mounted camera module, comprising: A capture analysis module is configured to capture image sequences by a vehicle-mounted camera, and analyze the image sequences to obtain a dust accumulation degree index of a lens surface using an image processing algorithm. A gradient calculation module is configured to calculate a gray value gradient of an edge region of a protective cover based on the dust accumulation degree index using a Sobel operator. A sealing detection module is configured to extract features of the gradient using a convolutional neural network model based on the gray value gradient to determine a sealing integrity score. A risk judgment module is configured to extract vibration-related displacement information from the image sequences based on the sealing integrity score, calculate a sensor offset amplitude based on the displacement information, and determine a potential damage risk level. A state evaluation module is configured to simulate the influence of environmental interference on image quality using an image processing algorithm based on the dust accumulation degree index, the sealing integrity score, and the potential damage risk level to obtain an overall working state evaluation value. A trend analysis module is configured to obtain historical image data sequences as a reference if the overall working state evaluation value exceeds the set normal range, compare differences between the current image sequences and the reference, and determine a dynamic change trend based on the differences. A calibration module is configured to generate a cleanliness recovery simulation path using an image processing algorithm based on the dynamic change trend to obtain a fault prevention adjustment parameter.

[0013] Compared with the prior art, the present application has the following beneficial effects: The application can quantitatively evaluate the dust accumulation degree of the lens surface by capturing image sequences in real time and analyzing pixel distribution; can accurately determine the sealing integrity of the protective cover by analyzing the gray value gradient of the protective cover edge and combining a convolutional neural network; can predict potential physical damage risks by monitoring the vibration-induced sensor offset when the sealing performance decreases. By fusing multi-dimensional data, the method can accurately evaluate the overall working state of the camera module and generate preventive maintenance parameters for real-time calibration, thereby significantly improving the reliability, stability and image quality of the camera in harsh environments, and providing efficient technical support for safety production in complex scenes such as mines. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a flow diagram of an intelligent detection method for a vehicle-mounted camera module provided by an embodiment of the application; Figure 2 is a structural diagram of an intelligent detection method for a vehicle-mounted camera module provided by an embodiment of the application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0016] Referring to Figure 1 The first embodiment of the application provides an intelligent detection method for a vehicle-mounted camera module, specifically comprising the following steps: S101, capturing an image sequence by a vehicle-mounted camera, and analyzing the image sequence to obtain a dust accumulation degree index of a lens surface by using an image processing algorithm; S102, calculating a gray value gradient of a protective cover edge region according to the dust accumulation degree index by using a Sobel operator; S103, extracting features of the gradient by using a convolutional neural network model according to the gray value gradient, and determining a sealing integrity score; S104, extracting vibration-related displacement information from the image sequence according to the sealing integrity score, calculating a sensor offset amplitude by using the displacement information, and judging a potential damage risk level; S105, simulating the influence of environmental interference on image quality by using an image processing algorithm according to the dust accumulation degree index, the sealing integrity score and the potential damage risk level, and obtaining an overall working state evaluation value; S106, if the overall working state evaluation value exceeds the set normal range, obtaining historical image data sequence as a reference, comparing the difference between the current image sequence and the reference, and determining the dynamic change trend based on the difference; S107, according to the dynamic change trend, generating a cleanliness recovery simulation path using an image processing algorithm, and obtaining a fault prevention adjustment parameter.

[0017] In step S101, an image sequence is captured by a vehicle-mounted camera, and an image processing algorithm is used to analyze the image sequence to obtain a dust accumulation degree index of the lens surface.

[0018] It should be noted that the purpose of this step is to quantify the degree of contamination of the lens surface. The vehicle-mounted camera as a data acquisition source continuously acquires video streams to provide raw data for subsequent analysis. The implementation of this step can be further divided into image sequence acquisition, image preprocessing, pixel distribution change analysis, dust area identification and quantification, and index generation and smoothing.

[0019] Regarding image sequence acquisition, in order to ensure the real-time and accuracy of the analysis, the camera needs to work with certain parameter specifications. Specifically, the vehicle-mounted camera in this embodiment performs continuous image capture at a frequency of 30 frames per second (30fps), and the image resolution of each frame is set to 1920x1080 pixels. Such a setting can ensure smoothness of the picture while providing high enough pixel details to facilitate subsequent algorithms to identify small dust particles or their aggregation areas. In order to balance between computational efficiency and data representativeness, the system does not process every frame, but uses a time sampling strategy to read the first 100 frames of the sequence as samples through the OpenCV library, records the initial timestamp t0=0 seconds, and then extracts one frame every second to form a 100-frame sub-sequence as the basis for analysis.

[0020] Regarding image preprocessing, due to the complex light and electromagnetic interference in the mine environment, the original image sequence may contain a large amount of noise, which will interfere with the subsequent judgment of pixel changes. Therefore, before analysis, the image needs to be preprocessed. In this embodiment, a Gaussian filter is used to smooth and denoise each frame of image. Specifically, the standard deviation σ of the Gaussian function can be set to 2.0. The choice of standard deviation determines the degree of smoothing, and a larger σ value will make the image more blurred, but the denoising effect will also be stronger. σ=2.0 here is a relatively optimal choice between retaining image edge details and effective denoising.

[0021] Regarding pixel distribution change analysis, this is the core of identifying dust accumulation. The system determines whether new dust has adhered by comparing and analyzing the pixel information changes between image frames within the analysis window. This process includes two parallel analysis dimensions: The first dimension is global pixel distribution change analysis. The system computes the three-channel (R, G, B) color histogram of each frame of image. Specifically, the distribution of R channel pixel values ranging from 0 to 255 is obtained by the cumulative histogram function =cv2.calcHist([img], [0], None,

[256] , [0, 256]) where the function is used to count the pixel data of the [0] channel (i.e. R channel) in the input image [img]. Here, None indicates that the whole image is calculated (no mask is used), and

[256] indicates that the pixel value range [0, 256] (i.e. 0, 255) is divided into 256 "boxes". The function call outputs an array , where the number of occurrences of each pixel value from 0 to 255 in the R channel is recorded. Subsequently, the G channel and B channel are processed similarly to obtain the three-channel histogram , , When new dust covers the lens, the color tone and brightness of the whole picture will change, causing the shape of the histogram to change. To quantify this change, the Earth Mover's Distance (EMD) algorithm is introduced in this embodiment. EMD can calculate the difference between two distributions, in this scenario, it calculates the "distance" between the histogram of the current frame and the histogram of the initial reference frame (or the previous frame). Specifically, if an EMD threshold of 0.05 is set, when the calculated EMD value exceeds the threshold, the system determines that there is a significant change in pixel distribution.

[0022] The second dimension is specific pixel region change analysis. Considering that mine dust is usually dark in color, its attachment will increase the proportion of low-brightness pixel regions in the image. Therefore, the system will pay special attention to the "dark area" where the pixel value is less than a certain threshold (for example, pixel value less than 50). The system calculates the proportion of the total number of dark area pixels in the total number of pixels (1920 1080) of each frame of image, denoted as Within an analysis window, if shows a clear growth trend over time, it can be strong evidence that dust is accumulating. To determine this trend, the system can perform a linear regression fit on the sequence within a time window, and calculate its slope k, which represents the dust accumulation rate. For example, if the slope k is greater than a preset value (e.g. k > 0.01), it is determined that there is a significant dust accumulation trend.

[0023] Regarding the dust area identification and quantification, after confirming the existence of dust accumulation through the above analysis, the system will further accurately locate the dust area through a pixel difference algorithm. It should be noted that the core idea of this algorithm is to identify the pixel points that have changed significantly as dust by comparing the current contaminated image with a "clean" reference image. This "clean" background reference frame can be a benchmark image taken and stored in the same scene after the camera is installed or cleaned.

[0024] Specifically, the calculation process is as follows: first, let the gray image of the current frame be , and the gray image of the background reference frame be , where (x, y) represents the pixel coordinates. The system will calculate the absolute difference D(x, y) of the gray value of each pixel in the image, and the calculation formula is: Next, the system will set a reasonable difference threshold T, for example, in the gray scale of 0-255, T can be set to 30. This threshold is used to distinguish between significant changes caused by real dust and minor changes caused by normal light fluctuations or sensor noise.

[0025] The system will traverse all the pixel differences D(x, y), and generate a binary "dust area mask" Mask(x, y) according to the following rules: if D(x, y) > T, then Mask(x, y) = 1 (indicating that the pixel is dust); if D(x, y) ≤ T, then Mask(x, y) = 0 (indicating that the pixel is background).

[0026] Finally, by calculating the total number of pixels with a value of 1 in the mask and dividing it by the total number of pixels in the image, we can get the "dust coverage ratio" .

[0027] The calculation formula is: .

[0028] For example, suppose we analyze a 100 100 pixel image area, the total number of pixels in this area is 10,000. After performing pixel difference and binary processing, the system counts that there are 1,500 pixel points with a value of 1 in the "dust area mask". Then, the dust coverage ratio of this area is calculated as follows: =(1500 / 10000) 100%=15%. This 15% value can be an important part of the "dust accumulation degree index" in the subsequent steps.

[0029] Finally, regarding the index generation and smoothing, the system fuses the structures of the above multi-dimensional analysis to generate the final "lens surface dust accumulation degree index", denoted as D. When fusing, the dynamic characteristics of dust accumulation need to be considered. For example, the dust accumulation rate k does not grow linearly indefinitely, but tends to be saturated. In order to make the index reflect the saturation effect of accumulation, an exponential function model is used to calculate D in this embodiment: D = 10 (1 - exp(-k 10)). Where k is the slope of the change in the dark area ratio, i.e. the dust accumulation rate. This formula maps the slope k to an index D ranging from 0 to 10. Finally, the index D can be combined with other static indicators such as the "dust coverage ratio" through fuzzy logic reasoning to obtain a comprehensive evaluation value. At the same time, in order to avoid the index fluctuating sharply due to single measurement error, the system also uses the mean filtering algorithm to smooth the continuously calculated D values to generate a stable degree sequence. Finally, when the smoothed index D is greater than the preset threshold (i.e. D > 5), the system triggers a cleaning alarm.

[0030] In step S102, according to the dust accumulation degree index, the Sobel operator is used to calculate the gray value gradient of the edge region of the protective cover.

[0031] It should be noted that the sealing performance of the protective cover is the key to preventing dust from entering the module. When the dust accumulation outside the lens is severe, the risk of sealing failure also increases. This step evaluates the sealing performance by analyzing the micro-texture features of the edge of the protective cover.

[0032] The implementation of this step can be divided into sub-processes such as edge image acquisition and gray gradient calculation.

[0033] Regarding edge image acquisition, the system needs to image the edge of the protective cover and the camera housing joint specifically. This may require a separate high-resolution industrial camera with macro function, or use the main camera to shoot at a specific angle and focal length. In this embodiment, a 5 million pixel high-resolution camera is used to capture the image of the edge region to ensure that there are enough details for subsequent analysis. After obtaining the image, the edge contour region of the protective cover is accurately separated by image segmentation algorithm to form a gray image containing only edge information.

[0034] Regarding the calculation of the gray scale gradient, which is the key to quantifying the edge texture feature. The gray scale gradient reflects the rate and direction of the change in pixel intensity in the image. In a perfect sealing edge, the texture is usually uniform and continuous; when there is a crack or dust particles are embedded, the pixel value of the region will jump sharply, thus producing an abnormal strong gradient. This embodiment adopts the Sobel operator to calculate the gray scale gradient. For example, let's say we are focusing on a 3x3 neighborhood centered on pixel P5 in the image, whose gray value matrix is and its position code P1 to P9 corresponds to the following: [P1, P2, P3] = [10, 15, 20], [P4, P5, P6] = [25, 30, 35], [P7, P8, P9] = [40, 45, 50] First, the horizontal gradient Gx is calculated by the horizontal convolution kernel [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]. The calculation formula is the weighted sum of the neighborhood pixel values and the kernel, that is: Its physical meaning is to calculate the weighted difference of the pixel values on the left and right sides of the center point, which reflects the change trend in the horizontal direction. Substituting the numerical value, we get Gx = 40. Then, the vertical gradient Gy is calculated by the vertical convolution kernel [[1, 2, 1], [0, 0, 0], [-1, -2, -1]]. The calculation formula is: to reflect the change trend in the vertical direction. Substituting the numerical value, we get Gy = (10+2 15+20) - (40+2 45+50) = -120. Finally, the total gradient amplitude G, which reflects the most severe degree of pixel change and is not affected by the direction, is calculated by the formula G = sqrt(Gx² + Gy²). Substituting the values of Gx and Gy, we get G = sqrt(40² + (-120)²) ≈ 126.5. The purpose of obtaining this calculation result is to generate a complete gradient map by performing the above calculation on each pixel point of the edge region of the protective cover. Each pixel value (such as 126.5 in the example) on this gradient map represents the edge intensity of the corresponding point in the original image, providing a quantitative and accurate input feature for the subsequent convolutional neural network model.

[0035] In step S103, according to the gray value gradient, a convolutional neural network model is used to extract features from the gradient to determine the sealing integrity score.

[0036] Regarding CNN feature extraction, traditional image processing methods struggle to reliably extract patterns related to sealing failure from complex gradient maps. Therefore, this embodiment introduces a Convolutional Neural Network (CNN). CNNs can automatically learn and extract abstract features from low to high levels of an image through multiple convolutional and pooling operations. Specifically, this embodiment employs a pre-trained CNN model with a VGG16 architecture, containing 13 convolutional layers and 3 fully connected layers. The model receives the gradient image calculated in the previous step as input. In the convolutional layers, small 3x3 convolutional kernels and the ReLU activation function are used to extract features layer by layer. For example, shallow networks can extract low-level features such as edge directions; as the network depth increases, the model can learn high-level features such as texture entropy values. Finally, these high-level features are flattened into a "feature vector".

[0037] Regarding the determination of the integrity score, the feature vector output by the CNN is matched with a "preset sealing template." This template is actually a set of weight parameters formed in the fully connected layer after the model has been trained on a large number of edge samples with known labels ("intact" or "invalid"). The sealing integrity score is output by weighted summation (inner product) of the feature vector and this set of weights, followed by a Softmax classifier. In this embodiment, the calculation process is as follows: This formula calculates the inner product of the feature vector and the weight vector, and then normalizes it by dividing by the feature norm to obtain the final sealing integrity score. For example, an extracted feature vector of [0.7, 0.5], after performing the inner product operation with the corresponding weight vector [0.6, 0.8], normalizes to obtain a final score of 0.85, indicating high integrity. This score is compared with the accumulated index threshold (e.g., >0.7 is considered acceptable) to achieve automated diagnosis and avoid human intervention.

[0038] In step S104, based on the sealing integrity score, vibration-related displacement information is extracted from the image sequence, and the sensor offset amplitude is calculated using the displacement information to determine the potential damage risk level.

[0039] It should be noted that this step is based on the judgment in step S103: once the sealing of the protective cover fails, the severe vibrations generated when the mine car travels on rough roads may be directly transmitted to the internal CMOS image sensor, causing it to undergo a slight physical displacement or angular deflection, i.e., "sensor offset". This offset can cause image distortion, focusing failure, and other consequences. Therefore, immediately initiating monitoring of sensor stability after confirming a decrease in sealing is a crucial step in preventing major failures.

[0040] The implementation of this step can be broken down into sub-processes such as trigger condition judgment, displacement vector extraction, offset magnitude calculation, and risk level judgment.

[0041] Regarding the trigger condition judgment, the system will preset a safety threshold for the sealing integrity score, such as 85%. The system will then continuously monitor the score output in step S103. Once the score falls below this preset threshold, the subsequent processes of this step will be automatically activated. For example, in a certain monitoring period, if the score calculated in step S102 is 82%, which is lower than the threshold, the system will immediately begin extracting and analyzing the displacement vector.

[0042] Regarding displacement vector extraction, to capture the minute movements of the sensor caused by vibration, the system needs to analyze the continuous image sequence captured by the main camera. When the sensor is displaced, the entire frame it captures will also translate or rotate, which is reflected in the image as all pixels undergoing a consistent macroscopic motion. This embodiment uses optical flow to calculate this motion, specifically the Lucas-Kanade algorithm. This algorithm calculates the velocity and direction of a pixel's motion by analyzing the positional changes of its small neighborhood over two consecutive frames; this result is a "displacement vector." By calculating and averaging the displacement vectors of multiple feature points in the image, the macroscopic motion vector of the entire image can be obtained, which is approximately equal to the sensor's displacement vector.

[0043] Regarding the calculation of the offset amplitude, this embodiment uses the Euclidean distance formula to calculate the sensor offset amplitude. Specifically, if within a very short time interval (e.g., between two frames, approximately 1 / 30 of a second), the optical flow method calculates that the horizontal and vertical components of the sensor's displacement vector are 0.15mm and 0.08mm respectively, then the sensor offset amplitude at that moment can be calculated using the formula √(0.15² + 0.08²), resulting in approximately 0.17mm. If the offset amplitude exceeds a preset safety value (e.g., 0.1mm), then a risk level assessment is initiated.

[0044] Regarding risk level assessment, a single offset amplitude value cannot be directly equated with damage risk, as risk is also related to the vibration frequency, duration, and the sensor's vibration resistance. Therefore, a comprehensive model is needed to determine the "potential damage risk level." This embodiment employs a machine learning model trained on historical data to accomplish this task. Specifically, a random forest classifier is used. The model's training data comes from a large number of laboratory destructive tests and historical fault records. The model's input features can include: the currently calculated sensor offset amplitude (e.g., 0.17 mm), the device vibration frequency measured by an accelerometer (e.g., 10 Hz), and the camera's material fatigue coefficient, etc. The model's output is a probability value representing "high risk." For example, inputting the above features into the trained model, the model outputs a risk probability of 78%. Finally, the system can map the risk level to an intuitive risk grade based on this probability value and a preset range (e.g., >75% for high risk, 40%-75% for medium risk, <40% for low risk). When a high risk is determined, the system can immediately generate an alarm signal to notify the background monitoring system.

[0045] In step S105, based on the dust accumulation level index, the sealing integrity score, and the potential damage risk level, an image processing algorithm is used to simulate the impact of environmental interference on image quality to obtain an overall working status evaluation value.

[0046] It should be noted that this is a comprehensive assessment step that integrates multi-source information, aiming to derive a single indicator that can comprehensively reflect the current health status of the camera.

[0047] Specifically, the system uses a Gaussian noise model to simulate these random environmental disturbances. For example, a noise level with a mean of 0 and a standard deviation of 0 can be added. A Gaussian noise layer was then added. The potential impact of the environment was then quantified by comparing image quality before and after the noise was added. Image quality was evaluated using the peak signal-to-noise ratio (PSNR) metric.

[0048] The formula for calculating PSNR is: ,in PSNR is the maximum possible value of the image pixel (255 in this case), and MSE is the mean square error between the original image and the noisy image. The higher the PSNR, the better the image quality.

[0049] By calculating the decrease in PSNR after adding noise, a quantified "environmental interference impact value" can be obtained, denoted as I. For example, if the original image PSNR is 30dB and becomes 25.78dB after adding noise, the impact value can be quantified as 0.14 according to the preset formula I=1-PSNR / 30, reflecting the blurring effect of environmental factors such as humidity or vibration on the monitoring image.

[0050] Finally, this embodiment employs a fuzzy logic reasoning system for fusion evaluation. Specifically, the system first needs to define membership functions for the input variables (D, S, R, I). The membership function maps a precise input value (e.g., dust index D=6) to the membership degree of one or more fuzzy sets (e.g., "low pollution", "medium pollution", "high pollution"). For example, a triangular membership function can be defined for the risk level R. Then, a set of "fuzzy rules" needs to be established. These rules are formulated by domain experts and are in the form of "IF-THEN" statements, such as: "IF dust accumulation 'high' AND sealing 'low' THEN working condition 'poor'".

[0051] Finally, the fuzzy inference engine calculates (using methods like Mamdani or Sugeno inference) based on the membership degrees of all input variables and all fuzzy rules, and after a "defuzzification" process, outputs a precise, final "overall working state evaluation value" (denoted as W). For example, the system uses a weighted average and correction method. First, it... and Combined into a basic risk value Then, the baseline risk value is corrected using the environmental disturbance impact value I. Finally, the evaluation value is obtained through a mapping function. ,For example Assuming D=6, S=0.954, I=0.14, then the calculated... It is approximately 6.99. This final value represents the overall health of the camera, facilitating subsequent threshold assessment and maintenance decisions.

[0052] In step S106, if the overall working status evaluation value exceeds the set normal range, a historical image data sequence is obtained as a benchmark, and the difference between the current image sequence and the benchmark is compared to determine the dynamic change trend based on the difference.

[0053] It should be noted that this step aims to upgrade from static anomaly assessment to dynamic trend prediction. By comparing with historical health data, the degree of "deviation" can be quantified, and its future trend can be predicted through time-series analysis. Specifically, this step is triggered when the overall work status assessment value calculated by S105 exceeds a preset normal range. For example, the normal range can be set to 0.8 to 1.2; if the currently calculated assessment value is 1.5, the system will automatically initiate this step.

[0054] The system first retrieves a sequence of images from its internal database that have been marked as operating in a healthy state over a past period (e.g., the past 30 days), serving as a "reference baseline." Assuming 1000 historical images are acquired, to ensure fairness and accuracy in the comparison, the system performs the same preprocessing on these images as on the current image sequence, including noise removal using Gaussian filtering (e.g., kernel size 5x5, standard deviation 1.0) and normalizing the pixel values ​​of all images to the [0, 1] range. To accurately compare the difference between the current state and the historical baseline, this embodiment employs a ResNet-50 convolutional neural network model. This model extracts features from both the currently acquired image sequence (e.g., the most recent 10 images) and the historical baseline sequence. ResNet-50 contains 50 convolutional layers, capable of extracting very deep and abstract features from the images. The model outputs a high-dimensional feature vector for each image sequence through its fully connected layers. The system then quantifies the difference between the average feature vector of the current sequence and the average feature vector of the historical baseline sequence by calculating the mean squared error (MSE).

[0055] To illustrate this calculation process more clearly, let's assume, for simplicity, that the feature vector output by the CNN is 4-dimensional. The average feature vector of the current sequence is... = [0.75, 0.32, 0.81, 0.55], while the average eigenvector of the historical health benchmark is = [0.65, 0.30, 0.78, 0.51]. The calculation of MSE first requires finding the squares of the differences between the two vectors in each dimension, i.e., [(0.75-0.65)², (0.32-0.30)², (0.81-0.78)², (0.55-0.51)²], which yields [0.01, 0.0004, 0.0009, 0.0016]. Then, these squared differences are summed to obtain a total of 0.0129. Finally, this sum is divided by the number of dimensions n (here n=4), a step derived from the formula... Definition. Therefore, the final MSE value is The purpose of obtaining this calculation result is that this MSE value (e.g., 0.003225) provides an objective and comparable quantitative indicator of "the degree to which the current state deviates from the normal baseline." It will serve as input to the subsequent LSTM time-series analysis model to predict the future trend of this deviation. Specifically, the input to the LSTM model is a sequence of difference vectors over a recent period (e.g., a sequence length of 10), which can have 128 hidden layer units and undergo sufficient training epochs (e.g., 100 epochs) to learn the change patterns. After training, the model can predict the difference value over a future period (e.g., the next 5 days). If the prediction shows a continuously increasing trend in the difference value (e.g., from 0.01 to 0.015), the system determines that the equipment status has a "dynamic deterioration trend" and can match this trend with the business rule base. When the predicted difference value exceeds a preset maintenance alarm threshold (e.g., 0.012), the system will automatically generate a maintenance request and send it to the equipment management system, thus achieving a closed loop of predictive maintenance.

[0056] In step S107, based on the dynamic change trend, an image processing algorithm is used to generate a cleanliness restoration simulation path and obtain fault prevention adjustment parameters. Specifically, the execution process of this step is as follows: First, the system needs to accurately identify which areas of the lens have the most severe dust accumulation. This embodiment employs a deep learning-based image segmentation model, specifically the U-Net architecture. This model is trained on a large number of labeled images containing different dust densities and morphologies (e.g., using 1000 labeled images, a learning rate of 0.001, and 50 epochs). After training, it can accurately segment the currently captured image, generating a binary "dust distribution mask." This mask image clearly indicates which pixels are covered by dust, thus revealing the dust accumulation pattern (e.g., whether it is concentrated in the center of the lens or distributed radially). After obtaining the dust distribution mask, the system generates an optimal "cleanliness restoration simulation path." This path is designed to guide future automated cleaning devices (if equipped) or manual maintenance to remove dust in the most efficient way.

[0057] This embodiment uses A Algorithm. The system first virtualizes the lens surface as a grid map (e.g., a grid size of 10x10 pixels). A The algorithm's cost function considers two factors simultaneously: the physical distance to the next grid and the dust coverage of the target grid. Thus, the algorithm prioritizes planning a path that passes through high-dust-density areas while maintaining a relatively short total length. Ultimately, the algorithm outputs a path sequence composed of coordinate points, such as [(50,50), (60,55), ...]. Finally, the system needs to convert this simulated path into specific, executable maintenance parameters. This process is accomplished using an optimization algorithm; in this embodiment, Particle Swarm Optimization (PSO) is employed.

[0058] It's important to note that PSO (Pool Swarm Optimization) is a swarm intelligence algorithm that simulates the foraging behavior of bird flocks. In this scenario, each "particle" represents a set of candidate maintenance parameters (a potential solution), such as {cleaning frequency, cleaning pressure}. The algorithm aims to guide the entire "particle swarm" to search the solution space to find the optimal combination of parameters that minimizes the estimated equipment failure rate. A simplified computational example illustrates one iteration of PSO. For instance, suppose our optimization objective is to minimize a cost function Cost(f, p), which combines maintenance cost and expected failure rate. We initialize two particles: Particle 1: initial position x1 = {frequency: 1 time / day, pressure: 1.0 times the standard value}, initial velocity v1 = {0, 0}; Particle 2: initial position x2 = {frequency: 3 times / day, pressure: 1.5 times the standard value}, initial velocity v2 = {0, 0}. First, the system calculates the cost value for each particle's current position. Suppose we calculate Cost(x1) = 10 and Cost(x2) = 8. Next, we define the "individual best position (pbest)" for each particle in the first iteration as its current position. Since Cost(x2) is lower, the "global best position (gbest)" is set to... Finally, the system will update particle 1's velocity and position, allowing it to "learn" towards its individual and global optimal positions. The velocity update is calculated using the formula... Driven by: where w is the inertia weight (e.g., 0.7), c1 and c2 are learning factors (e.g., 1.5), and r1 and r2 are random numbers. Assuming r1 = 0.5 and r2 = 0.8, the velocity update of particle 1's frequency dimension is: = 0.7 0 + 1.5 0.5 (1-1) + 1.5 0.8 (3-1) = 2.4. Similarly, calculate the velocity in the pressure dimension. After obtaining the new velocity vector, use the formula... The position of particle 1 is updated to a new location where the cost might be lower. The optimization goal of this algorithm is to find a set of parameters that minimizes the expected failure rate. For example, the algorithm might conclude that the optimal strategy is to increase the cleaning frequency from once a day to twice a day and adjust the pressure parameters of the cleaning device to 1.2 times the standard value. These specific values ​​are the final "failure prevention adjustment parameters".

[0059] In step S107, after generating a cleanliness restoration simulation path using an image processing algorithm based on the dynamic change trend and obtaining fault prevention adjustment parameters, the process further includes: The vehicle-mounted camera module is calibrated in real time by using the fault prevention adjustment parameters, integrating the sealing integrity score and the sensor offset amplitude.

[0060] It should be noted that this step is the final closed-loop link in the entire intelligent detection process. It not only executes the maintenance suggestions generated in step S107, but more importantly, it performs a comprehensive, real-time integrated calibration and effect verification of the camera's various performance aspects after execution, ensuring that the adjusted camera can reach the preset stable operating standards.

[0061] Specifically, the execution process of this step is as follows: After executing or recommending the execution of the maintenance operation generated by S107, the system first needs to perform a refined evaluation of the current state of the camera. It will integrate two key real-time data: one is the sealing integrity score (from the CNN model output of S103), and the other is the sensor offset amplitude (from the optical flow method and calibration algorithm calculation results of S104). In order to reduce random noise in the sensor offset data, the system can use the Kalman filter algorithm to smooth the sensor pose data of 10 consecutive frames, outputting a more stable and accurate offset value. Then, the system will construct a weighted evaluation model to calculate a comprehensive performance index to guide real-time calibration. For example, the calculation formula of this model can be: Overall performance index = 0.6 Sealing integrity score + 0.4 (1 - Standardized offset magnitude) The weights here (0.6 and 0.4) can be set empirically to reflect the importance of different factors. If the calculated overall performance index is lower than a preset threshold (e.g., a threshold of 60 out of 100), the system will initiate a real-time calibration procedure. Calibration measures may include fine-tuning the camera's internal parameters, such as increasing the lens focal length by 0.1 mm to optimize image sharpness, or adjusting the gain parameters of the image stabilization algorithm to better compensate for sensor vibration. After performing the calibration, the system will recalculate the overall performance index. This process may iterate multiple times until the overall performance index reaches or exceeds a preset stable output threshold (e.g., a stable threshold set to 65). Once the index is met, the system will record and output the current parameters and performance index as the "optimized stable output index." This final index confirms the success of this preventative maintenance and calibration process, marking the completion of the entire intelligent detection closed loop. Through this process, the present invention can not only discover and predict problems, but also proactively intervene and quantitatively verify the intervention effect, thereby ensuring that the camera module maintains optimal working performance in harsh environments.

[0062] Reference Figure 2 The second embodiment of the present invention provides an intelligent detection system for an in-vehicle camera module, characterized in that it includes: The capture and analysis module M201 is directly connected to the vehicle-mounted camera and is used to capture image sequences in real time. After capture, the image processing unit inside the module analyzes the image sequence. Specifically, it quantifies the contamination on the lens surface by analyzing changes in pixel distribution, and finally outputs an index of the degree of dust accumulation on the lens surface. The gradient calculation module M202, connected to the capture and analysis module M201, is used to calculate the grayscale gradient of the protective cover edge region using the Sobel operator based on the dust accumulation level index. The output (grayscale gradient) of this module is sent to the sealing detection module M203. The sealing performance detection module M203, connected to the gradient calculation module M202, receives the grayscale gradient output by the former. This module's function is to assess the camera's structural protection capability based on the severity of external contamination. It uses a convolutional neural network model to extract features from the received grayscale gradient to determine the sealing integrity score. The risk assessment module M204, connected to the sealing detection module M203, is a condition-triggered depth detection unit. It receives a sealing integrity score and determines whether it falls below a preset safety threshold. Once below the threshold, the module activates, extracting vibration-induced displacement information from the image sequence acquired by the capture and analysis module M201. Based on this information, it calculates the sensor's physical offset amplitude and assesses the potential damage risk level. The status assessment module M205 receives dust accumulation level indicators, sealing integrity scores, and potential damage risk levels from the capture and analysis module M201, the sealing integrity detection module M203, and the risk judgment module M204, respectively. The function of the status assessment module M205 is to integrate the dust accumulation level indicators, the sealing integrity score, and the potential damage risk level, perform weighted calculations or fuzzy logic reasoning, and obtain an overall working status assessment value. The trend analysis module M206, connected to the state assessment module M205, is used to acquire historical image data sequences as a benchmark if the overall working state assessment value exceeds a set normal range. It then compares the current image sequence acquired from the capture analysis module M201 with the benchmark, determining the dynamic trend based on the difference. The output of this module (dynamic trend) is sent to the calibration module M207.

[0063] The calibration module M207, serving as the system's execution output, connects to the status assessment module M205 and ultimately returns control commands to the vehicle-mounted camera. It receives the overall operational status assessment value, performs fault prediction and strategy generation based on this value, and finally calculates a set of fault prevention adjustment parameters. Subsequently, the calibration module M206 performs real-time performance calibration on the vehicle-mounted camera module to ensure it is always in optimal working condition.

[0064] It should be noted that the intelligent detection system for vehicle-mounted camera modules provided in this embodiment of the invention is used to execute all the process steps of the intelligent detection method for vehicle-mounted camera modules described in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

Claims

1. An intelligent detection method for an in-vehicle camera module, characterized in that, Executed by a computer, including: Image sequences are captured by an in-vehicle camera, and image processing algorithms are used to analyze the image sequences to obtain an index of the degree of dust accumulation on the lens surface; Based on the dust accumulation index, the grayscale gradient of the edge region of the protective cover is calculated using the Sobel operator. Based on the grayscale gradient, a convolutional neural network model is used to extract features from the gradient and determine the sealing integrity score. Based on the sealing integrity score, vibration-related displacement information is extracted from the image sequence, and the sensor offset amplitude is calculated using the displacement information to determine the potential damage risk level. Based on the dust accumulation level index, the sealing integrity score, and the potential damage risk level, an image processing algorithm is used to simulate the impact of environmental interference on image quality to obtain an overall working status evaluation value. If the overall working status evaluation value exceeds the set normal range, then historical image data sequence is obtained as a benchmark, the difference between the current image sequence and the benchmark is compared, and the dynamic change trend is determined based on the difference. Based on the dynamic change trend, an image processing algorithm is used to generate a cleanliness restoration simulation path and obtain fault prevention adjustment parameters.

2. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, The process of capturing image sequences using an in-vehicle camera and analyzing these sequences using image processing algorithms to obtain an index of dust accumulation on the lens surface includes: Capture image sequences of the mine environment using vehicle-mounted cameras; Image processing algorithms are used to analyze the pixel distribution changes between consecutive frames in the image sequence to determine the dust accumulation area; The dust coverage ratio is calculated based on the dust accumulation area to obtain the dust accumulation level index on the lens surface.

3. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, The step of extracting features from the gradient using a convolutional neural network model based on the grayscale gradient to determine the sealing integrity score includes: The grayscale gradient is input into a pre-trained convolutional neural network model to generate a gradient feature vector; The gradient feature vector is input into a softmax classifier, which outputs the probability distribution of the category to which the sealing property belongs. The sealing property integrity score is then calculated based on the weighted probability distribution.

4. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, The step of extracting vibration-related displacement information from the image sequence based on the sealing integrity score, calculating the sensor offset amplitude using the displacement information, and determining the potential damage risk level includes: If the sealing integrity score is lower than the preset integrity score threshold, then the vibration-induced displacement vector is extracted from the image sequence; The sensor offset amplitude is calculated using the vibration-induced displacement vector, and the potential damage risk level is determined based on the correlation between the sensor offset amplitude and historical damage data.

5. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, The method involves using an image processing algorithm to simulate the impact of environmental interference on image quality based on the dust accumulation level index, the sealing integrity score, and the potential damage risk level, to obtain an overall working status evaluation value, including: Image processing algorithms are used to simulate the impact of environmental interference on image quality; Based on the impact of environmental interference on image quality, the dust accumulation level index, and the sealing integrity score, the overall working status evaluation value is obtained through fuzzy logic reasoning.

6. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, If the overall working status evaluation value exceeds the set normal range, then historical image data sequences are obtained as a benchmark, and the difference between the current image sequence and the benchmark is compared. Based on the difference, the dynamic change trend is determined, including: If the overall working status evaluation value exceeds the set normal range, then the historical image data sequence is obtained as a benchmark to obtain the benchmark image sequence. A convolutional neural network is used to extract feature vectors from the current image sequence and the obtained reference image sequence, respectively. The differences are quantified by calculating the Euclidean distance between the feature vectors, and the dynamic trend is determined by time series analysis based on the differences.

7. The intelligent detection method for an in-vehicle camera module according to claim 1, characterized in that, The method further includes: The vehicle-mounted camera module is calibrated in real time by using the fault prevention adjustment parameters, integrating the sealing integrity score and the sensor offset amplitude.

8. An intelligent detection system for a vehicle-mounted camera module, characterized in that, include: The capture and analysis module is used to capture image sequences through an in-vehicle camera and analyze the image sequences using image processing algorithms to obtain an index of dust accumulation on the lens surface. The gradient calculation module is used to calculate the gray value gradient of the edge region of the protective cover based on the dust accumulation level index using the Sobel operator. The sealing performance detection module is used to extract features from the gradient using a convolutional neural network model based on the grayscale value gradient, and determine the sealing performance integrity score. The risk assessment module is used to extract vibration-related displacement information from the image sequence based on the sealing integrity score, calculate the sensor offset amplitude through the displacement information, and determine the potential damage risk level. The status assessment module is used to simulate the impact of environmental interference on image quality using image processing algorithms based on the dust accumulation degree index, the sealing integrity score and the potential damage risk level, to obtain an overall working status assessment value. The trend analysis module is used to obtain historical image data sequences as a benchmark if the overall working status evaluation value exceeds the set normal range, compare the current image sequence with the benchmark, and determine the dynamic change trend based on the difference. The calibration module is used to generate a cleanliness restoration simulation path using an image processing algorithm based on the dynamic change trend, and to obtain fault prevention adjustment parameters.

Images