Intelligent inspection method and system based on night vision digital glasses

Abstract

The invention discloses an intelligent inspection method and system based on night vision digital glasses. According to the method, a thermal infrared image, a visible light image, depth information and ambient light intensity data of an inspection area are collected in real time through night vision digital glasses. Feature level fusion is carried out on the infrared image and the visible light image by using an image fusion algorithm, depth information is corrected and optimized, and imaging parameters are automatically adjusted according to ambient light intensity data to ensure that the image is clear. Then, a deep learning algorithm is used to carry out real-time analysis on fused image data, a target is identified, and three-dimensional positioning and distance measurement are realized. And in combination with a preset risk model and a decision rule, the risk is evaluated, an acousto-optic early warning is sent to the inspection personnel through the glasses display module when an abnormality is found, and the risk information is transmitted to a remote monitoring center. In addition, inspection data are automatically recorded and an inspection report is generated. The method and the system effectively improve the inspection efficiency and accuracy, and are especially suitable for intelligent inspection tasks in complex environments.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent inspection, and particularly relates to an intelligent inspection method and system based on night vision digital glasses. BACKGROUND

[0002] In the fields of modern industrial production and security monitoring, the application of intelligent inspection technology is increasingly widespread. Traditional inspection methods mainly rely on periodic manual inspection or simple equipment monitoring, which is difficult to meet the efficient inspection needs in complex environments. In recent years, although the Chinese invention patent with the application number "202510096493.1" and the name "a day and night detection method and system based on laser window transmission" has certain innovativeness in the field of day and night detection, it still has some deficiencies and cannot fully adapt to the diversified needs of the intelligent inspection field.

[0003] Although this method can achieve day and night detection to some extent, it mainly focuses on the propagation of laser beams and the correction of wavefront distortion in the intelligent inspection scene, and lacks the detection and recognition ability of various types of targets (such as personnel, vehicles, equipment fault points, etc.) in the inspection environment, which is difficult to meet the comprehensive inspection needs in complex scenes. Secondly, this scheme does not fully consider the influence of environmental light intensity changes on the detection results, and cannot automatically adjust the imaging parameters according to the environmental light intensity, resulting in unstable detection performance under different lighting conditions (such as night or low light environment). The scheme does not mention the effective recording, storage and analysis of the inspection process data, which is difficult to realize the deep mining and intelligent management of the inspection data, and is not conducive to subsequent risk assessment and decision support. SUMMARY

[0004] The purpose of the present application is to provide an intelligent inspection method and system based on night vision digital glasses, which can solve the technical problems of limited recognition ability and poor adaptability in the prior art.

[0005] In order to achieve the above-mentioned purpose, a specific embodiment of the present application provides an intelligent inspection method based on night vision digital glasses, which comprises the following steps: using night vision digital glasses to collect real-time thermal infrared images, visible light images, depth information and environmental light intensity data of the inspection area; by image fusion algorithm, the infrared image and the visible light image are fused at feature level to generate a fusion image containing rich information, and the depth information is corrected and optimized, and the imaging parameters of the night vision digital glasses are automatically adjusted according to the environmental light intensity data to ensure clear image; using deep learning algorithm to analyze the fused image data in real time, recognizing various targets in the inspection area, and realizing three-dimensional positioning and distance measurement of the targets combined with depth information; According to the preset risk model and decision rule, the detected target and situation are risk evaluated, when potential risks or abnormal situations are found, sound and light early warning is given to the patrol personnel through the display module of the night vision digital glasses, and the risk information is transmitted to the remote monitoring center in real time; All data in the patrol process are automatically recorded, at least including images, videos, target information, risk evaluation results, and are stored in a local or cloud server, and a patrol report is generated.

[0006] In one or more embodiments of the application, the image fusion algorithm fuses the infrared image and the visible light image at the feature level, including the following steps: The infrared image and the visible light image are converted into gray images, the color space is unified, the infrared image and the visible light image are respectively subjected to gray processing, the color information is removed, the brightness information of the images is retained, two gray images are generated, and it is ensured that the subsequent processing steps can focus on the brightness difference and the detail features of the images; Edge features of the grayed infrared image and the visible light image are extracted by applying an edge detection operator, and a suitable edge detection operator is selected to perform convolution operation on the gray image, so as to highlight the outline information of the target in the image; Texture features are extracted by calculating the gradient direction histogram of the image, for each pixel point in the gray image, the gradient direction thereof is calculated, and the distribution of the gradient direction is counted to generate a gradient direction histogram, so as to describe the texture features of the image; The extracted edge features and texture features are subjected to normalization processing, so as to have the same scale range, the numerical range of the edge features and the texture features is normalized to a predetermined interval respectively, the numerical difference between different features is eliminated, the fairness and accuracy of the weighted fusion process are ensured, and preparation is made for feature fusion; The normalized features are weighted fused according to the image quality indicators, the image features with higher quality are highlighted, the weight coefficients of the normalized edge features and the texture features are determined according to the quality indicators such as the definition and contrast of the image, the weight coefficients are subjected to normalization processing, then the normalized edge features and the texture features are respectively subjected to linear weighted fusion, a fused feature image is generated, and the overall quality of the fused image is enhanced.

[0007] In one or more embodiments of the application, the correction and optimization of the depth information include the following steps: The depth map collected by the depth sensor is preprocessed to remove noise and outliers; The depth sensor is calibrated by using a calibration board to determine its intrinsic and extrinsic parameters; The depth information is geometrically corrected according to the binocular parallax principle or the structure light encoding and decoding algorithm; The depth information is matched with the pixels in the color image through the pixel correspondence between the depth information and the color image, so that the pixels in the depth information and the color image are matched, and a unified data basis is provided for subsequent target detection and three-dimensional positioning.

[0008] In one or more embodiments of the present application, the deep learning algorithm is constructed by including the following steps: A large amount of image data under the scene of inspection is collected, covering different environments, light conditions, target types and postures, providing sufficient data support for model training; The image data is labeled, including target category, position information and related attributes, and the labeled data will be used as supervision information for model training to guide the model to learn the feature representation of the target; The training data set is constructed and divided into training set, validation set and test set. The labeled image data is divided into training set, validation set and test set according to the predetermined proportion. The training set is used for optimization of model parameters, the validation set is used for adjustment of hyperparameters and selection of model, and the test set is used for evaluation of final performance of the model to ensure stability and generalization ability of the model on different data sets; The attention mechanism is introduced, and the attention module is added in the middle layer of the convolutional neural network, so that the network can automatically learn and focus on the key area in the image, and the detection ability of the inspection target is improved; The feature extraction network is optimized, and the feature pyramid network structure is adopted to enhance the feature extraction ability of the network for targets of different scales; Adjust the anchor box strategy, customize the size and aspect ratio of the anchor box according to the common size and shape of the inspection target, improve the matching efficiency of the candidate target box and the actual target, and reduce the invalid calculation at the same time; Increase the class balance loss function, design a class balance loss function to solve the problem of uneven distribution of different target categories in the inspection scene, so that the model pays more attention to the samples of minority categories during the training process.

[0009] The network parameters are trained using the stochastic gradient descent or Adam optimization algorithm to minimize the loss function. The specific steps include: initializing the network parameters, setting the learning rate, iteration number and other hyperparameters, calculating the loss function value between the model output and the labeled information through forward propagation, updating the network parameters through backward propagation using the stochastic gradient descent or Adam optimization algorithm, gradually reducing the loss function value, until the model converges, and ensuring that the model can accurately learn the feature representation and position information of the target.

[0010] In one or more embodiments of the present application, the risk model is constructed by the following steps: Organize experts to analyze the inspection task and identify the main risk factors in the inspection area to form a risk factor list; Detailed analysis is conducted on each risk factor to determine its characteristics and association with environmental factors, providing multi-dimensional information for risk assessment; Historical inspection data and accident cases are collected to quantitatively evaluate the occurrence probability, impact range and loss degree of risk factors, providing data support for the risk model; Statistical analysis methods or machine learning algorithms are used to establish the mapping relationship between risk factors and risk levels, realizing quantitative assessment of risks.

[0011] In one or more embodiments of the present application, the implementation of the early warning strategy includes the following steps: Real-time reception of target detection results output by the deep learning algorithm and risk values calculated by the risk assessment model ensures the real-time performance of the early warning system; When the risk value exceeds the preset threshold, the early warning mechanism is triggered, and the corresponding early warning measures are activated; According to the severity of the risk event, the corresponding early warning level and measures are started to ensure the rationality and effectiveness of the early warning response; In the process of sending early warning information, redundant transmission mechanism and error checking and correction algorithm are used to ensure reliable transmission of early warning information; In one or more embodiments of the present application, the automatic recording of inspection data includes the following steps: A data storage module is configured in the night vision digital glasses or edge computing device, and a database management system or file system is used to structure and manage the data, ensuring efficient and easy access to data storage; According to the key information such as inspection task, timestamp, geographic location, the data is classified and stored and indexed, facilitating data query and retrieval; Periodically synchronize the locally stored data to the cloud server, use encryption transmission protocol to prevent data leakage and tampering, and ensure the security and integrity of the data; According to the preset inspection report template, extract key information from the stored data to automatically generate an inspection report, providing a comprehensive report on the inspection work for management personnel.

[0012] In one or more embodiments of the present application, the automatic adjustment of imaging parameters of the night vision digital glasses includes the following steps: Real-time acquisition of ambient light intensity data collected by the ambient light sensor ensures that changes in ambient light intensity can be captured in a timely manner; Filter and calibrate the collected ambient light intensity data to eliminate sensor noise and errors and improve data accuracy; According to the processed ambient light intensity data, combined with the pre-set mapping relationship between imaging parameters and ambient light intensity, the imaging parameters of the infrared sensor and the visible light camera are calculated; The calculated imaging parameter is sent to a driving circuit of the sensor to control the working state of the sensor, so that the dynamic adjustment of the imaging parameter is realized.

[0013] In one or more embodiments of the present application, the three-dimensional positioning and distance measurement of the target comprises the following steps: A three-dimensional space coordinate system is established with the imaging center of the night vision digital glasses as the origin, so as to provide a coordinate framework for subsequent target positioning and distance measurement; According to the corrected depth information of the depth sensor, the depth value of the target pixel point is obtained, and it is converted into the Z coordinate in the three-dimensional space coordinate system, so as to ensure the accuracy and reliability of the depth information; For the target pixel point in the fused image, according to its two-dimensional coordinates in the image and parameters such as the focal length and pixel size of the depth sensor, the X coordinate and Y coordinate of the target pixel point in the three-dimensional space coordinate system are calculated, so as to realize the accurate three-dimensional positioning of the target; According to the three-dimensional coordinates of the target, the straight-line distance between the target and the night vision digital glasses is calculated, and the relative distance between the target and other key points or obstacles in the inspection area is calculated.

[0014] In another aspect of the present application, an intelligent inspection system based on night vision digital glasses is provided, which comprises: The night vision digital glasses integrate infrared sensors, visible light cameras, depth sensors, ambient light sensors, display modules and human-computer interaction modules, and are used for collecting inspection data and performing real-time display and interaction; The data processing server is connected with the night vision digital glasses, and is used for analyzing and processing the collected data; The remote monitoring center is connected with the data processing server, and is used for receiving risk warning information and inspection data in real time, and centrally monitoring and managing the inspection work; The mobile terminal application is connected with the data processing server, and is used for management personnel to view inspection data, receive warning information, and remotely command and dispatch the inspection task.

[0015] Advantages: Compared with the prior art, the present application has significant progress: The present application significantly improves the clarity and information richness of the inspection image by combining environmental light intensity adaptive adjustment of imaging parameters through image fusion algorithm and depth information optimization, especially enhances the imaging quality under complex lighting conditions, and ensures the accuracy and reliability of target detection.

[0016] The real-time analysis of the fused image data using deep learning algorithms can quickly and accurately identify various targets in the inspection area and realize three-dimensional positioning and distance measurement of the targets. This process greatly improves the inspection efficiency, enabling the system to process a large amount of inspection data in a short time and promptly discover potential problems.

[0017] The present application also has real-time risk assessment and early warning functions, which can immediately issue an audible and visual warning when an abnormal situation is detected, and transmit risk information to the remote monitoring center. At the same time, the inspection data is automatically recorded and the inspection report is generated, providing detailed information for subsequent analysis and effectively improving the intelligent level and management efficiency of the inspection work. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0019] Figure 1 Flowchart of the intelligent inspection method based on night vision digital glasses in an embodiment of the present application; Figure 2 Flowchart of the image fusion process in an embodiment of the present application; Figure 3 Flowchart of the correction and optimization process of depth information in an embodiment of the present application; Figure 4 Module schematic diagram of the intelligent inspection system in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the person skilled in the art better understand the technical solutions in the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0021] In view of the above technical problems, as shown in Figure 1 The present application introduces an intelligent inspection method based on night vision digital glasses, which includes the following steps: S101, using night vision digital glasses to collect real-time thermal infrared images, visible light images, depth information and ambient light intensity data of the inspection area; S102, through the image fusion algorithm, the infrared image and the visible light image are fused at feature level to generate a fusion image containing rich information, while the depth information is corrected and optimized, and the imaging parameters of the night vision digital glasses are automatically adjusted according to the ambient light intensity data to ensure clear images; S103, the fused image data is analyzed in real time by using a deep learning algorithm, various targets in the inspection area are recognized, and three-dimensional positioning and distance measurement of the targets are realized in combination with the depth information; S104, according to a preset risk model and decision rule, the detected targets and conditions are risk evaluated, when potential risks or abnormal conditions are found, sound and light warnings are immediately given to the inspection personnel through the display module of the night vision digital glasses, and risk information is transmitted to the remote monitoring center in real time; S105, all data in the inspection process are automatically recorded, at least including images, videos, target information, risk evaluation results, and stored in a local or cloud server, and an inspection report is generated.

[0022] The infrared sensor is used to capture the thermal radiation information of the object, and its wavelength range is usually 7-14 μm; the visible light camera is used to collect the visible light image of the inspection area, with a resolution of 1920x1080 pixels and a frame rate of 30 fps; the depth sensor is based on time of flight (ToF) technology, with a measurement range of 0.5-10 m and a depth accuracy of ±1 cm; the measurement range of the ambient light sensor is 0-100000 lux, with a resolution of 1 lux. When the inspection task is started, the night vision digital glasses collect the thermal infrared image, visible light image, depth information and ambient light intensity data of the inspection area in real time. The collection frequency is 30 times per second to ensure the real-time and continuity of the data.

[0023] In further embodiments, as shown in Figure 2 the image fusion processing includes: gray processing: the collected infrared image and visible light image are respectively subjected to gray processing, a weighted average method is adopted, the R, G, B three channel values of each pixel point in the infrared image are weighted and summed according to weights a, b, c to obtain the gray processed infrared image; the visible light image is subjected to gray processing by the same method. The gray processing formula is: G IR (i, j) = a x R IR (i, j) + b x G IR (i, j) + c x B IR (i, j); G VIS (i, j) = a x R VIS (i, j) + b x G VIS (i, j) + c x B VIS (i, j); G IR(i, j) and G VIS(i, j) represent the gray values of the infrared image and the visible light image at position (i, j) respectively; R IR(i, j), G IR(i, j), B IR(i, j) and R VIS(i, j), G VIS(i, j), B VIS(i, j) represent the R, G, B channel values of the infrared image and the visible light image at position (i, j) respectively; a, b, c are weight coefficients, and a+b+c=1. For example, in actual inspection, by reasonably setting the weights a, b, c, specific features in the infrared image or the visible light image can be highlighted. For example, in device inspection, if the thermal radiation feature of the device is more critical, the value of a can be appropriately increased to enhance the performance of the thermal radiation information of the infrared image in the fused image.

[0024] Edge feature extraction: the Sobel edge detection operator is applied to the grayed image to extract edge features. The gradient amplitude calculation formula of the Sobel edge detection is: G(i, j) = sqrt([Gx(i, j)]² + [Gy(i, j)]²) wherein Gx(i, j) and Gy(i, j) are the gradient values of the image in the horizontal direction and the vertical direction at position (i, j), and the calculation formula is: Gx(i, j) = (Img(i-1, j-1) + 2×Img(i-1, j) + Img(i-1, j+1)) - (Img(i+1, j-1) + 2×Img(i+1, j) + Img(i+1, j+1)); Gy(i, j) = (Img(i-1, j-1) + 2×Img(i, j-1) + Img(i+1, j-1)) - (Img(i-1, j+1) + 2×Img(i, j+1) + Img(i+1, j+1)); wherein Img(i, j) represents the gray value of the image at position (i, j). The edge feature can highlight the contour information of the target in the image, which is helpful for subsequent target detection and recognition. For example, when detecting the edge contour of the device, the Sobel operator can effectively extract the boundary of the device, making it easier to be recognized in a complex inspection environment.

[0025] Texture feature extraction: Calculate the gradient direction histogram of the image to extract texture features. The gradient direction calculation formula is: θ(i, j) = arctan(Gy(i, j) / Gx(i, j)); Quantize the gradient direction into multiple intervals, and count the number of pixels in each interval to generate a gradient direction histogram. Texture features reflect the distribution pattern and repetition frequency of pixel gray levels in the image, providing more detailed information for image fusion. For example, when detecting the texture features of the device surface, the texture changes of the device surface can be analyzed through the gradient direction histogram to determine whether there are abnormal conditions such as wear and tear, cracks, etc.

[0026] Feature normalization: Normalize the extracted edge features and texture features, with the normalization formula being: F_n = (F - F_min) / (F_max - F_min); where F is the original feature value, F_min and F_max are the minimum and maximum values of the feature value, respectively. Normalization eliminates numerical differences between different features, allowing features to have the same scale range and providing a basis for subsequent weighted fusion. For example, the numerical range of edge features and texture features may differ greatly, and after normalization, they can be unified into the interval [0, 1], allowing each feature to participate fairly in the weighted fusion process and improving the quality of the fused image.

[0027] Feature weighted fusion: According to the quality indicators of the image, the edge feature weight is determined to be m and the texture feature weight is determined to be n. Linearly weighted fusion of the normalized edge features and texture features generates a fused feature image. The weighted fusion formula is: F_fusion(i, j) = m × E_fn(i, j) + n × T_fn(i, j); where m + n = 1, E_fn(i, j) and T_fn(i, j) are the values of the normalized edge features and texture features at position (i, j), respectively. Through weighted fusion, edge features and texture features are organically combined to generate a fused image containing rich information. For example, when inspecting an area with a high concentration of devices, increasing the value of m appropriately can highlight the edge contours of the devices, helping the inspector quickly identify the boundaries and shapes of the devices; when detecting the fine texture of the device surface, the value of n can be appropriately increased to enhance the ability to capture the details of the device surface.

[0028] In further embodiments, the depth information correction and optimization includes: Depth map preprocessing: The depth map collected by the depth sensor is preprocessed, and a median filtering algorithm is used to remove noise and outliers. The filtered depth value calculation formula is: D_f(i, j) = Median(D(i-p:i+p, j-p:j+p)); where D(i-p:i+p, j-p:j+p) represents the depth value in the k×k window centered at position (i, j), p=(k-1) / 2, and Median represents the median operation. For example, in a factory inspection scene, the depth map collected by the depth sensor may be disturbed by environmental factors, resulting in noise and outliers. By using the median filtering algorithm, these noise and outliers can be effectively removed, making the depth map smoother and more accurate, and providing a more reliable data basis for subsequent depth information correction and optimization.

[0029] Depth sensor calibration: The depth sensor is calibrated using a calibration board to determine its intrinsic and extrinsic parameters. During the calibration process, the focal length, principal point coordinates, distortion coefficients, and other parameters of the camera are calculated based on the pixel coordinates and world coordinates of the checkerboard corner points. The calibration formula follows the camera imaging model: s×[u; v; 1]=A×[R|t]×[X; Y; Z; 1]; where s is the scaling factor, [u; v] is the image coordinate, A is the camera intrinsic matrix, R and t are the rotation matrix and translation vector, respectively, and [X; Y; Z] is the world coordinate. Through calibration, the accurate parameters of the depth sensor can be obtained, providing a basis for geometric correction of depth information. For example, during the calibration process, by shooting images of the calibration board at different angles and distances, the focal length, principal point coordinates, and other parameters of the depth sensor can be accurately calculated, thereby improving the accuracy of depth information.

[0030] Depth geometric correction: According to the binocular disparity principle, the depth information is geometrically corrected. The depth value calculation formula is: Z'=(B×f) / d; where Z' is the depth value, B is the baseline distance, f is the focal length, and d is the disparity. Geometric correction can correct the geometric distortion in depth information, making it more accurate in terms of geometric relationship with the actual scene. For example, in a binocular stereo vision system, the depth information of an object can be obtained by calculating the disparity between the left and right cameras. Using the binocular disparity principle to geometrically correct the depth information can improve the accuracy of depth measurement, making the depth information more accurately reflect the actual distance of the object.

[0031] Depth and color image registration: Registration is performed through pixel correspondence between depth information and color images. Based on the feature point matching method, feature points in depth images and color images are extracted, and the RANSAC algorithm is used to estimate the homography matrix H of the two. The registration formula is: [u_d; v_d; 1] = H x [u_c; v_c; 1]; where (u_c, v_c) is the coordinate of the pixel point in the color image, and (u_d, v_d) is the coordinate of the corresponding pixel point in the depth image. The purpose of registration is to make the pixel points in the depth information and the color image one-to-one correspondence, providing a unified data basis for subsequent target detection and three-dimensional positioning. For example, when inspecting a device, the position of the device in the color image can be associated with the corresponding depth information through registration, thereby realizing accurate measurement and analysis of the device.

[0032] In further embodiments, as shown in FIG. 8, the steps of deep learning target detection include: Figure 3 Data collection and labeling: Collect a large amount of image data under inspection scenes, covering different environments, lighting conditions, target types and poses, a total of N image data. Label the image data, label the target class, position information and related attributes. Dataset construction and division: Construct the training dataset, divide the labeled image data into training set, validation set and test set according to the proportion. The data augmentation formula is: I_aug = Aug(I_org, params); where Aug represents data augmentation operations such as random cropping, rotation, flipping, adjusting brightness and contrast, etc., and params is the parameter of the enhancement operation. Data augmentation can expand the size of the training dataset, improve the generalization ability and robustness of the model.

[0033] Model construction and training: Construct a deep learning network model, based on the YOLOv5 network framework, add an attention module in the middle layer of the convolutional neural network, optimize the feature extraction network, use the feature pyramid network structure, and adjust the anchor box strategy. The loss function is: L = λ1 x Lc + λ2 x L_coord + λ3 x L_conf;

[0034] where Lc is the classification loss, L_coord is the coordinate regression loss, L_conf is the confidence loss, and λ1, λ2, λ3 are the corresponding loss weight coefficients. Through training, the model can learn the feature representation and position information of the target, improving the accuracy and recall rate of target detection.

[0035] ​Target detection: input the fused image data into the trained model for target detection, output the target category, confidence and position information. The coordinates of the detection box are: x=x_c-w / 2, y=y_c-h / 2, where x_c, y_c are the center coordinates of the detection box, w, h are the width and height of the detection box. Target detection can identify various targets in the inspection area in real time, and provide basis for subsequent three-dimensional positioning and risk assessment.

[0036] Three-dimensional positioning: establish a three-dimensional coordinate system with the imaging center of the night vision digital glasses as the origin. According to the corrected depth information of the depth sensor, obtain the depth value Z of the target pixel point, and according to the two-dimensional coordinates (x, y) of the target pixel point in the image and the focal length f, pixel size s and other parameters of the depth sensor, calculate the X and Y coordinates of the target pixel point in the three-dimensional coordinate system, realize the accurate three-dimensional positioning of the target. The calculation formula is: X=(x-cx)×Z / fx, Y=(y-cy)×Z / fy; Where (cx, cy) is the principal point coordinate of the depth camera, fx and fy are the focal lengths of the depth camera in x and y directions respectively. Three-dimensional positioning can provide accurate position information of the target, and provide more comprehensive data support for subsequent risk assessment and decision support.

[0037] In further embodiments, the construction of the risk model is achieved by the following steps: Risk factor analysis and quantification: experts organize the analysis of the inspection task, identify the main risk factors in the inspection area, and form a risk factor list. Detailed analysis is made on each risk factor to determine its characteristics and association with environmental factors. Historical inspection data and accident cases are collected to quantitatively evaluate the occurrence probability, influence range and loss degree of risk factors. The risk value calculation formula is: Rv=p×i×l; Where p is the occurrence probability of the risk factor, i is the influence range, and l is the loss degree.

[0038] Risk model construction: a machine learning algorithm is used to establish the mapping relationship between risk factors and risk levels. The risk level division formula is: Rl=1, Rv<T1; Rl=2, T1≤Rv<T2; Rl=3, Rv≥T2; where T1 and T2 are the threshold values for risk level division. Through machine learning algorithm, the mapping relationship between risk factors and risk levels can be automatically learned, improving the accuracy and objectivity of risk assessment.

[0039] Early warning trigger and measures: Real-time receive the target detection results output by the deep learning algorithm and the risk values calculated by the risk assessment model. When the risk value exceeds the preset threshold, trigger the early warning mechanism and activate the corresponding early warning measures. The early warning level determination formula is: Wl = min(3, floor(Rv / Wt) + 1); where Wt is the early warning threshold, and floor represents the down rounding operation. According to the early warning level, take corresponding measures such as sound and light alarm, push notification, etc., to timely remind the inspection personnel to pay attention to the potential risks.

[0040] In further embodiments, the implementation of the early warning strategy includes the following steps: when the risk value exceeds the preset threshold, encode the early warning information. The encoding content includes key information such as risk position, risk level, risk type, etc. The encoding formula is: W_code = E(R_pos, R_level, R_type); where E is the encoding function, R_pos is the risk position, R_level is the risk level, and R_type is the risk type. For example, the risk position can be represented by latitude and longitude coordinates or grid numbers of the inspection area; the risk level is divided into low, medium and high levels; the risk type includes equipment failure, personnel violation, fire hazard, etc. Encode these information to generate a compact early warning code for quick transmission and analysis.

[0041] Redundant transmission mechanism is adopted to send early warning information through Wi-Fi and 4G networks at the same time, ensuring reliable transmission of early warning information. The transmission formula is: W_sent = Send(W_code, [Wi-Fi, 4G]), where Send represents the sending function, W_code is the encoded early warning information, and [Wi-Fi, 4G] is the transmission network list. Redundant transmission can improve the transmission success rate of early warning information and avoid the loss of early warning information due to network failure.

[0042] During the transmission of early warning information, error checking and correction algorithm such as CRC check is adopted to detect and correct errors in the transmitted data. The check formula is: CRC_result = CRC(W_code); Where CRC represents the cyclic redundancy check algorithm, and W_code is the encoded early warning information. If CRC_result indicates that the data has errors, resend the early warning information until the data is correctly transmitted. Error checking and correction algorithm can effectively improve the integrity and accuracy of early warning information, ensuring that the receiver can correctly parse the early warning information and take corresponding measures.

[0043] After receiving the warning message, the receiver decodes and processes it and provides feedback on the reception status to the sender. The feedback formula is: Feedback = Decode(W_code) + Status; where Decode represents the decoding function and Status represents the reception status (success or failure). Based on the feedback, the sender confirms whether the warning message was successfully delivered. If reception fails, timely remedial measures are taken, such as increasing transmission power or switching transmission networks, to ensure that the warning message is ultimately correctly received and processed by the receiver. For example, after receiving the warning message, the remote monitoring center immediately decodes it to obtain information such as the risk location, level, and type, and displays the corresponding alarm prompt on the monitoring interface. At the same time, a feedback signal is sent to the night vision digital glasses to confirm that the warning message has been successfully received. After receiving the feedback, the night vision digital glasses record the sending and receiving status of the warning message, providing data support for subsequent analysis and statistics.

[0044] In a further embodiment, automatically recording inspection data includes the following steps: Data storage structure design: A data storage module is configured in the night vision digital glasses or edge computing device, and a database management system is used to structure and manage the data. The database table structure design includes an inspection data table with fields such as images, videos, target information, risk assessment results, timestamps, and geographic locations.

[0045] Data classification storage and indexing: Data is classified, stored, and indexed according to key information such as inspection task number, timestamp, and geographic location. The index establishment formula is: Idx=Hash(Tid+Ts+Loc); Where Hash is the hash function, Tid is the inspection task number, Ts is the timestamp, and Loc is the geographic location. Through categorized storage and indexing, data for specific inspection tasks can be quickly queried and retrieved, improving data availability and management efficiency. For example, when querying data for a specific inspection task, the corresponding data record can be quickly located by the inspection task number, and further filtered and sorted based on the timestamp and geographic location, making it easier for users to view and analyze the inspection data.

[0046] Data synchronization and encrypted transmission: Regularly synchronize locally stored data to the cloud server, using encrypted transmission protocols to prevent data leakage and tampering, ensuring data security and integrity. The synchronization formula is: Sync_Result = Sync(Local_Data, Cloud_Server, K); where Sync represents the data synchronization function, Local_Data is the locally stored data, Cloud_Server is the cloud server, and K is the encryption key. The encryption transmission protocol can use AES-256 or other advanced encryption standards to ensure the confidentiality and integrity of the data during transmission. For example, every 30 minutes, the night vision digital glasses will encrypt the locally stored inspection data and synchronize it to the cloud server. After receiving the data, the cloud server will decrypt and store it, and return the synchronization result to the night vision digital glasses to confirm whether the data synchronization is successful.

[0047] Inspection report generation: According to the preset inspection report template, extract key information from the stored data to automatically generate an inspection report. The report generation formula is: Rpt = Temp + D_ext; where Temp is the inspection report template, and D_ext is the key data extracted from the database. The inspection report can include basic information of the inspection task, list of detected targets, risk assessment results, and description of abnormal situations, etc., providing a comprehensive report of the inspection work to the management personnel. For example, the inspection report template can predefine the format and content structure of the report, including cover, table of contents, main text, etc. When generating the report, fill in the content in the template according to the actual inspection data, and generate a complete inspection report for the management personnel to review and archive.

[0048] Automatic adjustment of imaging parameters of night vision digital glasses includes the following steps: Environment light intensity data processing: Real-time acquisition of environment light intensity data collected by environment light sensor, filtering and calibration processing. The filtering formula is: Lf = α × Lc + (1 - α) × Lp; where Lf is the filtered environment light intensity value, Lc is the current collected environment light intensity value, Lp is the previous filtered environment light intensity value, and α is the filtering coefficient.

[0049] Imaging parameter calculation: According to the processed environment light intensity data, combined with the pre-set mapping relationship between imaging parameters and environment light intensity, calculate the imaging parameters of infrared sensor and visible light camera. The imaging parameter calculation formula is: P_IR = P_IR_base + k1 × (Lf - L_ref); P_VIS = P_VIS_base + k2 × (Lf - L_ref); where P_IR and P_VIS are the imaging parameters of infrared sensor and visible light camera respectively, P_IR_base and P_VIS_base are the basic values of imaging parameters, k1 and k2 are the mapping coefficients, and L_ref is the reference environment light intensity value.

[0050] Dynamic adjustment of imaging parameters: the calculated imaging parameters are sent to the driving circuit of the sensor to control the working state of the sensor, realizing dynamic adjustment of the imaging parameters. The parameter adjustment formula is: S_p=S_p_prev+ΔP; where S_p is the adjusted sensor imaging parameter, S_p_prev is the sensor imaging parameter before adjustment, and ΔP is the imaging parameter adjustment amount. By dynamically adjusting the imaging parameters, the night vision digital glasses can automatically optimize the image quality under different lighting conditions, ensuring that the inspection personnel can obtain clear and accurate image information. For example, during the inspection process from indoors to outdoors, the ambient light intensity changes greatly. The night vision digital glasses automatically adjust the imaging parameters of the infrared sensor and visible light camera according to the data of the ambient light intensity sensor, so that the image can quickly adapt to the new lighting conditions, avoiding overexposure or underexposure, and improving the inspection efficiency and accuracy.

[0051] The three-dimensional positioning and distance measurement of the target includes the following steps: Three-dimensional coordinate system establishment: taking the imaging center of the night vision digital glasses as the origin, a three-dimensional coordinate system is established. The x-axis of the coordinate system is horizontal to the right, the y-axis is vertical upward, and the z-axis is perpendicular to the image plane and points to the distance, conforming to the right-hand coordinate system rule. For example, during the inspection process, the imaging center of the night vision digital glasses as the coordinate origin can conveniently calculate the position and distance of the target in space.

[0052] Depth information acquisition: according to the corrected depth information of the depth sensor, the depth value Z of the target pixel point is obtained. The depth value is directly measured by the depth sensor and optimized by the correction algorithm to ensure its accuracy and reliability. For example, the depth information after geometric correction and filtering processing can more truly reflect the actual distance between the target and the night vision digital glasses.

[0053] Three-dimensional coordinate calculation: for the target pixel point in the fusion image, according to its two-dimensional coordinates (x, y) in the image and parameters such as the focal length f of the depth sensor and the pixel size s, the X coordinate and Y coordinate of the target pixel point in the three-dimensional space coordinate system are calculated, realizing the accurate three-dimensional positioning of the target. X=(x-cx)×Z / fx; Y=(y-cy)×Z / fy; where (cx, cy) is the principal point coordinate of the depth camera, and fx and fy are the focal lengths of the depth camera in x and y directions, respectively. For example, the coordinates of a device target pixel point in the image are (x, y), combined with the depth value Z and camera parameters, the coordinates (X, Y, Z) of the device in the three-dimensional space coordinate system are calculated, so as to determine its accurate position.

[0054] Distance measurement: according to the three-dimensional coordinates of the target, the straight-line distance D between the target and the night vision digital glasses, and the relative distance between the target and other key points or obstacles in the inspection area are calculated. The distance includes: D = sqrt(X 2 + Y 2 + Z 2 ) ; D _ relative = sqrt((X-X_obs) 2 + (Y-Y_obs) 2 + (Z-Z_obs) 2 ) ; wherein (X_obs, Y_obs, Z_obs) are the three-dimensional coordinates of the key points or obstacles. For example, the straight-line distance D between the device and the night vision digital glasses is calculated, as well as the relative distance D _ relative between the device and other key devices or obstacles, to provide comprehensive spatial location information for the inspection personnel to obtain a more optimal planned inspection route and perform risk assessment.

[0055] As shown in Figure 4 The present application also provides an intelligent inspection system based on night vision digital glasses, comprising night vision digital glasses, which integrate infrared sensors, visible light cameras, depth sensors, ambient light sensors, display modules and human-computer interaction modules, for collecting inspection data and performing real-time display and interaction; a data processing server connected to the night vision digital glasses, for analyzing and processing the collected data; a remote monitoring center connected to the data processing server, for receiving risk warning information and inspection data in real time, and centrally monitoring and managing the inspection work; and a mobile terminal application connected to the data processing server, for management personnel to view inspection data, receive warning information, and remotely command and dispatch inspection tasks.

[0056] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. An intelligent inspection method based on night vision digital glasses, characterized in that: The method comprises the following steps: Use night vision digital glasses to collect thermal infrared images, visible light images, depth information and ambient light intensity data of the inspection area in real time; Through the image fusion algorithm, the infrared image and the visible light image are fused at the feature level to generate a fused image containing rich information. At the same time, the depth information is corrected and optimized, and the imaging parameters of the night vision digital glasses are automatically adjusted according to the ambient light intensity data to ensure clear images. Use deep learning algorithms to analyze the fused image data in real time, identify various targets within the inspection area, and combine depth information to achieve three-dimensional positioning and distance measurement of the targets; Based on the preset risk model and decision-making rules, the detected targets and situations are assessed for risk. When potential risks or abnormal situations are found, an audible and visual warning is immediately issued to the inspectors through the display module of the night vision digital glasses, and the risk information is transmitted to the remote monitoring center in real time. Automatically record all data during the inspection process, including at least images, videos, target information, and risk assessment results, and store them in a local or cloud server, while generating an inspection report.

2. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The image fusion algorithm performs feature-level fusion of infrared images and visible light images, including the following steps: Convert infrared and visible light images into grayscale images, unify the color space, and grayscale the infrared and visible light images separately to remove color information while retaining the brightness information of the images. This generates two grayscale images, ensuring that subsequent processing steps can focus on the brightness differences and detailed features of the images. Apply edge detection operators to extract edge features of grayscale infrared images and visible light images, and select appropriate edge detection operators to perform convolution operations on grayscale images to highlight the contour information of the target in the image; Calculate the gradient direction histogram of the image to extract texture features. For each pixel in the grayscale image, calculate its gradient direction and statistically analyze the distribution of the gradient direction to generate a gradient direction histogram to describe the texture features of the image. Normalize the extracted edge features and texture features to make them have the same scale range, normalize the value range of edge features and texture features to the predetermined interval respectively, eliminate the numerical differences between different features, ensure the fairness and accuracy of the weighted fusion process, and prepare for feature fusion; The normalized features are weightedly fused according to the image quality index to highlight the image features with higher quality. The weight coefficients of the normalized edge features and texture features are determined based on the quality indicators such as image clarity and contrast. The weight coefficients are normalized and then the normalized edge features and texture features are linearly weighted fused to generate a fused feature image and enhance the overall quality of the fused image.

3. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The correction and optimization of the depth information comprises the following steps: Preprocess the depth map collected by the depth sensor to remove noise and outliers; Use the calibration board to calibrate the depth sensor and determine its intrinsic and extrinsic parameters; Perform geometric correction on depth information based on binocular parallax principle or structured light encoding and decoding algorithm; The depth information is aligned with the pixel correspondence between the depth information and the color image, so that the depth information corresponds to the pixel points in the color image, providing a unified data basis for subsequent target detection and three-dimensional positioning.

4. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The deep learning algorithm is constructed by including the following steps: Collect a large amount of image data from inspection scenarios, covering different environments, lighting conditions, target types, and postures, to provide sufficient data support for model training; Label the image data, including target category, location information, and related attributes. The labeled data will serve as supervisory information for model training, guiding the model to learn the feature representation of the target; Construct a training dataset and divide it into training set, validation set and test set. Divide the labeled image data into training set, validation set and test set according to the predetermined ratio. The training set is used to optimize the model parameters, the validation set is used to adjust the hyperparameters and select the model, and the test set is used to evaluate the final performance of the model. The attention mechanism is introduced, and an attention module is added to the middle layer of the convolutional neural network, enabling the network to automatically learn and focus on key areas in the image, improving the detection ability of inspection targets. The feature extraction network is optimized and a feature pyramid network structure is adopted to enhance the network's feature extraction ability for targets of different scales. Adjust the anchor frame strategy and customize the size and aspect ratio of the anchor frame based on the common sizes and shapes of inspection targets to improve the matching efficiency between candidate target frames and actual targets while reducing invalid calculations; Use stochastic gradient descent or Adam optimization algorithm to train network parameters and minimize the loss function; initialize network parameters, set hyperparameters such as learning rate and number of iterations, calculate the loss function value between the model output and the annotation information through forward propagation, and use stochastic gradient descent or Adam optimization algorithm back propagation to update the network parameters, gradually reducing the loss function value until the model converges, ensuring that the model can accurately learn the feature representation and position information of the target.

5. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The risk model is constructed by the following steps: Organize experts to analyze inspection tasks, identify major risk factors within the inspection area, and compile a risk factor list; Conduct detailed analysis of each risk factor to determine its characteristics and its relationship with environmental factors, providing multi-dimensional information for risk assessment; Collect historical inspection data and accident cases to quantitatively assess the probability of occurrence, impact scope, and loss extent of risk factors, providing data support for risk models; Statistical analysis methods or machine learning algorithms are used to establish a mapping relationship between risk factors and risk levels to achieve quantitative risk assessment.

6. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The implementation of the early warning strategy includes the following steps: Receive target detection results output by deep learning algorithms and risk values ​​calculated by risk assessment models in real time to ensure the real-time nature of the early warning system; When the risk value exceeds the preset threshold, the early warning mechanism is triggered and the corresponding early warning measures are activated; According to the severity of the risk event, initiate the corresponding warning level and measures to ensure the rationality and effectiveness of the warning response; During the early warning information transmission process, redundant transmission mechanism and error checking and correction algorithm are adopted to ensure the reliable transmission of early warning information.

7. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The automatic recording of inspection data comprises the following steps: Configure a data storage module in night vision digital glasses or edge computing devices, and use a database management system or file system to structure and manage data to ensure efficient and accessible data storage; Categorize, store, and index data based on key information such as inspection tasks, timestamps, and geographic locations to facilitate data query and retrieval; Regularly synchronize locally stored data to cloud servers, using encrypted transmission protocols to prevent data leakage and tampering, ensuring data security and integrity; Based on the preset inspection report template, key information is extracted from the stored data and an inspection report is automatically generated to provide management personnel with a comprehensive inspection work report.

8. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The automatic adjustment of imaging parameters of the night vision digital glasses comprises the following steps: Acquire ambient light intensity data collected by the ambient light sensor in real time to ensure that changes in ambient light intensity can be captured in a timely manner; Filter and calibrate the collected ambient light intensity data to eliminate sensor noise and errors and improve data accuracy; Based on the processed ambient light intensity data, combined with the pre-set mapping relationship between imaging parameters and ambient light intensity, the imaging parameters of the infrared sensor and visible light camera are calculated; The calculated imaging parameters are sent to the driving circuit of the sensor to control the working state of the sensor and realize dynamic adjustment of the imaging parameters.

9. The intelligent inspection method based on night vision digital glasses according to claim 1, characterized in that: The three-dimensional positioning and distance measurement of the target includes the following steps: With the imaging center of the night vision digital glasses as the origin, a three-dimensional space coordinate system is established to provide a coordinate framework for subsequent target positioning and distance measurement; According to the depth information corrected by the depth sensor, the depth value of the target pixel is obtained and converted into the Z coordinate in the three-dimensional space coordinate system to ensure the accuracy and reliability of the depth information; For the target pixel in the fused image, the X and Y coordinates of the target pixel in the three-dimensional space coordinate system are calculated based on its two-dimensional coordinates in the image and parameters such as the focal length and pixel size of the depth sensor, thereby achieving accurate three-dimensional positioning of the target; According to the three-dimensional coordinates of the target, the straight-line distance between the target and the night vision digital glasses, as well as the relative distance between the target and other key points or obstacles in the inspection area are calculated.

10. An intelligent inspection system applied to the intelligent inspection method based on night vision digital glasses according to any one of claims 1 to 9, characterized in that: The intelligent inspection system includes: Night vision digital glasses, which integrate infrared sensors, visible light cameras, depth sensors, ambient light sensors, display modules and human-computer interaction modules to collect inspection data and display and interact with it in real time; A data processing server, connected to the night vision digital glasses, for analyzing and processing the collected data; A remote monitoring center, which is connected to the data processing server and is used to receive risk warning information and inspection data in real time, and to centrally monitor and manage inspection work; A mobile terminal application is connected to a data processing server for management personnel to view inspection data, receive early warning information, and remotely command and dispatch inspection tasks.

Images