A special equipment image inspection intelligent analysis system and method

Abstract

The present invention relates to the technical field of special equipment inspection image analysis, and discloses a special equipment image inspection intelligent analysis system and method, comprising an image acquisition module for real-time acquisition of regional special equipment images, a processing module for processing the acquired regional special equipment images, an identification and analysis module for identifying the type and operation status of special equipment and analyzing the processed special equipment images, a pruning unit for inspecting the identified special equipment type and optimizing the special equipment inspection model, an inspection module for evaluating the special equipment image inspection results, a feedback module for feeding back the special equipment inspection results and image analysis results, a report generation module for generating and inspecting special equipment inspection reports, and an interactive module for displaying the special equipment inspection results and storing the inspection results, thereby realizing intelligent inspection of special equipment.

Description

Technical Field

[0001] The invention relates to the technical field of special equipment inspection image analysis, and discloses a special equipment image inspection intelligent analysis system and method. Background Art

[0002] Special equipment inspection image technology has made significant progress, but it still faces many challenges and shortcomings in practical applications:

[0003] Under complex lighting and environmental conditions, the accuracy of image recognition algorithms may decrease. Secondly, most of the current image recognition algorithms are trained based on specific scenarios and data sets, and may lack universality for special equipment of different models and brands. More universal research and adaptation are needed. High-precision image inspection systems usually require expensive hardware equipment and complex software algorithms, which increase the deployment and maintenance costs of the system, especially for small special equipment facilities. It may not be cost-effective. Although image technology can help detect the failure of special equipment, the diagnosis and prediction of complex faults still need to be further improved in accuracy and precision. Future development requires more research and technological innovation to improve the reliability, practicality and user acceptance of the system. The "Elevator Self-Inspection Rules" (TSG T7008-2023 hereinafter referred to as the new inspection rules) require audio and video collection for relevant inspection items. The collected audio and video images can be analyzed intelligently to assist in the issuance of inspection reports. Taking the supervision inspection of passenger and cargo special equipment as an example, the audio and video records involve speed limiter safety clamps, buffers, traction capacity tests, braking performance tests, etc. Therefore, there is an urgent need for a new special equipment inspection system to meet the requirements of the new inspection rules.

[0004] For example, a Chinese patent application with publication number CN118047278A discloses a real-time detection method and system for special equipment door faults based on machine vision. The method collects image information of the current special equipment door at a first preset frequency through machine vision to generate a set of images to be tested. The real-time opening and closing distance of the special equipment door is obtained after analyzing and identifying the image information through a neural network model. The difference value of the real-time opening and closing distance in the real-time opening and closing data of two adjacent special equipment doors is calculated. By obtaining multiple image information when the special equipment door is opened and closed, and obtaining the opening and closing distance of the special equipment door, the calculation of the opening and closing difference realizes the dynamic tracking of the motion state of the special equipment door, obtains the control timing of the special equipment door and matches the actual motion state of the special equipment door with the motion state in the control timing. The abnormal state category of the current special equipment door can be effectively identified, which is convenient for users to check and make inspection or maintenance strategies in time, while reducing the system computing power and making full use of the control timing of the special equipment door to improve efficiency.

[0005] The above patent mainly determines the failure of special equipment by identifying the real-time opening and closing distance of the special equipment door, but cannot inspect and analyze the regional special equipment;

[0006] The above patent is to monitor special equipment in real time. The monitoring image algorithm mainly monitors and analyzes the opening and closing of special equipment doors in real time through time series data sets. Therefore, a large number of storage structures and computing units are required to process the time series data sets, which greatly increases the cost of subsequent software system maintenance and the cost of hardware equipment requirements;

[0007] The above patent mainly monitors faults during the operation of special equipment and does not involve all-round automatic identification and inspection of special equipment. Summary of the invention

[0008] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0009] In order to solve the above technical problems, the main purpose of the present invention is to provide a special equipment image inspection intelligent analysis system, comprising:

[0010] Image acquisition module, used to acquire the whole special equipment image in real time;

[0011] The processing module includes a preprocessing unit and an image region segmentation unit, wherein the preprocessing unit is used to process the collected regional special equipment image, and the image region segmentation unit is used to segment the preprocessed regional special equipment image into a plurality of regions;

[0012] The identification and analysis module includes an identification unit for identifying the type and operating status of special equipment, an analysis unit for analyzing processed images of special equipment in different areas, and a calling unit for calling special equipment inspection criteria for the identified special equipment type;

[0013] An inspection module, including a special equipment inspection model for inspecting the identified special equipment type, a pruning unit for tuning the special equipment inspection model, and an evaluation unit for evaluating special equipment image inspection results;

[0014] Feedback module, used to feedback special equipment inspection results and image analysis results;

[0015] Report generation module, used to generate and verify special equipment inspection reports;

[0016] Interactive module, used to display special equipment inspection results and store inspection results.

[0017] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0018] Collecting an overall special equipment image by an image collector, wherein the overall special equipment image includes images of all structures of the special equipment;

[0019] Process the collected overall special equipment image, remove noise by filtering, and enhance the image clarity of the filtered overall special equipment image by color conversion, and then enhance the converted new image by edge inspection and grayscale processing;

[0020] The image region segmentation unit includes extracting the feature map F of the image, generating candidate regions, and the candidate regions include the regions that need to be identified and the regions that do not need to be identified for the special equipment, and then mapping each candidate region to a feature map of a fixed size through the RoI Align layer. The size is K×K, and then each candidate area after mapping is classified, and the category probability P and bounding box regression value △ are output. The pixel-level mask of each candidate area is generated through the mask branch to determine whether the pixels in the candidate area belong to the special equipment to be detected target, and the probability that the pixels in the candidate area belong to the special equipment to be detected target is output;

[0021] The total loss of region segmentation is calculated through classification loss, bounding box regression loss and mask loss, and the total loss is optimized through the optimizer to finally complete the regional segmentation of the overall image of special equipment.

[0022] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0023] The filtering reduces the noise effect in the image by weighted averaging the pixels of the regional special equipment image. The calculation expression of the filtering is as follows:

[0024] Γ ( x , y ) = ∑ i = 1 n [ f ( x i , y i ) × e x i 2 + y i 2 2 × γ γ × 2 ]

[0025] in, is the pixel of the regional special equipment image after filtering, is the filter standard deviation coefficient, is the original value of the regional special equipment image pixel, is the x-coordinate of the ith pixel value, is the y coordinate of the i-th pixel value, i is an ordinal number of 1, 2, 3, ..., n, n is an arbitrary integer, and e is an exponential constant;

[0026] Color conversion converts the color of the filtered regional special equipment image into a more easily recognizable color space by converting the color image into a grayscale image;

[0027] The grayscale image is used to process edge and texture information in the regional special equipment image.

[0028] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0029] The grayscale processing performs approximate calculation of the longitudinal gradient and transverse gradient of the pixels of the grayscale image through the Sobel operator. The longitudinal gradient calculation expression is as follows:

[0030] G Y = [ α 1 , α 2 ,... α n − 1 ] T × A

[0031] in, is the coefficient matrix for calculating the longitudinal gradient, and A is the grayscale image;

[0032] The lateral gradient calculation expression is as follows:

[0033] G X = [ β 1 , β 2 ,... β n − 1 ] T × A

[0034] in, Calculate the coefficient matrix for the lateral gradient;

[0035] The grayscale processing calculation expression is as follows:

[0036]

[0037] Among them, G is the regional special equipment image after grayscale processing;

[0038] The analysis unit receives the special equipment safety inspection rules and technical review, performs preliminary analysis on the processed regional special equipment image through the special equipment safety inspection rules and technical review of the special equipment classification, and extracts the feature value of the image;

[0039] The feature value extraction of the image is preliminarily performed according to the special equipment safety inspection rules corresponding to the special equipment to be inspected.

[0040] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0041] The special equipment inspection model includes a convolution layer, an input layer, an image data processing layer and an output layer;

[0042] The convolution process is completed through the convolution layer, and the input image size, step size and padding are set. The x-axis calculation expression of the output feature map of the convolution process is as follows:

[0043]

[0044] in, is the x-axis size of the output feature map, is the x-axis length of the original input image, is the preset size of the convolution kernel, L is the step size, and F is the padding value;

[0045] The y-axis calculation expression of the output feature map of the convolution process is as follows:

[0046]

[0047] in, is the y-axis size of the output feature map, The y-axis length of the original input image;

[0048] The feature map after convolution processing is input to the image data processing layer through the input layer, and the feature value of the feature map is further extracted through the image data processing layer, and finally the data of the regional special equipment image is output through the output layer.

[0049] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0050] The special equipment inspection model is built using a convolutional neural network. The activation function calculation expression in the convolutional neural network is as follows:

[0051]

[0052] in, is the activation function, is the input regional special equipment image feature value;

[0053] The output layer is connected to the image data processing layer;

[0054] Through multi-label classification of the inspection structure of the special equipment to be inspected, the loss function is fitted to the classification result. The calculation expression is as follows:

[0055] MES =− ∑ m = 1 n [ Q m × ln p m 1 − p m + ln( 1 − p m )] n

[0056] in, is the loss term output by the multi-label corresponding loss function, is the label coefficient, is the feature probability corresponding to the m label.

[0057] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0058] The evaluation unit is used to evaluate the characteristic values ​​of multiple labels of the special equipment to be inspected output by the special equipment inspection model, and output the structural index of the special equipment to be inspected;

[0059] The evaluation expression is as follows:

[0060]

[0061] in, is the accuracy of the feature values ​​corresponding to multiple labels, is the correct value of the special equipment tag characteristic value to be inspected. is the value of the error of the characteristic value of the special equipment label to be inspected, j is an ordinal number taking the value of 1, 2, 3, ..., z, and z is the number of special equipment labels to be inspected, taking a positive integer.

[0062] As a preferred solution of the special equipment image inspection intelligent analysis system of the present invention, wherein:

[0063] The feedback module includes data transmission and reception, feedback mode, and security and privacy protection;

[0064] The report generation module includes report generation, report content, quality control and verification, review and approval;

[0065] The interactive module includes real-time data display, test result storage and user interactive operations.

[0066] An intelligent analysis method for special equipment image inspection, comprising:

[0067] S1, collecting the overall special equipment image, and preliminarily processing the collected overall special equipment image, and performing regional segmentation on the preliminarily processed overall special equipment image;

[0068] S2. Identify and analyze the preliminarily processed regional special equipment images, and call preset safety inspection rules according to the identified special equipment types;

[0069] S3. Establish a special equipment inspection model to analyze and inspect the regional special equipment images after preliminary processing;

[0070] S4. Perform image inspection on the special equipment through the loss item of the characteristic value corresponding to the special equipment type inspection label, and finally evaluate the image inspection result, and the special equipment inspection model outputs the special equipment inspection result;

[0071] S5. Feedback special equipment inspection results and regional special equipment image analysis results, and output special equipment inspection results and inspection reports;

[0072] S6. Display through a visualization screen and save to a storage structure.

[0073] A computer device includes: a memory for storing instructions; a processor for executing the instructions, so that the device implements the above-mentioned special equipment image inspection intelligent analysis method.

[0074] A computer-readable storage medium stores a computer program, which, when executed, implements the above-mentioned special equipment image inspection intelligent analysis method.

[0075] Beneficial effects of the present invention:

[0076] The present invention recognizes the collected special equipment images through the neural convolution algorithm, which greatly improves the accuracy of special equipment inspection;

[0077] The present invention establishes a special equipment inspection model, combines special equipment inspection criteria and collects special equipment images, analyzes the special equipment area, and generates a special equipment inspection report, which greatly reduces the cost and subjectivity of manual analysis, improves the accuracy of analysis, and shortens the time period of special equipment inspection;

[0078] The present invention establishes a feedback module to continuously optimize and correct the image recognition algorithm and image analysis model, thereby improving the accuracy of image recognition and analysis;

[0079] The collected audio and video images are analyzed intelligently to assist in the issuance of inspection reports. Taking the supervision and inspection of passenger and cargo special equipment as an example, the audio and video records involve speed limiter safety clamps, buffers, traction capacity tests, braking performance tests, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0081] Figure 1 This is a system composition diagram of a special equipment image inspection intelligent analysis system of the present invention;

[0082] Figure 2 This is a flow chart of an intelligent analysis method for image inspection of special equipment according to the present invention;

[0083] Figure 3 The present invention discloses an image processing area workflow of an intelligent analysis system for image inspection of special equipment. DETAILED DESCRIPTION

[0084] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0085] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0086] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0087] Embodiment 1

[0088] like Figure 1 As shown, a special equipment image inspection intelligent analysis system includes:

[0089] Image acquisition module, used to acquire the whole special equipment image in real time;

[0090] Wherein, the overall special equipment image is collected by the image collector, and the overall special equipment image includes images of all structures of the special equipment;

[0091] The overall special equipment image includes images of all positions and structures of the special equipment to be inspected;

[0092] The processing module includes a preprocessing unit and an image region segmentation unit, wherein the preprocessing unit is used to process the collected regional special equipment image, and the image region segmentation unit is used to segment the preprocessed regional special equipment image into a plurality of regions;

[0093] The preprocessing unit includes filtering, color conversion, and grayscale processing

[0094] Furthermore, filtering reduces the noise impact in the image by weighted averaging the pixels of the regional special equipment image. The calculation expression of the filtering is as follows:

[0095] Γ ( x , y ) = ∑ i = 1 n [ f ( x i , y i ) × e x i 2 + y i 2 2 × γ γ × 2 ]

[0096] in, is the pixel of the regional special equipment image after filtering, is the filter standard deviation coefficient, is the original value of the regional special equipment image pixel, is the x-coordinate of the ith pixel value, is the y coordinate of the i-th pixel value, i is an ordinal number of 1, 2, 3, ..., n, n is an arbitrary integer, and e is an exponential constant;

[0097] Furthermore, the color conversion converts the color of the filtered regional special equipment image into a more easily recognizable color space by converting the color image into a grayscale image;

[0098] Furthermore, the grayscale image is used to process the edge and texture information in the regional special equipment image, and greatly reduces the computational complexity of subsequent inspection of special equipment;

[0099] Furthermore, the grayscale processing uses the Sobel operator to approximate the longitudinal gradient and transverse gradient of the pixels of the grayscale image. The longitudinal gradient calculation expression is as follows:

[0100] G Y = [ α 1 , α 2 ,... α n − 1 ] T × A

[0101] in, is the coefficient matrix for calculating the longitudinal gradient, and A is the grayscale image;

[0102] The calculation expression of the transverse gradient is as follows:

[0103] G X = [ β 1 , β 2 ,... β n − 1 ] T × A

[0104] in, Calculate the coefficient matrix for the lateral gradient;

[0105] The grayscale processing calculation expression is as follows:

[0106]

[0107] Among them, G is the regional special equipment image after grayscale processing;

[0108] Further, assume that the preprocessed image is I, with a size of H×W, where H is the height and W is the width, and the annotation map is G. The annotation map is a matrix with the same size as I. Each pixel value in the annotation map G indicates which region the pixel belongs to.

[0109] Furthermore, the image I is adjusted to the target size by bilinear interpolation, and the target size image is denoised to obtain the processed target size image ;

[0110] Further, the processed target size image is segmented by an image region segmentation unit;

[0111] The image region segmentation unit includes extracting the feature map F of the image, generating candidate regions, and the candidate regions include the regions that need to be identified and the regions that do not need to be identified for the special equipment, and then mapping each candidate region to a feature map of a fixed size through the RoI Align layer. The size is K×K, and then each candidate area after mapping is classified, and the category probability P and bounding box regression value △ are output. The pixel-level mask of each candidate area is generated through the mask branch to determine whether the pixels in the candidate area belong to the special equipment to be detected target, and the probability that the pixels in the candidate area belong to the special equipment to be detected target is output;

[0112] Furthermore, the total loss of regional segmentation is calculated through classification loss, bounding box regression loss and mask loss, and the total loss is optimized through the optimizer, and finally the regional segmentation of the overall image of special equipment is completed, and the image inspection intelligent analysis of the target area is performed;

[0113] The classification loss calculation expression is as follows:

[0114]

[0115] in, is the classification loss value of special equipment images, is the total number of categories, is the categorical count value, is the category probability of special equipment image classification, is the true category of the i-th candidate region;

[0116] The category probability of classification refers to the probability of judging whether the regional image is the target image;

[0117]

[0118] in, is the bounding box loss, To predict the bounding box regression value, is the true bounding box regression value, SmoothL1 is a loss function that combines linear loss and quadratic function loss, is the total number of predicted bounding boxes, is the count of the predicted bounding box;

[0119] SmoothL1 is calculated by expressing the difference between the predicted value and the true value. If the absolute value of the difference is less than 1, the loss value is 0.5 times the square of the difference. If the absolute value of the difference is not less than 1, the loss value is the absolute value of the difference minus 0.5.

[0120] The mask loss function is calculated by the binary cross entropy loss function;

[0121] The identification and analysis module includes an identification unit for identifying the type and operating status of special equipment, a calling unit for calling special equipment inspection criteria for the identified special equipment type, and an analysis unit for analyzing the processed regional special equipment image;

[0122] The type of special equipment is identified by receiving the processed regional special equipment image through the identification unit, and the type of special equipment is identified according to the characteristics and structure of the special equipment. The special equipment type includes passenger special equipment, freight elevator and other types of special equipment;

[0123] Furthermore, the recognition unit is implemented through other visible features such as specific structures, button layouts, etc. in the image.

[0124] Furthermore, the retrieval unit includes the retrieval of special equipment safety inspection rules and technical review;

[0125] Furthermore, according to the identified type of special equipment, the preset safety inspection rules are called. The rules include the release of the new inspection regulations TSG T7008-2023, which requires audio and video collection for relevant inspection items. The collected audio and video images can be analyzed intelligently to assist in the issuance of inspection reports. Taking the supervision and inspection of passenger and cargo special equipment as an example, the audio and video records involve speed limiter safety clamps, buffers, traction capacity tests, braking performance tests, etc.

[0126] Technical review includes inspection image correction, inspection method research and technical research management;

[0127] The analysis unit is used to further analyze the processed regional special equipment image and extract features;

[0128] The analysis unit receives the special equipment safety inspection rules and technical review, performs preliminary analysis on the processed regional special equipment image through the special equipment safety inspection rules and technical review of the special equipment classification, and extracts the feature value of the image;

[0129] Furthermore, the feature value extraction of the image is initially performed according to the special equipment safety inspection rules corresponding to the special equipment to be inspected;

[0130] like Figure 1 As shown, the inspection module includes a special equipment inspection model for inspecting the identified special equipment type, a pruning unit for tuning the special equipment inspection model, and an evaluation unit for evaluating the special equipment image inspection result;

[0131] Among them, the special equipment inspection model includes convolution layer, input layer, image data processing layer and output layer;

[0132] like Figure 3As shown, the collected special equipment image is loaded, and the special equipment image is preliminarily processed to determine whether the image is correct. If the image is preliminarily determined to be an erroneous image, the erroneous and repeated special equipment image data is removed. If the image is a correct image, the image size, step size and padding value are set to perform convolution operation on the processed special equipment image, and the special equipment regional feature value is output, and the monitoring of the classification inspection structure of the special equipment to be inspected by multi-label classification is completed. Finally, the monitoring result is fitted and calculated through the loss function and the special equipment detection result is evaluated;

[0133] At the same time, optimizing the generalization learning ability of the convolutional neural network shortens the calculation time for setting the image size, step size, and padding value. It also shortens the calculation time for the convolution operation process on the processed special equipment images and improves the model response speed.

[0134] Among them, the convolution processing is completed through the convolution layer, the input image size, step size and padding are set, and the x-axis calculation expression of the output feature map of the convolution processing is as follows:

[0135]

[0136] in, is the x-axis size of the output feature map, is the x-axis length of the original input image, is the preset size of the convolution kernel, L is the step size, and F is the padding value;

[0137] The y-axis calculation expression of the output feature map of the convolution process is as follows:

[0138]

[0139] in, is the y-axis size of the output feature map, The y-axis length of the original input image;

[0140] Furthermore, the feature map after convolution processing is input to the image data processing layer through the input layer, and the feature value is further extracted from the feature map through the image data processing layer, and finally the data of the regional special equipment image is output through the output layer;

[0141] The activation function is an indispensable component of the convolutional neural network. It enhances the nonlinear expression ability of the neural network. It usually uses a nonlinear method to activate feature extraction, transfer the extracted feature information, and improve the generalization learning ability of the convolutional neural network. The activation function calculation expression is as follows:

[0142]

[0143] in, is the activation function, is the input regional special equipment image feature value;

[0144] The output layer is connected to the image data processing layer;

[0145] The loss function fits the correct features of special equipment image recognition through the binary method. If the correct features in the special equipment image are recognized with a small number of recognitions, it means that the recognition effect of the special equipment inspection model is good. If the correct features in the special equipment image are recognized with a large number of recognitions, it means that the recognition effect of the special equipment inspection model is poor, and the special equipment inspection model needs to be tuned through the pruning unit.

[0146] Since the regional special equipment image contains multiple eigenvalues, the loss function is fitted through the multi-label classification inspection structure of the special equipment to be inspected. The calculation expression is as follows:

[0147] MES =− ∑ m = 1 n [ Q m × ln p m 1 − p m + ln( 1 − p m )] n

[0148] in, is the loss term output by the multi-label corresponding loss function, is the label coefficient, is the feature probability corresponding to the m label;

[0149] Monitor and predict the results of multi-label classification tasks. Through the loss function pair, each target label can be regarded as an independent binary classification task, and the classification loss of each label is independent of each other.

[0150] is the feature probability corresponding to the m label, which actually means the probability that the feature value of special equipment image recognition is the correct feature. The value of is less than 1 and greater than 0;

[0151] The label coefficient value is 0 or 1;

[0152] By calculating the probability of the exact values ​​of the eigenvalues ​​corresponding to the multiple labels required to be inspected by the safety inspection rules in the regional special equipment image, the loss items of the special equipment inspection model for inspecting each label of the special equipment to be inspected are obtained, and the loss items of the eigenvalues ​​corresponding to the labels are fed back to the special equipment inspection model, and the special equipment inspection model is optimized through the pruning unit;

[0153] Further, the evaluation unit is used to evaluate the characteristic values ​​of multiple labels of the special equipment to be inspected output by the special equipment inspection model, and output the structural index of the special equipment to be inspected;

[0154] The evaluation expression is as follows:

[0155]

[0156] in, is the accuracy of multi-label corresponding feature values, is the correct value of the special equipment tag characteristic value to be inspected. is the value of the error of the characteristic value of the special equipment label to be inspected, j is an ordinal number taking 1, 2, 3, ..., z, z is the number of special equipment labels to be inspected, taking a positive integer;

[0157] The correct value of the characteristic value of the label of the special equipment to be inspected is the characteristic value corresponding to the label value of a certain indicator of the special equipment to be inspected, which is recorded as 1. The characteristic values ​​corresponding to the same label are repeatedly sampled, and the characteristic values ​​corresponding to all labels are fully evaluated;

[0158] Feedback module, used to feedback special equipment inspection results and image analysis results;

[0159] The feedback module includes data transmission and reception, feedback methods, and security and privacy protection;

[0160] Data transmission and reception include real-time and stability;

[0161] Real-time is used to ensure real-time reception of data from special equipment inspection and image analysis.

[0162] Stability is used to ensure the stability of data transmission and minimize information loss or delay.

[0163] Feedback methods include interface feedback, notification system and API interface;

[0164] Interface feedback displays inspection results and analysis data through the user interface, including charts, digital data or alarm messages.

[0165] Notification Systems Use notification or alarm systems to send important information or warnings to appropriate personnel, such as special equipment failures or safety issues.

[0166] API interface provides an API interface to allow other systems or services to access feedback data for further processing or integration;

[0167] Security and privacy protection are used to encrypt and protect transmitted data to prevent unauthorized access or information leakage, and ensure that the design of the feedback module complies with relevant laws, regulations and privacy protection policies;

[0168] Report generation module, used to generate and verify special equipment inspection reports;

[0169] The report generation module includes report generation, report content, quality control and verification, review and approval;

[0170] Report generation automatically generates special equipment inspection reports through predefined templates and rules, and generates reports in different formats such as PDF, HTML or Excel according to user needs.

[0171] The report content includes key information such as the operating status of special equipment, safety assessment, fault inspection, maintenance recommendations, etc. Charts, trend charts or camera images are inserted through charts and images to intuitively display inspection results and analyze data.

[0172] Quality control and verification ensure that the data in the report is accurate, reflects the actual status and problems of special equipment, follows industry standards and specifications, and ensures that the report content meets technical requirements.

[0173] The review and approval process provides a review interface or process, allowing relevant personnel to review and approve the report content, and supports electronic signature function for final confirmation of the integrity and reliability of the report.

[0174] An interactive module for displaying and storing special equipment inspection results;

[0175] The interactive module includes real-time data display, test result storage and user interactive operations;

[0176] Real-time data display includes monitoring interface and graphical display;

[0177] Monitoring interface provides an intuitive monitoring interface that displays the current operating status of special equipment, real-time data of various sensors (such as temperature, pressure, speed, etc.) and any detected abnormalities or warning information;

[0178] Graphical display clearly shows the status and operation of each part of the special equipment through charts, indicator lights or text descriptions;

[0179] Inspection result storage includes data logging and database storage;

[0180] Data logging records the results and data of each test, including timestamp, specific data, output of the test algorithm, etc.

[0181] Database storage stores data in a scalable database for long-term preservation and subsequent analysis.

[0182] User interaction operations include operation interface and permission management;

[0183] The operation interface provides a user-friendly operation interface, allowing operators to view historical data, export reports, set monitoring parameters, etc.

[0184] Permission management sets different permissions based on user roles to ensure that only authorized personnel can access and modify relevant data.

[0185] Embodiment 2

[0186] like Figure 2 As shown, a special equipment image inspection intelligent analysis method includes:

[0187] S1, collecting the overall special equipment image, and preliminarily processing the collected overall special equipment image, and performing regional segmentation on the preliminarily processed overall special equipment image;

[0188] S2. Identify and analyze the preliminarily processed regional special equipment images, and call preset safety inspection rules according to the identified special equipment types;

[0189] S3. Establish a special equipment inspection model to analyze and inspect the regional special equipment images after preliminary processing;

[0190] S4. Perform image inspection on the special equipment through the loss item of the characteristic value corresponding to the special equipment type inspection label, and finally evaluate the image inspection result, and the special equipment inspection model outputs the special equipment inspection result;

[0191] S5. Feedback special equipment inspection results and regional special equipment image analysis results, and output special equipment inspection results and inspection reports;

[0192] S6. Display through a visualization screen and save to a storage structure.

[0193] Embodiment 3

[0194] A computer device includes: a memory for storing instructions; a processor for executing the instructions, so that the device implements the above-mentioned special equipment image inspection intelligent analysis method.

[0195] A computer-readable storage medium stores a computer program, which, when executed, implements the above-mentioned special equipment image inspection intelligent analysis method.

[0196] Importantly, it should be noted that the construction and arrangement of the present application shown in a plurality of different exemplary embodiments are only exemplary. Although only two embodiments are described in detail in this disclosure, it should be readily understood by those who refer to this disclosure that many modifications are possible, for example, the size, scale, structure, shape and ratio of various elements, and parameter values ​​(e.g., temperature, pressure, etc.), installation arrangement, use of materials, color, directional changes, etc., without substantially departing from the novel teachings and advantages of the subject matter described in the application. For example, the element shown as regional shaping can be composed of multiple parts or elements, the position of the element can be inverted or otherwise changed, and the nature or number or position of the discrete element can be changed or changed. Therefore, all such modifications are intended to be included in the scope of the present invention. The order or sequence of any process or method steps can be changed or reordered according to alternative embodiments. Any "device plus function" clause is intended to cover the structure of the execution function described in this article, and is not only structurally equivalent but also equivalent structure. Without departing from the scope of the present invention, other replacements, modifications, changes and omissions can be made in the design, operating conditions and arrangement of the exemplary embodiment. Therefore, the invention is not limited to a specific embodiment, but extends to several modifications still falling within the scope of the appended claims.

[0197] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment may not be described (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention).

[0198] It should be understood that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but for those of ordinary skill having the benefit of this disclosure, the development effort will be a routine task of design, fabrication, and production without undue experimentation.

[0199] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A special equipment image inspection intelligent analysis system, characterized in that: include: Image acquisition module, used to acquire the whole special equipment image in real time; The processing module includes a preprocessing unit and an image region segmentation unit, wherein the preprocessing unit is used to process the collected regional special equipment image, and the image region segmentation unit is used to segment the preprocessed regional special equipment image into a plurality of regions; The identification and analysis module includes an identification unit for identifying the type and operating status of special equipment, an analysis unit for analyzing processed images of special equipment in different areas, and a calling unit for calling special equipment inspection criteria for the identified special equipment type; The analysis unit receives the special equipment safety inspection rules and technical review, performs preliminary analysis on the processed regional special equipment image through the special equipment safety inspection rules and technical review of the special equipment classification, and extracts the feature value of the image; The feature value extraction of the image is preliminarily performed according to the special equipment safety inspection rules corresponding to the special equipment to be inspected; An inspection module, including a special equipment inspection model for inspecting the identified special equipment type, a pruning unit for tuning the special equipment inspection model, and an evaluation unit for evaluating special equipment image inspection results; The special equipment inspection model includes a convolution layer, an input layer, an image data processing layer and an output layer; The convolution process is completed through the convolution layer, and the input image size, step size and padding are set. The x-axis calculation expression of the output feature map of the convolution process is as follows: ； in, is the x-axis size of the output feature map, is the x-axis length of the original input image, is the preset size of the convolution kernel, L is the step size, and F is the padding value; The y-axis calculation expression of the output feature map of the convolution process is as follows: ； in, is the y-axis size of the output feature map, The y-axis length of the original input image; The size of the feature map is ; The feature map after convolution processing is input to the image data processing layer through the input layer, and the feature value is further extracted from the feature map through the image data processing layer, and finally the data of the regional special equipment image is output through the output layer; The special equipment inspection model is built using a convolutional neural network. The activation function calculation expression in the convolutional neural network is as follows: ； in, is the activation function, is the input regional special equipment image feature value; The output layer is connected to the image data processing layer; Through multi-label classification of the inspection structure of the special equipment to be inspected, the loss function is fitted to the classification result. The calculation expression is as follows: ； in, is the loss term output by the multi-label corresponding loss function, is the label coefficient, is the feature probability corresponding to the m label; The evaluation unit is used to evaluate the characteristic values ​​of multiple labels of the special equipment to be inspected output by the special equipment inspection model, and output the structural index of the special equipment to be inspected; The evaluation expression is as follows: ； in, is the accuracy of the feature values ​​corresponding to multiple labels, is the correct value of the special equipment tag characteristic value to be inspected. is the value of the error of the characteristic value of the special equipment label to be inspected, j is an ordinal number taking 1, 2, 3, ..., z, z is the number of special equipment labels to be inspected, taking a positive integer; Feedback module, used to feedback special equipment inspection results and image analysis results; Report generation module, used to generate and verify special equipment inspection reports; Interactive module, used to display special equipment inspection results and store inspection results.

2. According to claim 1, a special equipment image inspection intelligent analysis system is characterized by: Collecting an overall special equipment image by an image collector, wherein the overall special equipment image includes images of all structures of the special equipment; Process the collected overall special equipment image, remove noise by filtering, and enhance the image clarity of the filtered overall special equipment image by color conversion, and then enhance the converted new image by edge inspection and grayscale processing; The image region segmentation unit includes extracting the feature map F of the image, generating candidate regions, and the candidate regions include the regions that need to be identified and the regions that do not need to be identified for the special equipment, and then mapping each candidate region to a feature map of a fixed size through the RoI Align layer. The size is K×K, and then each candidate area after mapping is classified, and the category probability P and bounding box regression value △ are output. The pixel-level mask of each candidate area is generated through the mask branch to determine whether the pixels in the candidate area belong to the special equipment to be detected target, and the probability that the pixels in the candidate area belong to the special equipment to be detected target is output; The total loss of region segmentation is calculated through classification loss, bounding box regression loss and mask loss, and the total loss is optimized through the optimizer to finally complete the regional segmentation of the overall image of special equipment.

3. The special equipment image inspection intelligent analysis system according to claim 2, characterized in that: The filtering reduces the noise effect in the image by weighted averaging the pixels of the regional special equipment image. The calculation expression of the filtering is as follows: ； in, is the pixel of the regional special equipment image after filtering, is the filter standard deviation coefficient, is the original value of the regional special equipment image pixel, is the x-coordinate of the ith pixel value, is the y coordinate of the i-th pixel value, i is an ordinal number of 1, 2, 3, ..., n, n is an arbitrary integer, and e is an exponential constant; Color conversion converts the color of the filtered regional special equipment image into a more easily recognizable color space by converting the color image into a grayscale image; The grayscale image is used to process edge and texture information in the regional special equipment image.

4. The special equipment image inspection intelligent analysis system according to claim 3 is characterized by: The grayscale processing performs approximate calculation of the longitudinal gradient and transverse gradient of the pixels of the grayscale image through the Sobel operator. The longitudinal gradient calculation expression is as follows: ； in, is the coefficient matrix for calculating the longitudinal gradient, and A is the grayscale image; The lateral gradient calculation expression is as follows: ； in, Calculate the coefficient matrix for the lateral gradient; The grayscale processing calculation expression is as follows: ； Among them, G is the regional special equipment image after grayscale processing.

5. The special equipment image inspection intelligent analysis system according to claim 4 is characterized in that: The feedback module includes data transmission and reception, feedback mode, and security and privacy protection; The report generation module includes report generation, report content, quality control and verification, review and approval; The interactive module includes real-time data display, test result storage and user interactive operations.

6. A special equipment image inspection intelligent analysis method, implemented based on a special equipment image inspection intelligent analysis system according to any one of claims 1 to 5, characterized in that: include: S1, collecting the overall special equipment image, and preliminarily processing the collected overall special equipment image, and performing regional segmentation on the preliminarily processed overall special equipment image; S2. Identify and analyze the preliminarily processed regional special equipment images, and call preset safety inspection rules according to the identified special equipment types; S3. Establish a special equipment inspection model to analyze and inspect the regional special equipment images after preliminary processing; S4. Perform image inspection on the special equipment through the loss item of the characteristic value corresponding to the special equipment type inspection label, and finally evaluate the image inspection result, and the special equipment inspection model outputs the special equipment inspection result; S5. Feedback special equipment inspection results and regional special equipment image analysis results, and output special equipment inspection results and inspection reports; S6. Display through a visualization screen and save to a storage structure.

7. A computer device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instructions so that the device implements a special equipment image inspection intelligent analysis method as described in claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, an intelligent analysis method for special equipment image inspection as described in claim 6 is implemented.

Images