Automatic sample collection method based on artificial intelligence cloud edge collaboration

Abstract

An automatic sample collection method based on artificial intelligence cloud edge collaboration comprises the steps of deploying a high-definition camera and an image recognition technology to capture equipment defects, automatically recognizing and associating defect information, uploading the defect information to a cloud platform for sample cleaning and classification, and issuing high-quality samples to a station end system for model training and optimization. Real-time monitoring and early warning are realized, and finally, sample data are shared through the cloud, so that the intelligent level and the sample utilization efficiency of the whole inspection system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial equipment monitoring and defect detection, and particularly to a method for automatic sample collection based on artificial intelligence cloud-edge collaboration. Background Art

[0002] In the field of industrial production, the monitoring and defect detection of equipment and facilities are of great importance. However, there are many prominent problems in the current related technologies:

[0003] First, there are significant defects in the algorithm training samples. The sample resources are seriously insufficient, and the collection efficiency is extremely low. In addition, due to the diverse types of equipment from which the pictures are sourced, the picture quality is uneven. Moreover, the pictures collected by the on-site inspection system have not been fully utilized, greatly affecting the effectiveness and accuracy of subsequent model training.

[0004] Second, there are limitations in the use of the artificial intelligence platform. Its main users are algorithm manufacturers and platform administrators, and there is a lack of collaboration and sharing in the management of models and samples among business use departments. This makes it difficult to effectively promote management work such as the iterative update of specific substation inspection algorithms in daily operations, and the flexible invocation of equipment intelligent inspection models cannot be achieved, seriously restricting the efficiency and effectiveness of work.

[0005] Third, after the provincial-level artificial intelligence platform optimizes the algorithm and generates a new version of the model, it faces the dilemma of the lack of application channels for the model to be distributed to the on-site edge side. This makes it difficult for the new optimization results to be promptly put into actual use, hindering the development and application process of equipment monitoring and defect detection technologies. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a method for automatic sample collection based on artificial intelligence cloud-edge collaboration.

[0007] The method for automatic sample collection based on artificial intelligence cloud-edge collaboration is characterized by the following specific steps:

[0008] Step 1: Deploy a remote intelligent inspection system. Deploy a remote intelligent inspection system on industrial equipment and facilities, which includes high-definition cameras and image recognition technology, and ensure that equipment defects can be clearly captured under various lighting and weather conditions.

[0009] Step 2: Automatically identify equipment defects. Automatically detect and identify equipment defects through image recognition technology; associate the identified defect images with equipment information, including equipment type, location, time, etc.

[0010] Step 3: Upload defect images to the cloud. Upload the defect images and equipment information to the artificial intelligence platform in the cloud.

[0011] Step 4: Sample cleaning, which is carried out on the cloud platform; the defective image samples uploaded in Step 3 are cleaned and labeled to ensure the sample quality; then they are classified according to the device type and defect characteristics to build a high-quality sample library;

[0012] Step 5: Distribute the cleaned samples to the on-site inspection system. The cleaned samples are automatically distributed to the on-site inspection system, which is used for model training and defect recognition to improve the utilization efficiency of the samples;

[0013] Step 6: Model training and optimization. In the on-site inspection system, use the distributed samples for model training and optimization; through deep learning algorithms, continuously improve the recognition accuracy and robustness of the model;

[0014] Step 7: Real-time monitoring and warning. Use the trained model to perform defect recognition and warning on the device images collected in real time; promptly detect device defects and improve the intelligence level of the remote intelligent inspection system;

[0015] Step 8: Share and utilize the sample data. Share the collected sample data on the cloud to promote the utilization and exchange of samples. The sample data can be obtained and used through the cloud platform to improve the intelligence level of the inspection system.

[0016] Furthermore, the deployment of the remote intelligent inspection system in Step 1 can be expressed as:

[0017] Step 1.1: Select monitoring devices. Select network cameras with high resolution and night vision functions as monitoring devices; these cameras can provide clear video streams so that details of the devices can be captured even in low-light environments;

[0018] Step 1.2: Install monitoring devices. Install multiple network cameras at key positions, such as above and around production lines and machinery; use brackets and protective covers to protect the cameras to ensure their stability and prevent accidental damage; at the same time, ensure that the angles and positions of the cameras can cover the areas to be monitored to the greatest extent;

[0019] Step 1.3: Deploy image recognition technology. Integrate deep learning-based image recognition technology into the remote intelligent inspection system; this technology analyzes a large amount of historical image data to learn how to recognize various device defects, such as cracks, wear, looseness, etc.; the system uses a pre-trained model and fine-tunes it for specific industrial scenarios to improve the recognition accuracy;

[0020] Step 1.4: Device Information Association. In the remote intelligent inspection system, through the built-in image recognition technology, the identified defect images are associated with device information; the device information includes the type, location, operating status, etc. of the device, and this information can be obtained from the device management system through the wireless network; through the association, the system can provide detailed background information for each defect image, facilitating subsequent analysis and processing;

[0021] Step 1.5: Data Transmission Preparation. In order to upload the identified defect images and device information to the provincial artificial intelligence platform in the cloud, the remote intelligent inspection system needs to configure an appropriate data transmission mechanism; an encrypted VPN connection is set up in the system to ensure the security and stability of the data during transmission; at the same time, according to the interface requirements of the cloud platform, the format and protocol of data upload are configured;

[0022] Step 1.6: System Testing and Optimization. After the deployment is completed, a comprehensive test is carried out on the remote intelligent inspection system; the test includes aspects such as the image quality of the camera, the accuracy of image recognition, and the stability of data transmission; by simulating different device defects and environmental conditions, the performance of the system is evaluated.

[0023] Furthermore, the process of automatically identifying device defects in Step 2 can be expressed as follows:

[0024] Step 2.1, Image Preprocessing Stage. Convert the color image to a grayscale image. The formula for converting a color image to a grayscale image is:

[0025] I gray (x,y) = 0.299I(x,y,1) + 0.587I(x,y,2) + 0.114I(x,y,3)

[0026] where I(x,y,c) represents the color image, (x,y) represents the pixel coordinates of the color image, c = 1, 2, 3 represent red, green, and blue respectively, and I gray (x,y) represents the grayscale image;

[0027] Then, perform normalization processing on the grayscale image. The formula for normalizing to the interval [0,1] is:

[0028]

[0029] where I norm (x,y) represents the normalized grayscale image;

[0030] Step 2.2, Feature Extraction Stage. Use the processed grayscale image as the input of the feature extraction network. Let the input image be X, the convolution kernel be K, and the bias be b. The calculation formula for the output Y of the convolutional layer is:

[0031]

[0032] Among them, i and j are the pixel coordinates in the output feature map, and m and n are the coordinates in the convolution kernel; by stacking multiple convolutional layers, features at different levels can be extracted;

[0033] The pooling layer is used to reduce the data dimension and prevent overfitting; taking max pooling as an example, assuming the input feature map is F and the pooling window size is k×k, the calculation formula for the output P after pooling is:

[0034]

[0035] Among them, i and j are the pixel coordinates of the output after pooling, P is the extracted feature, and m and n are the coordinates during pooling;

[0036] Step 2.3, defect classification stage. After feature extraction, the obtained feature vector P is classified using SVM; the decision function of SVM is:

[0037] y = sign(w·P + b)

[0038] Among them, y is the image classification result, 1 indicates a defect, -1 indicates normal, sign is the sign function, w is the weight vector of SVM, b is the bias of SVM, and P is the extracted feature; in the training stage, the following objective function needs to be minimized:

[0039]

[0040] Meanwhile, satisfying the constraint condition y i (w·P i + b) ≥ 1, where i = 1, 2, …, n, n is the number of training samples, y i is the class label of the i-th training sample, and P i is the feature vector of the i-th training sample;

[0041] Step 2.4, associated device information stage. When a device defect is identified, the defect image is associated with the device information; a device information matrix is formed, which includes information such as device type, device location, and time;

[0042] The association between the identified defect image and the device information is achieved by establishing an association matrix, and the elements in the matrix represent the association degree between the defect image and the device information; the row elements corresponding in the association matrix record the indexes and specific values of the device information type, location, and time for subsequent query and processing.

[0043] Furthermore, the process of sample cleaning on the cloud platform in step 4 can be expressed as follows:

[0044] Step 4.1. Image quality screening: For the uploaded defective image samples, check the image clarity quality index; define the measure of image clarity as the variance S of the image 2 :

[0045]

[0046] where x i represents the gray value of the image pixel point, represents the average value of the gray values of all pixel points in the image, and n is the total number of pixel points; set the minimum variance threshold T1. When the variance of the image is less than T1, it is considered that the image clarity is insufficient, and it is removed from the sample;

[0047] Step 4.2. Remove irrelevant interference information: Use image segmentation to separate the device main body and the area where the defect is located, remove other irrelevant parts, extract the key areas of the device and the defect, remove background interference, and make the sample focus on the content related to the truly valuable device defects;

[0048] Step 4.3. Check and correct the annotation accuracy: Check whether the existing image annotation information is accurate; use a combination of manual sampling inspection and partial automated verification to correct the mislabeled information;

[0049] Step 4.4. Classify according to device type and defect characteristics: Classify and organize the samples after the previous cleaning steps according to the device type and defect characteristics; achieve this by establishing a classification rule library. For example, for motor equipment, if the defect characteristic is winding short circuit, it is classified into the motor - short circuit defect category; if it is bearing wear, it is classified into the motor - bearing wear category, etc.; the samples under each category jointly constitute a high - quality sample library, which is convenient for subsequent targeted data analysis, model training, etc. based on different device and defect situations.

[0050] Furthermore, the process of sending the cleaned samples to the substation inspection system in Step 5 can be expressed as follows:

[0051] Step 5.1, Sample preparation: After confirming that the sample data on the cloud platform has completed the cleaning and annotation process, select appropriate sample data according to the requirements of the substation inspection system;

[0052] Step 5.2, Data packaging: Package the selected sample data to ensure that the data will not be lost or damaged during transmission, so as to ensure data security and transmission efficiency;

[0053] Step 5.3, Transmission mechanism: Transmit the file through the File Transfer Protocol SFTP to ensure the stability and security of the network connection during data transmission;

[0054] Step 5.4, the distribution process, starts to distribute the packaged sample data to the on-site inspection system, monitors the data transmission process to ensure that all data successfully reaches the destination;

[0055] Step 5.5, data verification, after the on-site inspection system receives the data, verifies the integrity and correctness of the data. If data corruption or loss is found, the relevant data needs to be retransmitted;

[0056] Step 5.6, sample deployment, deploys the received sample data into the database or file system of the on-site inspection system for subsequent model training.

[0057] The sample automatic collection method based on artificial intelligence cloud-edge collaboration of the present invention has the following beneficial effects: The technical effects of the present invention are as follows:

[0058] 1. The present invention significantly improves the efficiency and quality of sample collection, ensures the diversity and accuracy of samples, and provides a reliable data basis for subsequent model training and defect identification.

[0059] 2. The present invention realizes the continuous optimization and update of the model. Through cloud-edge collaboration, it can adjust the model in a timely manner according to real-time data and feedback, improving the accuracy of equipment defect identification.

[0060] 3. The present invention strengthens the sharing and utilization of sample data, promotes collaborative work between different departments and systems, and improves the overall work efficiency and intelligent level. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flowchart of the present invention.

[0062] Figure 2 is a flowchart of automatically identifying equipment defects of the present invention.

[0063] Figure 3 is a deployment architecture diagram of the on-site inspection system and the remote intelligent inspection system of the present invention.

[0064] Figure 4 is a business architecture diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0065] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments:

[0066] The present invention proposes a method for automatic sample collection based on artificial intelligence cloud-edge collaboration. First, a remote intelligent inspection system is deployed to collect device images, identify defects, and upload the associated device information to the cloud. The samples are cleaned in the cloud, a sample library is constructed by classification, and then it is sent to the station-side inspection system. The station-side is used for model training and optimization to achieve real-time monitoring and early warning, as well as the shared utilization of sample data. Through cloud-edge collaboration, a complete process is formed to improve the intelligent level of device monitoring and defect identification. The flowchart of the invention is as Figure 1 shown.

[0067] Step 1: Deploy a remote intelligent inspection system. Deploy a remote intelligent inspection system on industrial equipment and facilities, which includes high-definition cameras and image recognition technology, and ensure that device defects can be clearly captured under various lighting and weather conditions.

[0068] Step 1.1: Select monitoring devices. Select network cameras with high resolution and night vision functions as monitoring devices. These cameras can provide clear video streams so that details of the devices can be captured even in low-light environments.

[0069] Step 1.2: Install monitoring devices. Install multiple network cameras at key positions, such as above and around production lines and machinery. Use brackets and protective covers to protect the cameras to ensure their stability and prevent accidental damage. At the same time, ensure that the angles and positions of the cameras can cover the areas to be monitored to the greatest extent.

[0070] Step 1.3: Deploy image recognition technology. Integrate deep learning-based image recognition technology into the remote intelligent inspection system. This technology analyzes a large amount of historical image data to learn how to identify various device defects, such as cracks, wear, looseness, etc. The system uses a pre-trained model and fine-tunes it for specific industrial scenarios to improve the accuracy of recognition.

[0071] Step 1.4: Associate device information. In the remote intelligent inspection system, through the built-in image recognition technology, associate the identified defect images with device information. Device information includes the type, location, operating status, etc. of the device, and this information can be obtained from the device management system through a wireless network. Through association, the system can provide detailed background information for each defect image, facilitating subsequent analysis and processing.

[0072] Step 1.5: Prepare for data transmission. In order to upload the identified defect images and device information to the provincial artificial intelligence platform in the cloud, the remote intelligent inspection system needs to configure a suitable data transmission mechanism. Set up an encrypted VPN connection in the system to ensure the security and stability of data during transmission. At the same time, configure the format and protocol of data upload according to the interface requirements of the cloud platform.

[0073] Step 1.6: System Testing and Optimization. After deployment, a comprehensive test was conducted on the remote intelligent inspection system. The test included aspects such as the image quality of the camera, the accuracy of image recognition, and the stability of data transmission. By simulating different equipment defects and environmental conditions, the performance of the system was evaluated.

[0074] Step 2: Automatically Identify Equipment Defects. Through image recognition technology, automatically detect and identify equipment defects. Associate the identified defect images with equipment information, including equipment type, location, time, etc. The flowchart for automatically identifying equipment defects is as Figure 2 shown.

[0075] Step 2.1, Image Preprocessing Stage. Convert the color image to a grayscale image. The formula for converting a color image to a grayscale image is:

[0076] I gray (x,y) = 0.299I(x,y,1) + 0.587I(x,y,2) + 0.114I(x,y,3)

[0077] where I(x,y,c) represents the color image, (x,y) represents the pixel coordinates of the color image, c = 1, 2, 3 represent red, green, and blue respectively, and I gray (x,y) represents the grayscale image.

[0078] Then, perform normalization processing on the grayscale image. The formula for normalizing to the interval [0,1] is:

[0079]

[0080] where I norm (x,y) represents the normalized grayscale image.

[0081] Step 2.2, Feature Extraction Stage. Use the processed grayscale image as the input of the feature extraction network. Let the input image be X, the convolution kernel be K, and the bias be b. The calculation formula for the output Y of the convolution layer is:

[0082]

[0083] where i and j are the pixel coordinates in the output feature map, and m and n are the coordinates in the convolution kernel. By stacking multiple convolution layers, features at different levels can be extracted.

[0084] The pooling layer is used to reduce the data dimension and prevent overfitting. Taking max pooling as an example, let the input feature map be F, the pooling window size be k×k, and the calculation formula for the output P after pooling is:

[0085]

[0086] Among them, i and j are the pixel coordinates of the output after pooling, P is the extracted feature, and m and n are the coordinates during pooling.

[0087] Step 2.3, defect classification stage. After feature extraction, the obtained feature vector P is classified using SVM. The decision function of SVM is:

[0088] y = sign(w·P + b)

[0089] Among them, y is the image classification result, 1 indicates a defect, -1 indicates normal, sign is the sign function, w is the weight vector of SVM, b is the bias of SVM, and P is the extracted feature. In the training stage, the following objective function needs to be minimized:

[0090]

[0091] While satisfying the constraint condition y i (w·P i + b) ≥ 1, where i = 1, 2, …, n, n is the number of training samples, y i is the class label of the i-th training sample, and P i is the feature vector of the i-th training sample.

[0092] Step 2.4, associated device information stage. When a device defect is identified, the defect image is associated with the device information. A device information matrix is formed, which contains information such as device type, device location, and time.

[0093] The association between the identified defect image and the device information is achieved by establishing an association matrix. The elements in the matrix represent the degree of association between the defect image and the device information. The corresponding row elements in the association matrix record the indexes and specific values of the device information type, location, and time for subsequent query and processing.

[0094] Step 3: Upload the defect image to the cloud. The defect image and the device information are uploaded to the artificial intelligence platform in the cloud.

[0095] Step 4: Sample cleaning. Sample cleaning is performed on the cloud platform. The defect image samples uploaded in Step 3 are cleaned and labeled to ensure the sample quality. Then, they are classified according to the device type and defect characteristics to construct a high-quality sample library.

[0096] Step 4.1. Image quality screening. For the uploaded defect image samples, check the image clarity quality index. Define the measure of image clarity as the variance S of the image 2 :

[0097]

[0098] Among them, xi represents the grayscale value of an image pixel represents the average grayscale value of all image pixels, where n is the total number of pixels. Set the minimum variance threshold T1. When the variance of the image is less than T1, it is considered that the image clarity is insufficient, and it is removed from the sample.

[0099] Step 4.2. Remove irrelevant interference information. Use image segmentation to separate the main body of the device and the area where the defect is located, remove other irrelevant parts, extract the key areas of the device and the defect, and remove background interference, so that the sample focuses on the content related to the truly valuable device defects.

[0100] Step 4.3. Check and correct the annotation accuracy. Check whether the existing image annotation information is accurate. Adopt a method of manual sampling inspection combined with partial automated verification to correct the mislabeled information.

[0101] Step 4.4. Classify according to device type and defect characteristics

[0102] Classify and organize the samples after the previous cleaning steps according to the device type and defect characteristics. This is achieved by establishing a classification rule library. For example, for motor equipment, if the defect characteristic is winding short circuit, it is classified into the motor - short circuit defect category; if it is bearing wear, it is classified into the motor - bearing wear category, etc. The samples under each category jointly constitute a high - quality sample library, which is convenient for subsequent targeted data analysis, model training and other operations based on different devices and defect situations.

[0103] Step 5: Send the cleaned samples to the substation inspection system. Automatically send the cleaned samples to the substation inspection system, which is used for model training and defect identification to improve the utilization efficiency of the samples.

[0104] Step 5.1, Sample preparation. After confirming that the sample data on the cloud platform has completed the cleaning and annotation process, select appropriate sample data according to the requirements of the substation inspection system.

[0105] Step 5.2, Data packaging. Package the selected sample data to ensure that the data will not be lost or damaged during transmission to ensure data security and transmission efficiency.

[0106] Step 5.3, Transmission mechanism. Transmit the file through the File Transfer Protocol SFTP to ensure the stability and security of the network connection during data transmission.

[0107] Step 5.4, Sending process. Start sending the packaged sample data to the substation inspection system, monitor the data transmission process to ensure that all data successfully reaches the destination.

[0108] Step 5.5, Data Verification. After the on-site inspection system receives the data, verify the integrity and correctness of the data. If damaged or missing data is found, the relevant data needs to be re-transmitted.

[0109] Step 5.6, Sample Deployment. Deploy the received sample data to the database or file system of the on-site inspection system for subsequent model training.

[0110] Step 6: Model Training and Optimization. In the on-site inspection system, use the issued samples for model training and optimization. Through deep learning algorithms, continuously improve the recognition accuracy and robustness of the model.

[0111] Step 7: Real-time Monitoring and Warning. Use the trained model to perform defect recognition and warning on the real-time collected device images. Detect device defects in a timely manner and improve the intelligence level of the remote intelligent inspection system.

[0112] Step 8: Sharing and Utilizing Sample Data. Share the collected sample data in the cloud to promote the utilization and exchange of samples. Sample data can be obtained and used through the cloud platform to improve the intelligence level of the inspection system.

[0113] The remote intelligent inspection system and the on-site inspection system are two collaborating systems in the method of automatic sample collection based on artificial intelligence cloud-edge collaboration, jointly constituting a complete monitoring and defect recognition process.

[0114] Among them, the remote intelligent inspection system is deployed at the monitoring site, responsible for real-time collection of device image data, automatically identifying device defects through image recognition technology, and uploading the identified defect images and device information to the artificial intelligence platform in the cloud. The trained and optimized model can be deployed in the remote intelligent inspection system for real-time defect recognition and warning, and continuous real-time monitoring, and upload the newly identified defect data to the cloud.

[0115] The on-site inspection system is responsible for receiving the preliminary recognition results uploaded by the remote intelligent inspection system and performing preprocessing, such as data cleaning, annotation, etc., receiving the cleaned, annotated, and classified sample data from the cloud for local model training and optimization, responsible for model update and maintenance, and handling some complex defect recognition tasks, and making necessary adjustments and optimizations according to real-time data and model feedback to improve monitoring efficiency and accuracy.

[0116] The two systems work together through cloud-edge collaboration to achieve real-time data sharing and continuous model optimization. The on-site inspection system can perform system optimization and function upgrade according to the feedback of the remote intelligent inspection system, while the remote intelligent inspection system can improve the monitoring quality according to the model update of the on-site inspection system. The deployment architecture diagram is as Figure 3 shown.

[0117] The remote intelligent inspection centralized monitoring system interacts horizontally with the new generation of substation integrated control system equipment monitoring system, and vertically runs through the power grid business resource middle platform, unified video platform, artificial intelligence platform, and substation intelligent inspection system. It includes function modules such as information overview, query and statistics, intelligent inspection, three-dimensional inspection, equipment operation and maintenance, video management, and configuration management. It integrates data such as audio and video, digital meters, on-line monitoring, fire protection, and power environment to form an intelligent inspection report, and synchronizes the inspection tasks and results to the power grid resource business middle platform. At the same time, it interacts with the artificial intelligence platform to realize sample annotation, sample upload, and algorithm management. It mainly realizes connection with the provincial artificial intelligence platform to achieve functions such as sample collection and algorithm model distribution. The architecture diagram is as Figure 4 shown.

[0118] The above is only a preferred embodiment of the present invention, and it is not any other form of limitation to the present invention. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope protected by the present invention.

Claims

1. The method for automatic sample collection based on artificial intelligence cloud-edge collaboration is as follows, and is characterized by: Step 1: Deploy a remote intelligent patrol system. Deploy a remote intelligent patrol system on industrial equipment and facilities, which includes high-definition cameras and image recognition technology, and ensure that equipment defects can be clearly captured under various lighting and weather conditions; Step 2: Automatically identify equipment defects. Automatically detect and identify equipment defects through image recognition technology; associate the identified defect images with equipment information, including equipment type, location, time, etc.; Step 3: Upload defect images to the cloud. Upload the defect images and equipment information to the artificial intelligence platform in the cloud; Step 4: Sample cleaning. Perform sample cleaning on the cloud platform; clean and annotate the defect image samples uploaded in Step 3 to ensure sample quality; then classify them according to equipment type and defect characteristics to build a high-quality sample library; Step 5: Send the cleaned samples to the station-side patrol system. Automatically send the cleaned samples to the station-side patrol system, which is used for model training and defect recognition to improve the utilization efficiency of the samples; Step 6: Model training and optimization. In the station-side patrol system, use the sent samples for model training and optimization; through deep learning algorithms, continuously improve the recognition accuracy and robustness of the model; Step 7: Real-time monitoring and early warning. Use the trained model to perform defect recognition and early warning on the real-time collected equipment images; Discover equipment defects in a timely manner and improve the intelligence level of the remote intelligent patrol system; Step 8: Share and utilize sample data. Share the collected sample data in the cloud to promote the utilization and exchange of samples. The sample data can be obtained and used through the cloud platform to improve the intelligence level of the patrol system.

2. The sample automatic collection method based on artificial intelligence cloud-edge collaboration according to claim 1, wherein: The deployment of the remote intelligent patrol system in Step 1 can be expressed as: Step 1.1: Select monitoring equipment. Select network cameras with high resolution and night vision function as monitoring equipment; these cameras can provide clear video streams so that details of the equipment can be captured even in low-light environments; Step 1.2: Install monitoring equipment. Install multiple network cameras at key locations, such as above and around production lines and machinery; use brackets and protective covers to protect the cameras to ensure their stability and prevent accidental damage; at the same time, ensure that the angles and positions of the cameras can cover the areas to be monitored to the greatest extent; Step 1.3: Deploy image recognition technology. Integrate deep learning-based image recognition technology into the remote intelligent patrol system; this technology analyzes a large amount of historical image data to learn how to identify various equipment defects, such as cracks, wear, looseness, etc.; The system uses a pre-trained model and fine-tunes it for specific industrial scenarios to improve the accuracy of recognition; Step 1.4: Device information association. In the remote intelligent inspection system, through the built-in image recognition technology, the identified defect images are associated with the device information; the device information includes the type, location, operating status, etc. of the device, and this information can be obtained from the device management system through the wireless network; through the association, the system can provide detailed background information for each defect image, facilitating subsequent analysis and processing; Step 1.5: Data transmission preparation. In order to upload the identified defect images and device information to the provincial artificial intelligence platform in the cloud, the remote intelligent inspection system needs to configure an appropriate data transmission mechanism; an encrypted VPN connection is set up in the system to ensure the security and stability of the data during transmission; at the same time, according to the interface requirements of the cloud platform, the format and protocol of data upload are configured; Step 1.6: System testing and optimization. After the deployment is completed, the remote intelligent inspection system is comprehensively tested; the testing includes aspects such as the image quality of the camera, the accuracy of image recognition, and the stability of data transmission; by simulating different device defects and environmental conditions, the performance of the system is evaluated.

3. The sample automatic collection method based on artificial intelligence cloud-edge collaboration according to claim 1, characterized in that: The process of automatically identifying device defects in Step 2 can be represented as follows: Step 2.1, Image preprocessing stage. Convert the color image to a grayscale image. The formula for converting a color image to a grayscale image is: I gray (x,y) = 0.299I(x,y,1) + 0.587I(x,y,2) + 0.114I(x,y,3) Among them, I(x, y, c) represents a color image, (x, y) represents the pixel coordinates of the color image, and c = 1, 2, 3 respectively represent red, green, and blue. I gray (x, y) represents a grayscale image; Then perform normalization processing on the grayscale image. The formula for normalizing to the interval [0,1] is: Among them, I norm (x, y) represents the normalized grayscale image; Step 2.2, Feature extraction stage. Use the processed grayscale image as the input of the feature extraction network. Let the input image be X, the convolution kernel be K, and the bias be b. The calculation formula for the output Y of the convolutional layer is: where i and j are the pixel coordinates in the output feature map, and m and n are the coordinates in the convolution kernel; through the stacking of multiple convolutional layers, features at different levels can be extracted; The pooling layer is used to reduce the data dimension and prevent overfitting; taking max pooling as an example, let the input feature map be F, the pooling window size be k×k, and the calculation formula for the output P after pooling is: where i and j are the pixel coordinates of the output after pooling, P is the extracted feature, and m and n are the coordinates during pooling; Step 2.3, Defect classification stage. After feature extraction, the obtained feature vector P is classified using SVM; the decision function of SVM is: y = sign(w·P + b) where y is the image classification result, 1 indicates defective, -1 indicates normal, sign is the sign function, w is the weight vector of SVM, b is the bias of SVM, and P is the extracted feature; in the training stage, the following objective function needs to be minimized: Simultaneously satisfy the constraint condition y i (w·P i +b)≥1, where i = 1, 2, …, n, n is the number of training samples, y i is the class label of the i-th training sample, P i is the feature vector of the i-th training sample; Step 2.4, Associated device information stage. When a device defect is identified, the defect image is associated with the device information; a device information matrix is formed, which includes information such as device type, device location, and time; The association between the identified defect image and the device information is achieved by establishing an association matrix. The elements in the matrix represent the degree of association between the defect image and the device information; the corresponding row elements in the association matrix record the indexes and specific values of the device information type, location, and time for subsequent query and processing.

4. The sample automatic collection method based on artificial intelligence cloud-edge collaboration according to claim 1, wherein: The process of sample cleaning on the cloud platform in Step 4 can be expressed as follows: Step 4.

1. Image quality screening. For the uploaded defective image samples, check the image clarity quality index; define the measurement index of image clarity as the variance S of the image 2 : Among them, x i represents the gray value of the image pixel, represents the average value of the gray values of all the image pixels, and n is the total number of pixels; set the minimum variance threshold T1. When the variance of the image is less than T1, it is considered that the clarity of the image is insufficient, and it is removed from the sample; Step 4.

2. Remove irrelevant interference information. Use image segmentation to separate the main body of the device and the area where the defect is located, remove other irrelevant parts, extract the key areas of the device and the defect, remove background interference, and focus the sample on the content related to the truly valuable device defects; Step 4.

3. Check and correct the annotation accuracy. Check whether the existing image annotation information is accurate; use a combination of manual spot checks and partial automated verification to correct the mislabeled information; Step 4.

4. Classify according to device type and defect characteristics. Classify and organize the samples after the previous cleaning steps according to the device type and defect characteristics; achieve this by establishing a classification rule library. For example, for motor devices, if the defect characteristic is winding short circuit, it is classified into the motor - short circuit defect category; if it is bearing wear, it is classified into the motor - bearing wear category, etc.; the samples under each category together constitute a high - quality sample library, which is convenient for subsequent targeted data analysis, model training, etc. based on different device and defect situations.

5. The sample automatic collection method based on artificial intelligence cloud-edge collaboration according to claim 1, characterized in that: The process of sending the cleaned samples to the substation inspection system in Step 5 can be expressed as follows: Step 5.1, Sample preparation. After confirming that the sample data on the cloud platform has completed the cleaning and annotation process, select the appropriate sample data according to the requirements of the substation inspection system; Step 5.2, Data packaging. Package the selected sample data to ensure that the data will not be lost or damaged during transmission, so as to ensure data security and transmission efficiency; Step 5.3, Transmission mechanism. Transmit the file through the File Transfer Protocol SFTP to ensure the stability and security of the network connection during data transmission; Step 5.4, Sending process. Start sending the packaged sample data to the substation inspection system, monitor the data transmission process, and ensure that all data reaches the destination successfully; Step 5.5, Data verification. After the substation inspection system receives the data, verify the integrity and correctness of the data. If data damage or loss is found, the relevant data needs to be re - transmitted; Step 5.6, Sample deployment. Deploy the received sample data to the database or file system of the substation inspection system for subsequent model training use.

Images