Rotation machine equipment condition monitoring system and monitoring method based on image recognition

Abstract

The invention belongs to the technical field of rotating machine equipment monitoring, and discloses a rotating machine equipment condition monitoring system and method based on image recognition, and the method comprises the steps: collecting the video data of each visual angle of rotating machine equipment; marking an identification area in the video data of each visual angle of the rotating machine equipment, the identification area being an area where the rotating machine equipment is located; the method comprises the following steps of: acquiring each frame of picture in an identification area of video data, performing gray processing on the frame picture, acquiring a gray value of each pixel block in the frame picture, marking the gray value as a real-time gray value, acquiring a gray value of each pixel block in a preset standard picture, marking the gray value as a standard gray value, and marking the standard gray value as a real-time gray value; comparing and analyzing the real-time gray value of each block in the frame picture and a standard gray value at the same position in the standard picture, judging whether the pixel blocks are subjected to abnormal marking or not, and obtaining the number of abnormal marks; according to the invention, the maintenance processing efficiency of the abnormal rotating machine equipment by workers can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of transfer equipment monitoring, and more particularly to a transfer equipment condition monitoring system and method based on image recognition. Background Art

[0002] Patent publication number CN118521556A discloses a computer room equipment status monitoring method and system based on image recognition. The method includes: collecting images of the equipment's operating status and preprocessing them; comparing the processed images with pre-stored reference photos to identify changes in the cabinet's U-shaped status, equipment status changes, and equipment labels; and outputting the identified photos and equipment label information. The invention uses the SIFT algorithm to identify the recognition points of the image to be tested, compares it to a standard image, and determines whether the equipment's operating status is abnormal based on the Euclidean distance between the key feature vectors in the two images.

[0003] The existing technology still has the following problems:

[0004] When the abnormality type is identified, it is not possible to output warning instructions of different hazard levels according to the type of abnormality, and the corresponding processing methods of warning instructions of different hazard levels are not output, which reduces the efficiency of staff in maintaining and processing abnormal transfer equipment.

[0005] In view of this, the present invention proposes a transfer equipment condition monitoring system and monitoring method based on image recognition to solve the above problems. Summary of the Invention

[0006] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a transfer equipment condition monitoring system and monitoring method based on image recognition.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The method for monitoring the condition of transfer equipment based on image recognition includes:

[0009] Collect video data from all viewing angles of the transfer equipment;

[0010] In the video data of each viewing angle of the transfer device, an identification area is marked, and the identification area is the area where the transfer device is located;

[0011] Obtain each frame of the video data in the recognition area, perform grayscale processing on the frame, obtain the grayscale value of each pixel block in the frame and mark it as a real-time grayscale value, obtain the grayscale value of each pixel block in the preset standard picture and mark it as a standard grayscale value, compare and analyze the real-time grayscale value of each block in the frame with the standard grayscale value of the same position in the standard picture, determine whether to mark the pixel block as abnormal, obtain the number of abnormal marks, and determine whether to generate an abnormal instruction based on the number of abnormal marks;

[0012] Extracting pixel blocks with abnormal marks, and mapping positions of the pixel blocks with abnormal marks in the frame image to a blank background layer to obtain an extracted pattern composed of pixel blocks with abnormal marks;

[0013] Input the extracted pattern into the trained anomaly recognition model to obtain the anomaly type;

[0014] According to the abnormality type and the number of abnormally marked pixel blocks added per unit time for each abnormality type, early warning instructions of different hazard levels are generated.

[0015] Furthermore, the method for determining whether to mark a pixel block as abnormal includes:

[0016] If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is greater than the preset difference threshold, the corresponding pixel block is marked as abnormal;

[0017] If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is less than or equal to the preset difference threshold, the corresponding pixel block will not be marked as abnormal.

[0018] Furthermore, the method for determining whether an abnormal instruction is generated includes:

[0019] If the number of exception tags is less than or equal to the exception number threshold, no exception instruction is generated; if the number of exception tags is greater than the exception number threshold, an exception instruction is generated.

[0020] Furthermore, the training method of the anomaly recognition model includes:

[0021] Collect historical extracted patterns in advance, the extracted patterns are patterns corresponding to the anomaly types, and labels are set for the anomaly types of the extracted patterns. The extracted patterns and the labels corresponding to the extracted patterns constitute a set of training data, i groups of training data constitute a sample set, i is an integer greater than 1, and the sample set is divided into a training set and a test set; the extracted patterns in the training set are used as the input of the anomaly recognition model, the labels in the training set are used as the output of the anomaly recognition model, the anomaly recognition model is trained to obtain an initial anomaly recognition model, minimizing the sum of prediction accuracies is the training goal, the initial anomaly recognition model is evaluated using the test set, and the initial anomaly recognition model when the sum of prediction accuracies reaches convergence is used as the constructed anomaly recognition model.

[0022] Further, the anomaly types include flames, lubricating oil leakage, and deposit accumulation.

[0023] Furthermore, the method for obtaining the number of pixel blocks with abnormal markings added per unit time for each abnormal type is as follows;

[0024] The frame corresponding to the abnormal instruction generation time corresponding to each abnormal type is taken as the starting frame, the frame corresponding to the end node of the unit time is taken as the ending frame, and the number of pixel blocks with abnormal marks corresponding to the ending frame is subtracted from the number of pixel blocks with abnormal marks corresponding to the starting frame to obtain the unit change number.

[0025] Furthermore, the warning instructions of different hazard levels include a first-level rising instruction, a first-level falling instruction, a second-level rising instruction, a second-level falling instruction, and a third-level instruction; and the generation method of the first-level rising instruction, the first-level falling instruction, the second-level rising instruction, the second-level falling instruction, and the third-level instruction includes:

[0026] When the abnormality type is flame and the unit change quantity is greater than or equal to 0, a first-level rising instruction is generated;

[0027] When the abnormal type is flame and the unit change quantity is less than 0, a first-level descending instruction is generated. The processing priority of the first-level descending instruction is lower than that of the first-level ascending instruction.

[0028] When the abnormality type is lubricating oil leakage and the unit change quantity is greater than or equal to 0, a second-level rising instruction is generated. The processing priority of the second-level rising instruction is lower than that of the first-level falling instruction.

[0029] When the abnormality type is lubricating oil leakage and the unit change quantity is less than 0, a secondary down instruction is generated. The secondary down instruction has a lower processing priority than the secondary up instruction.

[0030] When the anomaly type is sediment accumulation and the unit change number is greater than or equal to 0, a third-level instruction is generated, and the processing priority of the third-level instruction is lower than the second-level descent instruction;

[0031] When the anomaly type is sediment accumulation and the unit change number is less than 0, it means that there is sediment in the identification area that is not continuously accumulated, and no third-level instruction is generated.

[0032] Furthermore, the historical extraction patterns collected in advance also include line noses. When the abnormality type is a line nose, the real-time coordinates of the edge pixel points of the extraction pattern corresponding to the line nose in the identification area are obtained, and the real-time coordinates are compared and analyzed with the preset coordinates to determine whether to generate an offset instruction. If an offset instruction is generated, the corresponding transfer equipment will be warned; the identification area is a rectangle, and a coordinate system is established with any corner point of the rectangle as the origin. The preset coordinates are the coordinates of all edge pixel points at the initial installation position of the line nose.

[0033] Furthermore, the method for determining whether to generate an offset instruction includes:

[0034] If any real-time coordinates are inconsistent with the coordinates in the preset coordinates, an offset instruction is generated; if they are consistent, no offset instruction is generated;

[0035] The priority of the offset instruction processing is between the second-level descending instruction and the third-level instruction. The offset instruction also includes an offset first-level instruction and an offset second-level instruction. The generation method of the offset first-level instruction and the offset second-level instruction includes:

[0036] Obtain the distance between the real-time coordinate with the largest coordinate value and the preset coordinate with the largest coordinate value. If the distance is greater than or equal to the preset distance, generate a first-level offset instruction.

[0037] If the spacing is smaller than the preset spacing, a second-level offset instruction is generated, and the processing priority of the second-level offset instruction is lower than that of the first-level offset instruction.

[0038] A transfer equipment condition monitoring system based on image recognition, used to implement a transfer equipment condition monitoring method based on image recognition, the system comprising:

[0039] Data acquisition module, collecting video data from all viewing angles of the transfer equipment;

[0040] The area marking module marks the identification area in the video data of each viewing angle of the transfer device. The identification area is the area where the transfer device is located;

[0041] A video analysis module obtains each frame in the recognition area of ​​the video data, performs grayscale processing on the frame, obtains the grayscale value of each pixel block in the frame and marks it as a real-time grayscale value, obtains the grayscale value of each pixel block in a preset standard image and marks it as a standard grayscale value, compares and analyzes the real-time grayscale value of each pixel block in the frame with the standard grayscale value of the same position in the standard image, determines whether to mark the pixel block as abnormal, obtains the number of abnormal marks, and determines whether to generate an abnormal instruction based on the number of abnormal marks;

[0042] The extraction module extracts the pixel blocks with abnormal marks, and maps the positions of the pixel blocks with abnormal marks in the frame image to the blank background layer to obtain an extracted pattern composed of the pixel blocks with abnormal marks;

[0043] The pattern recognition module inputs the extracted pattern into the trained anomaly recognition model to obtain the anomaly type;

[0044] The first warning module generates warning instructions of different hazard levels according to the abnormality type and the number of pixel blocks with abnormal marks added per unit time for each abnormality type.

[0045] The technical effects and advantages of the transfer equipment condition monitoring system and monitoring method based on image recognition of the present invention are as follows:

[0046] By collecting video data from multiple angles of the transfer equipment, the recognition area is marked in the video data to reduce the processing load of subsequent image recognition. Anomaly markers are generated by changing the real-time grayscale value with the standard grayscale value. Preliminary anomaly generation is completed based on the anomaly markers and the number of anomaly markers. Pixel blocks with anomaly markers are then extracted, and the positions of the pixel blocks with anomaly markers in the frame are mapped to a blank background layer to obtain a clear extracted pattern composed of pixel blocks with anomaly markers. The extracted pattern is identified to obtain the anomaly type. Based on the type of anomaly, anomalies formed in the early stages can be discovered and handled in a timely manner. Different hazard level warning instructions are output based on the type of anomaly. Based on the corresponding handling methods of the warning instructions with different hazard levels, the efficiency of maintenance and handling of abnormal transfer equipment by staff is improved. The handling priority is determined based on the warning instructions with different hazard levels, thereby minimizing the losses caused by transfer equipment anomalies. Secondly, the warning instructions with different hazard levels can be input into the transfer equipment control system, and the transfer equipment control system will operate accordingly based on the warning instructions with different hazard levels, further reducing the losses caused by transfer equipment anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of a transfer equipment condition monitoring system based on image recognition in Example 1 of the present invention;

[0048] Figure 2 Schematic diagram of a transfer equipment condition monitoring system based on image recognition in Example 2 of the present invention;

[0049] Figure 3 Schematic diagram of a transfer equipment condition monitoring system based on image recognition in Example 3 of the present invention;

[0050] Figure 4 Schematic diagram of a method for monitoring the condition of transfer equipment based on image recognition in Example 4 of the present invention;

[0051] Figure 5 Schematic diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] Example 1

[0054] See also Figure 1 As shown, the transfer equipment condition monitoring system based on image recognition described in this embodiment includes a data acquisition module, an area labeling module, a video analysis module, an extraction module, a pattern recognition module and a first warning module, and each module is connected through a wired and / or wireless network.

[0055] The data acquisition module is used to collect video data from various viewing angles of the transfer equipment, specifically through multiple high-definition cameras set up around the transfer equipment;

[0056] The viewing angle includes the front, back, left, right and top. If the transfer device is installed in the air, the viewing angle also includes the bottom. If the transfer device is installed against the wall, the viewing angle does not include the back. The viewing angle is determined by technical personnel based on the actual installation position of the transfer device.

[0057] The purpose of collecting video data from multiple angles is that video data collected from a single angle is not comprehensive enough. If a transfer equipment fails, the location of the failure is not unique. Collecting video data from multiple angles can avoid the failure location being in the blind spot of a single angle, resulting in the inability to detect the failure or potential hidden danger in a timely manner.

[0058] The area marking module is used to mark the identification area in the video data of each viewing angle of the transfer device. The identification area is the area where the transfer device is located.

[0059] The video analysis module is used to obtain each frame in the recognition area of ​​the video data, perform grayscale processing on the frame, obtain the grayscale value of each pixel block in the frame, and mark it as a real-time grayscale value, and obtain the grayscale value of each pixel block in the preset standard picture, and mark it as a standard grayscale value, compare and analyze the real-time grayscale value of each block in the frame with the standard grayscale value of the same position in the standard picture, determine whether to mark the pixel block as abnormal, obtain the number of abnormal marks, and determine whether to generate an abnormal instruction based on the number of abnormal marks.

[0060] The method for determining whether to mark a pixel block as abnormal includes:

[0061] If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is greater than the preset difference threshold, the corresponding pixel block is marked as abnormal; if the absolute value of the difference between the real-time grayscale value and the standard grayscale value is less than or equal to the preset difference threshold, the corresponding pixel block is not marked as abnormal.

[0062] Methods for determining whether an abnormal instruction is generated include:

[0063] If the number of abnormal marks is less than or equal to the abnormal number threshold, no abnormal instruction is generated; if the number of abnormal marks is greater than the abnormal number threshold, an abnormal instruction is generated; the average of the difference between the real-time grayscale value and the standard grayscale value in multiple frames with known abnormal types is used as the preset difference threshold; the abnormal number threshold is set in the same way.

[0064] The extraction module is used to extract pixel blocks with abnormal marks, map the positions of the pixel blocks with abnormal marks in the frame image to the blank background layer to obtain a clear extracted pattern composed of pixel blocks with abnormal marks, and send the extracted pattern to the pattern recognition module.

[0065] The pattern recognition module inputs the extracted pattern into the trained anomaly recognition model to obtain the anomaly type.

[0066] The training methods for the anomaly recognition model include:

[0067] Collect historical extracted patterns in advance, where the extracted patterns are patterns corresponding to the extracted anomaly types, and set labels for the anomaly types of the extracted patterns. The extracted patterns and the labels corresponding to the extracted patterns constitute a set of training data, i groups of training data constitute a sample set, i is an integer greater than 1, and the sample set is divided into a training set and a test set. The extracted patterns in the training set are used as the input of the anomaly recognition model, and the labels in the training set are used as the output of the anomaly recognition model. The anomaly recognition model is trained to obtain an initial anomaly recognition model, and the minimization of the sum of prediction accuracies is used as the training goal. The initial anomaly recognition model is evaluated using the test set, and the initial anomaly recognition model when the sum of prediction accuracies reaches convergence is used as the constructed anomaly recognition model. The calculation formula for prediction accuracy is: Zg = (ag - wg) 2 , where g is the number of each group of training data, Zg is the prediction accuracy, ag is the predicted value of the label corresponding to the g-th group of training data, and wg is the actual value of the label corresponding to the g-th group of training data; the anomaly recognition model can be a convolutional neural network model, a transfer learning model, or other suitable machine learning model, which is not specifically limited here.

[0068] During the collection phase, staff can set tags based on the type of anomaly, including flames, lubricant leaks, and sediment accumulation.

[0069] Flames such as fire in transfer equipment or insulation breakdown can cause cable damage, loss of protection of the cable insulation layer, and easy leakage of electricity; sediment accumulation such as dust accumulation at the cable connection or on the surface of the equipment will lead to degradation of its insulation performance, partial discharge and other faults, affecting heat dissipation and corroding transfer parts; lubricating oil leakage is mainly caused by the leakage of lubricating oil in the equipment due to poor sealing or material aging, which reduces its insulation performance. At the same time, the lubricating oil may also contaminate the insulating oil, affecting the insulation and heat dissipation functions of the insulating oil.

[0070] The first warning module generates warning instructions of different degrees of harm according to the type of abnormality and the number of pixel blocks with abnormal marks added per unit time for each abnormality type, and then sends the warning instructions of different degrees of harm to the staff information receiving terminal for display, so that the staff can understand the type of abnormality and the degree of harm, choose the order of priority processing, and reduce losses.

[0071] The method for obtaining the number of pixel blocks with abnormal marks added per unit time for each abnormal type is as follows;

[0072] The frame corresponding to the abnormal instruction generation time corresponding to each abnormal type is taken as the starting frame, and the frame corresponding to the end node of the unit time is taken as the ending frame. The number of pixel blocks marked with abnormalities corresponding to the ending frame is subtracted from the number of pixel blocks marked with abnormalities corresponding to the starting frame to obtain the unit change quantity. According to the unit change quantity, it can be judged whether the degree of harm of each abnormal type is on an upward or downward trend. The unit time is specifically set according to the shooting frequency of the high-definition camera. The higher the shooting frequency, the shorter the unit time, and vice versa.

[0073] Warning instructions of different hazard levels include a first-level rising instruction, a first-level falling instruction, a second-level rising instruction, a second-level falling instruction, and a third-level instruction; and methods for generating the first-level rising instruction, the first-level falling instruction, the second-level rising instruction, the second-level falling instruction, and the third-level instruction include:

[0074] When the abnormality type is flame and the unit change number is greater than or equal to 0, a first-level rising instruction is generated, indicating that it is in a diffusion combustion state. If not handled in time, the combustion will spread to the surrounding transfer equipment, which will increase the combustion loss. It is necessary to power off the combustion transfer equipment and its surrounding transfer equipment.

[0075] When the abnormality type is flame and the unit change number is less than 0, a first-level descending instruction is generated, indicating that the combustion is controlled at this time. The processing priority of the first-level descending instruction is lower than that of the first-level ascending instruction, and the combustion converter equipment can be powered off.

[0076] When the abnormality type is lubricating oil leakage and the unit change quantity is greater than or equal to 0, a secondary rising instruction is generated, indicating that the leakage is in a diffusion state at this time. The circuit of the transfer equipment can be cut off without affecting the surrounding transfer equipment. The processing priority of the secondary rising instruction is lower than that of the primary descending instruction.

[0077] When the abnormality type is lubricating oil leakage and the unit change quantity is less than 0, a secondary descent instruction is generated, indicating that the leakage is under control or is intermittent, and the transfer equipment circuit does not need to be cut off. The processing priority of the secondary descent instruction is lower than that of the secondary rise instruction.

[0078] When the anomaly type is sediment accumulation and the unit change number is greater than or equal to 0, a third-level instruction is generated, indicating that sediment is continuously accumulating in the identification area. For example, birds are building nests in the identification area or dust is continuously accumulating. The nest building materials may contain filamentous or flaky metal, which will cause the risk of short circuit to transfer equipment or other transfer equipment. Dust accumulation will reduce the insulation performance between exposed metal joints because the insulation decreases after the dust absorbs water. The processing priority of the third-level instruction is lower than the second-level descent instruction.

[0079] When the anomaly type is sediment accumulation and the unit change number is less than 0, no level 3 instruction is generated, indicating that there is non-continuous sediment accumulation in the identified area and no processing is required for the time being, and monitoring can be continued.

[0080] This embodiment collects video data from multiple angles of the transfer equipment and then marks the identified areas in the video data, reducing the processing load of subsequent image recognition. Abnormality markers are generated by changing the real-time grayscale value with the standard grayscale value. Preliminary abnormality generation is completed based on the abnormality markers and the number of abnormality markers. Pixel blocks with abnormality markers are then extracted and their positions in the frame are mapped onto a blank background layer to obtain a clear extracted pattern composed of pixel blocks with abnormality markers. The extracted pattern is identified to obtain the abnormality type. Based on the abnormality type, early abnormalities can be detected and handled promptly. Warning instructions of varying severity are output based on the abnormality type. Based on the corresponding handling methods for the warning instructions of varying severity, the efficiency of maintenance and handling of abnormal transfer equipment by staff is improved. Processing priorities are determined based on the warning instructions of varying severity, minimizing losses caused by abnormalities in the transfer equipment. Furthermore, the warning instructions of varying severity can be input into the transfer equipment control system, which then operates accordingly, further reducing losses caused by abnormalities in the transfer equipment.

[0081] Example 2

[0082] In the actual monitoring process, warning instructions of different hazard levels are generated by marking pixel blocks with abnormal marks in the identification area and according to the number of unit changes. It is difficult to cover all types of abnormalities. For example, if the wire nose is loose, its position will shift. When it shakes under the action of external force, sparks will be generated, which may cause a fire hazard. Although its position is shifted, it is still in the identification area. The grayscale value of each pixel block in the frame has not changed. Similarly, the number of unit changes has not changed.

[0083] To resolve the issue of incomplete exception type coverage, see Figure 2 As shown, this embodiment provides a transfer equipment condition monitoring system based on image recognition, and also includes a second early warning module, which pre-collects historical extraction patterns including line noses. When the abnormality type is a line nose, the second early warning module obtains the real-time coordinates of the edge pixel points of the extraction pattern corresponding to the line nose in the identification area, compares and analyzes the real-time coordinates with the preset coordinates, and determines whether to generate an offset instruction. If an offset instruction is generated, the corresponding transfer equipment will be warned; the identification area is a rectangle, and a coordinate system is established with any corner point of the rectangle as the origin, and the preset coordinates are the coordinates of all edge pixel points at the initial installation position of the line nose.

[0084] Methods for determining whether to generate an offset instruction include:

[0085] If any real-time coordinate is inconsistent with the coordinate in the preset coordinate, an offset instruction is generated; if they are consistent, no offset instruction is generated.

[0086] The priority of the offset instruction processing is between the second-level descending instruction and the third-level instruction. The offset instruction also includes an offset first-level instruction and an offset second-level instruction. The generation method of the offset first-level instruction and the offset second-level instruction includes:

[0087] Get the distance between the real-time coordinate with the largest coordinate value and the preset coordinate with the largest coordinate value. If the distance is greater than or equal to the preset distance, generate a first-level offset instruction, indicating that the wire nose offset is large at this time, the corresponding shaking amplitude is also large, the degree of poor contact is also greater, and the sparks generated during shaking are also greater; the preset distance setting method is: set different degrees of looseness of the wire nose, and when shaking it with the same external force, the shaking amplitude change distance that generates sparks is used as the preset distance.

[0088] If the distance is smaller than the preset distance, a secondary offset instruction is generated, indicating that the shaking amplitude is within a safe range. The secondary offset instruction has a lower processing priority than the primary offset instruction.

[0089] Example 3

[0090] See also Figure 3As shown, this embodiment further improves the design based on Example 1. The difference is that this embodiment provides a transfer equipment condition monitoring system based on image recognition, and also includes a picture adjustment module. The picture adjustment module is used to immediately adjust the frame picture shooting environment light to be consistent with the standard picture shooting environment light after obtaining each frame picture in the recognition area of ​​the video data.

[0091] The method of adjusting to be consistent includes adjusting the color and brightness in the frame image to be consistent with the preset color and preset brightness respectively; this can be done through image processing software, and the preset color and preset brightness are the color and brightness of the standard image respectively.

[0092] The picture adjustment module can eliminate the error in pixel value caused by the inconsistency between the ambient light of the frame picture shooting and the ambient light of the standard picture shooting to the greatest extent, thereby improving the accuracy of the warning.

[0093] Example 4

[0094] See also Figure 4 As shown, for the parts not described in detail in this embodiment, please refer to the description of Example 1. A method for monitoring the status of transfer equipment based on image recognition is provided, including:

[0095] Collect video data from all viewing angles of the transfer equipment;

[0096] In the video data of each viewing angle of the transfer device, an identification area is marked, and the identification area is the area where the transfer device is located;

[0097] Obtain each frame of the video data in the recognition area, perform grayscale processing on the frame, obtain the grayscale value of each pixel block in the frame and mark it as a real-time grayscale value, obtain the grayscale value of each pixel block in the preset standard picture and mark it as a standard grayscale value, compare and analyze the real-time grayscale value of each block in the frame with the standard grayscale value of the same position in the standard picture, determine whether to mark the pixel block as abnormal, obtain the number of abnormal marks, and determine whether to generate an abnormal instruction based on the number of abnormal marks;

[0098] Extracting pixel blocks with abnormal marks, and mapping positions of the pixel blocks with abnormal marks in the frame image to a blank background layer to obtain an extracted pattern composed of pixel blocks with abnormal marks;

[0099] Input the extracted pattern into the trained anomaly recognition model to obtain the anomaly type;

[0100] According to the abnormality type and the number of abnormally marked pixel blocks added per unit time for each abnormality type, early warning instructions of different hazard levels are generated.

[0101] Furthermore, the method for determining whether to mark a pixel block as abnormal includes:

[0102] If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is greater than the preset difference threshold, the corresponding pixel block is marked as abnormal;

[0103] If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is less than or equal to the preset difference threshold, the corresponding pixel block will not be marked as abnormal.

[0104] Furthermore, the method for determining whether an abnormal instruction is generated includes:

[0105] If the number of exception tags is less than or equal to the exception number threshold, no exception instruction is generated; if the number of exception tags is greater than the exception number threshold, an exception instruction is generated.

[0106] Furthermore, the training method of the anomaly recognition model includes:

[0107] Collect historical extracted patterns in advance, the extracted patterns are patterns corresponding to the anomaly types, and labels are set for the anomaly types of the extracted patterns. The extracted patterns and the labels corresponding to the extracted patterns constitute a set of training data, i groups of training data constitute a sample set, i is an integer greater than 1, and the sample set is divided into a training set and a test set; the extracted patterns in the training set are used as the input of the anomaly recognition model, the labels in the training set are used as the output of the anomaly recognition model, the anomaly recognition model is trained to obtain an initial anomaly recognition model, minimizing the sum of prediction accuracies is the training goal, the initial anomaly recognition model is evaluated using the test set, and the initial anomaly recognition model when the sum of prediction accuracies reaches convergence is used as the constructed anomaly recognition model.

[0108] Further, the anomaly types include flames, lubricating oil leakage, and deposit accumulation.

[0109] Furthermore, the method for obtaining the number of pixel blocks with abnormal markings added per unit time for each abnormal type is as follows;

[0110] The frame corresponding to the abnormal instruction generation time corresponding to each abnormal type is taken as the starting frame, the frame corresponding to the end node of the unit time is taken as the ending frame, and the number of pixel blocks with abnormal marks corresponding to the ending frame is subtracted from the number of pixel blocks with abnormal marks corresponding to the starting frame to obtain the unit change number.

[0111] Furthermore, the warning instructions of different hazard levels include a first-level rising instruction, a first-level falling instruction, a second-level rising instruction, a second-level falling instruction, and a third-level instruction; and the generation method of the first-level rising instruction, the first-level falling instruction, the second-level rising instruction, the second-level falling instruction, and the third-level instruction includes:

[0112] When the abnormality type is flame and the unit change quantity is greater than or equal to 0, a first-level rising instruction is generated;

[0113] When the abnormal type is flame and the unit change quantity is less than 0, a first-level descending instruction is generated. The processing priority of the first-level descending instruction is lower than that of the first-level ascending instruction.

[0114] When the abnormality type is lubricating oil leakage and the unit change quantity is greater than or equal to 0, a second-level rising instruction is generated. The processing priority of the second-level rising instruction is lower than that of the first-level falling instruction.

[0115] When the abnormality type is lubricating oil leakage and the unit change quantity is less than 0, a secondary down instruction is generated. The secondary down instruction has a lower processing priority than the secondary up instruction.

[0116] When the anomaly type is sediment accumulation and the unit change number is greater than or equal to 0, a third-level instruction is generated, and the processing priority of the third-level instruction is lower than the second-level descent instruction;

[0117] When the anomaly type is sediment accumulation and the unit change number is less than 0, it means that there is sediment in the identification area that is not continuously accumulated, and no third-level instruction is generated.

[0118] Furthermore, the historical extraction patterns collected in advance also include line noses. When the abnormality type is a line nose, the real-time coordinates of the edge pixel points of the extraction pattern corresponding to the line nose in the identification area are obtained, and the real-time coordinates are compared and analyzed with the preset coordinates to determine whether to generate an offset instruction. If an offset instruction is generated, the corresponding transfer equipment will be warned; the identification area is a rectangle, and a coordinate system is established with any corner point of the rectangle as the origin. The preset coordinates are the coordinates of all edge pixel points at the initial installation position of the line nose.

[0119] Furthermore, the method for determining whether to generate an offset instruction includes:

[0120] If any real-time coordinates are inconsistent with the coordinates in the preset coordinates, an offset instruction is generated; if they are consistent, no offset instruction is generated;

[0121] The priority of the offset instruction processing is between the second-level descending instruction and the third-level instruction. The offset instruction also includes an offset first-level instruction and an offset second-level instruction. The generation method of the offset first-level instruction and the offset second-level instruction includes:

[0122] Obtain the distance between the real-time coordinate with the largest coordinate value and the preset coordinate with the largest coordinate value. If the distance is greater than or equal to the preset distance, generate a first-level offset instruction.

[0123] If the spacing is smaller than the preset spacing, a second-level offset instruction is generated, and the processing priority of the second-level offset instruction is lower than that of the first-level offset instruction.

[0124] Example 5

[0125] See also Figure 5As shown, this embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned transfer equipment condition monitoring method based on image recognition is implemented.

[0126] Since the electronic device described in this embodiment is an electronic device used to implement the transfer equipment condition monitoring method based on image recognition in the embodiment of this application, based on the transfer equipment condition monitoring method based on image recognition described in the embodiment of this application, those skilled in the art can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application will not be described in detail here. As long as those skilled in the art implement the electronic device used by the transfer equipment condition monitoring method based on image recognition in the embodiment of this application, it falls within the scope of protection to be provided by this application.

[0127] Example 6

[0128] This embodiment discloses a computer-readable storage medium having a rewritable computer program stored thereon;

[0129] When the computer program is executed on a computer device, the computer device is enabled to execute the above-mentioned transfer equipment condition monitoring method based on image recognition.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0131] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for monitoring the condition of transfer equipment based on image recognition, characterized in that: include: Collect video data from all viewing angles of the transfer equipment; In the video data of each viewing angle of the transfer device, an identification area is marked, and the identification area is the area where the transfer device is located; Obtain each frame of the video data in the recognition area, perform grayscale processing on the frame, obtain the grayscale value of each pixel block in the frame and mark it as a real-time grayscale value, obtain the grayscale value of each pixel block in the preset standard picture and mark it as a standard grayscale value, compare and analyze the real-time grayscale value of each block in the frame with the standard grayscale value of the same position in the standard picture, determine whether to mark the pixel block as abnormal, obtain the number of abnormal marks, and determine whether to generate an abnormal instruction based on the number of abnormal marks; Extracting pixel blocks with abnormal marks, and mapping positions of the pixel blocks with abnormal marks in the frame image to a blank background layer to obtain an extracted pattern composed of pixel blocks with abnormal marks; Input the extracted pattern into the trained anomaly recognition model to obtain the anomaly type; According to the abnormality type and the number of abnormally marked pixel blocks added per unit time for each abnormality type, early warning instructions of different hazard levels are generated.

2. The method for monitoring the condition of transfer equipment based on image recognition according to claim 1, characterized in that: The method for determining whether to mark a pixel block as abnormal includes: If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is greater than the preset difference threshold, the corresponding pixel block is marked as abnormal; If the absolute value of the difference between the real-time grayscale value and the standard grayscale value is less than or equal to the preset difference threshold, the corresponding pixel block will not be marked as abnormal.

3. The method for monitoring the condition of transfer equipment based on image recognition according to claim 2, characterized in that: Methods for determining whether an abnormal instruction is generated include: If the number of exception tags is less than or equal to the exception number threshold, no exception instruction is generated; if the number of exception tags is greater than the exception number threshold, an exception instruction is generated.

4. The method for monitoring the condition of transfer equipment based on image recognition according to claim 3, characterized in that: The training methods for the anomaly recognition model include: Collect historical extracted patterns in advance, the extracted patterns are patterns corresponding to the anomaly types, and labels are set for the anomaly types of the extracted patterns. The extracted patterns and the labels corresponding to the extracted patterns constitute a set of training data, i groups of training data constitute a sample set, i is an integer greater than 1, and the sample set is divided into a training set and a test set; the extracted patterns in the training set are used as the input of the anomaly recognition model, the labels in the training set are used as the output of the anomaly recognition model, the anomaly recognition model is trained to obtain an initial anomaly recognition model, minimizing the sum of prediction accuracies is the training goal, the initial anomaly recognition model is evaluated using the test set, and the initial anomaly recognition model when the sum of prediction accuracies reaches convergence is used as the constructed anomaly recognition model.

5. The method for monitoring the condition of transfer equipment based on image recognition according to claim 4, characterized in that: Types of anomalies include flames, lubricating oil leaks, and deposit buildup.

6. The method for monitoring the condition of transfer equipment based on image recognition according to claim 5, characterized in that: The method for obtaining the number of pixel blocks marked with abnormality added per unit time for each abnormality type includes: The frame corresponding to the abnormal instruction generation time corresponding to each abnormal type is taken as the starting frame, the frame corresponding to the end node of the unit time is taken as the ending frame, and the number of pixel blocks with abnormal marks corresponding to the ending frame is subtracted from the number of pixel blocks with abnormal marks corresponding to the starting frame to obtain the unit change number.

7. The method for monitoring the condition of transfer equipment based on image recognition according to claim 6, characterized in that: Warning instructions of different hazard levels include a first-level rising instruction, a first-level falling instruction, a second-level rising instruction, a second-level falling instruction, and a third-level instruction; and methods for generating the first-level rising instruction, the first-level falling instruction, the second-level rising instruction, the second-level falling instruction, and the third-level instruction include: When the abnormality type is flame and the unit change quantity is greater than or equal to 0, a first-level rising instruction is generated; When the abnormal type is flame and the unit change quantity is less than 0, a first-level descending instruction is generated. The processing priority of the first-level descending instruction is lower than that of the first-level ascending instruction. When the abnormality type is lubricating oil leakage and the unit change quantity is greater than or equal to 0, a second-level rising instruction is generated. The processing priority of the second-level rising instruction is lower than that of the first-level falling instruction. When the abnormality type is lubricating oil leakage and the unit change quantity is less than 0, a secondary down instruction is generated. The secondary down instruction has a lower processing priority than the secondary up instruction. When the anomaly type is sediment accumulation and the unit change number is greater than or equal to 0, a third-level instruction is generated, and the processing priority of the third-level instruction is lower than the second-level descent instruction; When the anomaly type is sediment accumulation and the unit change number is less than 0, it means that there is sediment in the identification area that is not continuously accumulated, and no third-level instruction is generated.

8. The method for monitoring the condition of transfer equipment based on image recognition according to claim 7, characterized in that: The historical extraction patterns collected in advance also include line noses. When the abnormality type is a line nose, the real-time coordinates of the edge pixel points of the extraction pattern corresponding to the line nose in the identification area are obtained, and the real-time coordinates are compared and analyzed with the preset coordinates to determine whether to generate an offset instruction. If an offset instruction is generated, the corresponding transfer equipment will be warned; the identification area is a rectangle, and a coordinate system is established with any corner point of the rectangle as the origin. The preset coordinates are the coordinates of all edge pixel points at the initial installation position of the line nose.

9. The method for monitoring the condition of transfer equipment based on image recognition according to claim 8, characterized in that: Methods for determining whether to generate an offset instruction include: If any real-time coordinates are inconsistent with the coordinates in the preset coordinates, an offset instruction is generated; if they are consistent, no offset instruction is generated; The priority of the offset instruction processing is between the second-level descending instruction and the third-level instruction. The offset instruction also includes an offset first-level instruction and an offset second-level instruction. The generation method of the offset first-level instruction and the offset second-level instruction includes: Obtain the distance between the real-time coordinate with the largest coordinate value and the preset coordinate with the largest coordinate value. If the distance is greater than or equal to the preset distance, generate a first-level offset instruction. If the spacing is smaller than the preset spacing, a second-level offset instruction is generated, and the processing priority of the second-level offset instruction is lower than that of the first-level offset instruction.

10. The transfer equipment condition monitoring system based on image recognition is characterized by: A system for implementing the transfer equipment condition monitoring method based on image recognition according to any one of claims 1 to 9, comprising: Data acquisition module, collecting video data from all viewing angles of the transfer equipment; The area marking module marks the identification area in the video data of each viewing angle of the transfer device. The identification area is the area where the transfer device is located; A video analysis module obtains each frame in the recognition area of ​​the video data, performs grayscale processing on the frame, obtains the grayscale value of each pixel block in the frame and marks it as a real-time grayscale value, obtains the grayscale value of each pixel block in a preset standard image and marks it as a standard grayscale value, compares and analyzes the real-time grayscale value of each pixel block in the frame with the standard grayscale value of the same position in the standard image, determines whether to mark the pixel block as abnormal, obtains the number of abnormal marks, and determines whether to generate an abnormal instruction based on the number of abnormal marks; The extraction module extracts the pixel blocks with abnormal marks, and maps the positions of the pixel blocks with abnormal marks in the frame image to the blank background layer to obtain an extracted pattern composed of the pixel blocks with abnormal marks; The pattern recognition module inputs the extracted pattern into the trained anomaly recognition model to obtain the anomaly type; The first warning module generates warning instructions of different hazard levels according to the abnormality type and the number of pixel blocks with abnormal marks added per unit time for each abnormality type.

Images