A fault warning system based on intelligent image recognition and its application

Abstract

The present invention provides a fault warning system based on intelligent image recognition and its application, wherein the fault warning system comprises an image acquisition module, an image recognition and classification module, an image processing module and a fault warning module; the image acquisition module is used for acquiring image data when the equipment is in operation, and outputting the image data to the image recognition and classification module; the image recognition and classification module is used for classifying and processing the acquired image data, and outputting images of a specified category to the image processing module; the image processing module is used for processing images of a specified category, and outputting processing results to the fault warning module; the fault warning module is used for analyzing and processing the processing results. The present invention can combine image recognition with equipment fault warning, improve the effectiveness and timeliness of equipment fault monitoring, and solve the problems that the existing image recognition application field is single and equipment monitoring is not convenient and effective enough.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and processing, and in particular to a fault warning system based on intelligent image recognition and application thereof. Background Art

[0002] Image recognition refers to the technology of using computers to process, analyze and understand images to identify targets and objects of various different patterns. It is a practical application of deep learning algorithms. Image recognition may be based on the main features of the image. Each image has its own characteristics, such as the letter A has a point, P has a circle, and the center of Y has an acute angle. Studies on eye movements during image recognition have shown that the line of sight is always focused on the main features of the image, that is, where the image contour has the largest curvature or the contour direction changes suddenly, and these places have the largest amount of information.

[0003] In existing technologies, image recognition is mostly used in the fields of face recognition or product recognition. However, many intelligent and automated equipment also need recognition and monitoring during operation. Existing image monitoring systems are only used to monitor the operation of workshops and the flow of personnel. They can only record videos, and it is difficult to provide effective fault warnings for equipment operation failures. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a fault warning system based on intelligent image recognition and its application, which can combine image recognition with equipment fault warning, improve the effectiveness and timeliness of equipment fault monitoring, and solve the problems of the single application field of existing image recognition and the inconvenient and ineffective equipment monitoring.

[0005] In order to achieve the above object, the present invention is implemented by the following technical solutions: a fault warning system based on intelligent image recognition, the fault warning system includes an image acquisition module, an image recognition and classification module, an image processing module and a fault warning module;

[0006] The image acquisition module is used to acquire image data when the device is operating, and output the image data to the image recognition and classification module;

[0007] The image recognition and classification module is used to classify the acquired image data and output images of a specified category to the image processing module;

[0008] The image processing module is used to process the images of the specified category and output the processing results to the fault warning module;

[0009] The fault warning module is used to analyze and process the processing result, obtain the fault warning result, and output the fault warning result to the maintenance terminal.

[0010] Furthermore, the image acquisition module includes an infrared image acquisition unit and a device image acquisition unit, the infrared image acquisition unit is used to acquire an infrared image of the device when it is in operation, and the device image acquisition unit is used to acquire an image of a warning light when the device is in operation.

[0011] Furthermore, the image recognition and classification module includes a recognition unit and a classification unit; the recognition unit is configured with a recognition strategy, and the recognition strategy includes: recognizing the acquired infrared image, recognizing the infrared thermal imaging area in the image, and respectively obtaining the proportion of infrared thermal imaging areas of different levels;

[0012] Recognize the acquired warning light image of the equipment when it is in operation, and identify the number of warning lights in the image;

[0013] The classification unit is configured with a classification strategy, which includes: classifying the level of infrared imaging according to the brightness of the infrared imaging, and classifying the infrared imaging into first-level images, second-level images, and third-level images in descending order of brightness; outputting the proportion of the area belonging to the first-level image in the acquired infrared image to the image processing module;

[0014] According to the number of warning lights, the warning levels are divided into the first warning level, the second warning level and the third warning level.

[0015] Further, the image processing module includes a preprocessing unit, the preprocessing unit is configured with a preprocessing strategy, the preprocessing strategy includes: substituting the proportion of the area belonging to the first level image in the acquired infrared image and the warning level of the warning light image into a preprocessing formula to obtain a preprocessing reference value;

[0016] When the preprocessing reference value is greater than or equal to the first preprocessing reference threshold, an emergency monitoring signal is then issued;

[0017] When the preprocessing reference value is less than the first preprocessing reference threshold, normal image monitoring continues.

[0018] The preprocessing formula is configured as: Yck = (Td1 × k1) (Jj+k2) ; Among them, Yck is the preprocessing reference value, Td1 is the proportion of the area belonging to the first-level image in the acquired infrared image, Jj is the warning level of the warning light image, k1 is the proportional coefficient of the first-level image, and k2 is the balance index of the warning level.

[0019] Furthermore, the image processing module further includes an emergency monitoring unit, the emergency monitoring unit is configured with an emergency monitoring strategy, the emergency monitoring strategy includes: when an emergency monitoring signal is received, an infrared image and a warning light image of the device in operation are acquired once every first monitoring time interval;

[0020] Then, each time the infrared image and warning light image of the first number are obtained, processing is performed once;

[0021] The proportions of the first-level images, the second-level images and the third-level images in each infrared image in the first number are obtained respectively, and the warning light level of each warning light image in the first number is obtained; and then the obtained data is substituted into the emergency monitoring formula to obtain the emergency monitoring reference value.

[0022] Furthermore, the emergency monitoring formula is configured as:

[0023] ; Wherein, Pjj is the emergency monitoring reference value, T1d represents the proportion of first-level images in each infrared image acquired, T2d represents the proportion of second-level images in each infrared image acquired, T3d represents the proportion of third-level images in each infrared image acquired, m is the number of the first number, a1 is the proportion coefficient of first-level images, a2 is the proportion coefficient of second-level images, and a3 is the proportion coefficient of third-level images.

[0024] Furthermore, the image processing module also includes a key monitoring unit, which is configured with a key monitoring strategy. The key monitoring strategy includes: dividing the image of the equipment operation into key areas, and then acquiring the first-level image ratio of the key areas in the infrared images acquired for the first time, and marking them as the first-level key area ratio, and bringing the first-level key area ratio acquired each time in the first time into the key monitoring formula to obtain the key monitoring value.

[0025] Furthermore, the key monitoring formula is configured as:

[0026] Among them, Pzd is the key monitoring value, Z1b is the first-level key area ratio, and α is the key monitoring ratio coefficient.

[0027] Further, the fault warning module is configured with a fault warning strategy, and the fault warning strategy includes: when the emergency monitoring reference value is greater than or equal to the first warning threshold, outputting a device stop operation signal;

[0028] When the emergency monitoring reference value is greater than the second warning threshold and less than the first warning threshold, a maintenance check signal is output;

[0029] When the emergency monitoring reference value is less than the second warning threshold, a normal monitoring signal is output;

[0030] When the key monitoring value is greater than or equal to the first key monitoring threshold, a device stop operation signal is output;

[0031] When the key monitoring value is greater than or equal to the second key monitoring threshold and less than the first key monitoring threshold, a maintenance check signal is output;

[0032] When the key monitoring value is less than the second key monitoring threshold, a normal monitoring signal is output.

[0033] An application of a fault warning system based on intelligent image recognition, wherein the fault warning system is applied to warning monitoring of large-scale automation equipment in enterprise factories.

[0034] The beneficial effects of the present invention are as follows: the present invention can acquire image data of the equipment during operation through the image acquisition module, and output the image data to the image recognition and classification module, and then the image recognition and classification module is used to classify and process the acquired image data, and output the images of the specified category to the image processing module. Through the classification processing, the amount of tasks for subsequent image processing can be reduced, thereby achieving effective and accurate image processing and monitoring. The image processing module can then process the images of the specified category, and output the processing results to the fault warning module. Finally, the fault warning module can analyze and process the processing results, and obtain the fault warning results, and output the fault warning results to the maintenance terminal. This design can achieve equipment fault monitoring through image recognition processing, and can improve the timeliness and effectiveness of equipment monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0036] Figure 1 A schematic diagram showing the connection between the fault warning system of the present invention and the maintenance terminal;

[0037] Figure 2 It is a module principle block diagram of the present invention. DETAILED DESCRIPTION

[0038] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0039] See also Figure 1 and Figure 2, a fault warning system based on intelligent image recognition, the fault warning system includes an image acquisition module, an image recognition and classification module, an image processing module and a fault warning module; when the fault warning system based on intelligent image recognition is applied in specific applications, the fault warning system can be applied to the warning monitoring of large-scale automation equipment in enterprise factories.

[0040] The image acquisition module is used to acquire image data when the device is in operation, and output the image data to the image recognition and classification module; the image acquisition module includes an infrared image acquisition unit and a device image acquisition unit, the infrared image acquisition unit is used to acquire infrared images when the device is in operation, and the device image acquisition unit is used to acquire warning light images when the device is in operation. The infrared image acquisition unit uses an infrared thermal imager to acquire infrared images, and the device image acquisition unit uses a camera to acquire warning light images when the device is in operation.

[0041] The image recognition and classification module is used to classify the acquired image data and output the images of the specified category to the image processing module; the image recognition and classification module includes a recognition unit and a classification unit; the recognition unit is configured with a recognition strategy, and the recognition strategy includes: recognizing the acquired infrared image, recognizing the infrared thermal imaging area in the image, and respectively obtaining the proportions of the infrared thermal imaging areas of different levels;

[0042] Recognize the acquired warning light image of the equipment when it is in operation, and identify the number of warning lights in the image;

[0043] The classification unit is configured with a classification strategy, which includes: dividing the level of infrared imaging according to the brightness of the infrared imaging, and dividing the infrared imaging into first-level images, second-level images and third-level images in order from high to low brightness; outputting the proportion of the area belonging to the first-level image in the acquired infrared image to the image processing module; the temperatures of the devices represented by the first-level images, the second-level images and the third-level images are also arranged from high to low. In the process of acquiring infrared images, it is necessary to find the best angle for the device to shoot based on multiple experiments, so that each image acquired is acquired at the same angle, and the corresponding position of each component of the device in the picture is also fixed, so as to facilitate the comparison between the later images and the important components of the device.

[0044] According to the number of warning lights, the warning levels are divided into the first warning level, the second warning level and the third warning level.

[0045] The image processing module is used to process the image of the specified category and output the processing result to the fault warning module; the image processing module includes a preprocessing unit, and the preprocessing unit is configured with a preprocessing strategy, and the preprocessing strategy includes: substituting the proportion of the area belonging to the first level image in the acquired infrared image and the warning level of the warning light image into the preprocessing formula to obtain a preprocessing reference value;

[0046] When the preprocessing reference value is greater than or equal to the first preprocessing reference threshold, an emergency monitoring signal is then issued;

[0047] When the preprocessing reference value is less than the first preprocessing reference threshold, normal image monitoring continues.

[0048] The preprocessing formula is configured as: Yck = (Td1 × k1) (Jj+k2) ; Among them, Yck is the preprocessing reference value, Td1 is the proportion of the area belonging to the first-level image in the acquired infrared image, Jj is the warning level of the warning light image, k1 is the proportional coefficient of the first-level image, k2 is the balance index of the warning level, and the settings of k1 and k2 are both greater than zero. k1 and k2 are used to convert the proportion of the area of ​​the first-level image and the warning level of the warning light image to form a numerical value that can be used for mutual reference.

[0049] The image processing module also includes an emergency monitoring unit, which is configured with an emergency monitoring strategy. The emergency monitoring strategy includes: when an emergency monitoring signal is received, an infrared image of the equipment in operation and a warning light image are acquired once every first monitoring period; when emergency monitoring is required, it is indicated that the result of the processing by the preprocessing unit shows that the equipment has operational risks.

[0050] Then, each time the infrared image and warning light image of the first number are obtained, processing is performed once;

[0051] The proportions of the first-level images, the second-level images and the third-level images in each infrared image in the first number are obtained respectively, and the warning light level of each warning light image in the first number is obtained; and then the obtained data is substituted into the emergency monitoring formula to obtain the emergency monitoring reference value.

[0052] The emergency monitoring formula is configured as:

[0053] ;

[0054] Among them, Pjj is the emergency monitoring reference value, T1d represents the proportion of first-level images in each infrared image acquired, T2d represents the proportion of second-level images in each infrared image acquired, T3d represents the proportion of third-level images in each infrared image acquired, m is the number of first times, a1 is the proportion coefficient of first-level images, a1 is set to be greater than 1, if the proportion of first-level images continues to increase, it will have a positive effect on the final emergency monitoring reference value, a2 is the proportion coefficient of second-level images, a3 is the proportion coefficient of third-level images, a2 and a3 are set to be less than zero, if the proportion of second-level images and third-level images gradually decreases, it will have a positive effect on the final emergency monitoring reference value.

[0055] The image processing module also includes a key monitoring unit, and the key monitoring unit is configured with a key monitoring strategy. The key monitoring strategy includes: dividing the image of the equipment operation into key areas, and then obtaining the first-level image ratio of the key area in the infrared image obtained for the first time, and marking it as the first-level key area ratio, and bringing the first-level key area ratio of each of the first times obtained into the key monitoring formula to obtain the key monitoring value. The key monitoring strategy can be run simultaneously with the emergency monitoring strategy. As long as the output result of one of the two requires an alert, a warning signal can be sent to the maintenance terminal. Key monitoring is usually the monitoring of high-speed running parts of the equipment, such as the engine of the equipment or the motor, transmission parts, cutting parts of the equipment, etc. These parts are prone to high temperatures when operating for a long time, and continuous high temperatures or abnormally high temperatures will cause damage to the equipment.

[0056] The key monitoring formula configuration is: Among them, Pzd is the key monitoring value, Z1b is the first-level key area ratio, and α is the key monitoring ratio coefficient, wherein α is set based on the importance of the key area of ​​the equipment in the operation of the entire equipment, and α is set to be greater than zero.

[0057] The fault warning module is used to analyze and process the processing result, obtain the fault warning result, and output the fault warning result to the maintenance terminal.

[0058] The fault warning module is configured with a fault warning strategy, and the fault warning strategy includes: when the emergency monitoring reference value is greater than or equal to the first warning threshold, outputting a device stop operation signal;

[0059] When the emergency monitoring reference value is greater than the second warning threshold and less than the first warning threshold, a maintenance check signal is output;

[0060] When the emergency monitoring reference value is less than the second warning threshold, a normal monitoring signal is output;

[0061] When the key monitoring value is greater than or equal to the first key monitoring threshold, a device stop operation signal is output;

[0062] When the key monitoring value is greater than or equal to the second key monitoring threshold and less than the first key monitoring threshold, a maintenance check signal is output;

[0063] When the key monitoring value is less than the second key monitoring threshold, a normal monitoring signal is output.

[0064] Working principle: The present invention can obtain image data when the equipment is operating through the image acquisition module, and output the image data to the image recognition and classification module, and then use the image recognition and classification module to classify and process the acquired image data, and output the images of the specified category to the image processing module. Through the classification processing, the workload of subsequent image processing can be reduced, thereby achieving effective and accurate image processing monitoring. The image processing module can then process the images of the specified category and output the processing results to the fault warning module. Finally, the fault warning module can analyze and process the processing results, and obtain the fault warning results, which are then output to the maintenance terminal.

[0065] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A fault warning system based on intelligent image recognition, characterized in that: The fault warning system includes an image acquisition module, an image recognition and classification module, an image processing module and a fault warning module; The image acquisition module is used to acquire image data when the device is operating, and output the image data to the image recognition and classification module; The image recognition and classification module is used to classify the acquired image data and output images of a specified category to the image processing module; The image processing module is used to process the images of the specified category and output the processing results to the fault warning module; The fault warning module is used to analyze and process the processing result, obtain the fault warning result, and output the fault warning result to the maintenance terminal; The image acquisition module includes an infrared image acquisition unit and a device image acquisition unit, wherein the infrared image acquisition unit is used to acquire an infrared image of the device when it is in operation, and the device image acquisition unit is used to acquire an image of a warning light when the device is in operation; The image recognition and classification module includes a recognition unit and a classification unit; the recognition unit is configured with a recognition strategy, and the recognition strategy includes: recognizing the acquired infrared image, recognizing the infrared thermal imaging area in the image, and respectively obtaining the proportion of the infrared thermal imaging area of ​​different levels; Recognize the acquired warning light image of the equipment when it is in operation, and identify the number of warning lights in the image; The classification unit is configured with a classification strategy, which includes: classifying the level of infrared imaging according to the brightness of the infrared imaging, and classifying the infrared imaging into first-level images, second-level images, and third-level images in descending order of brightness; outputting the proportion of the area belonging to the first-level image in the acquired infrared image to the image processing module; According to the number of warning lights, they are divided into the first warning level, the second warning level and the third warning level; The image processing module includes a preprocessing unit, and the preprocessing unit is configured with a preprocessing strategy, and the preprocessing strategy includes: substituting the proportion of the area belonging to the first level image in the acquired infrared image and the warning level of the warning light image into the preprocessing formula to obtain a preprocessing reference value; When the preprocessing reference value is greater than or equal to the first preprocessing reference threshold, an emergency monitoring signal is then issued; When the preprocessing reference value is less than the first preprocessing reference threshold, continuing normal image monitoring; The preprocessing formula is configured as: Yck = (Td1 × k1) (Jj+k2) ; Among them, Yck is the preprocessing reference value, Td1 is the proportion of the area belonging to the first-level image in the acquired infrared image, Jj is the warning level of the warning light image, k1 is the proportional coefficient of the first-level image, and k2 is the balance index of the warning level.

2. A fault warning system based on intelligent image recognition according to claim 1, characterized in that: The image processing module further includes an emergency monitoring unit, wherein the emergency monitoring unit is configured with an emergency monitoring strategy, wherein the emergency monitoring strategy includes: when an emergency monitoring signal is received, acquiring an infrared image and a warning light image of the device in operation once every first monitoring time interval; Then, each time the infrared image and warning light image of the first number are obtained, processing is performed once; The proportions of the first-level images, the second-level images and the third-level images in each infrared image in the first number are obtained respectively, and the warning light level of each warning light image in the first number is obtained; and then the obtained data is substituted into the emergency monitoring formula to obtain the emergency monitoring reference value.

3. A fault warning system based on intelligent image recognition according to claim 2, characterized in that: The emergency monitoring formula is configured as: ； Among them, Pjj is the emergency monitoring reference value, T1d represents the proportion of first-level images in each infrared image acquired, T2d represents the proportion of second-level images in each infrared image acquired, T3d represents the proportion of third-level images in each infrared image acquired, m is the total number of acquisitions, a1 is the proportion coefficient of first-level images, a2 is the proportion coefficient of second-level images, and a3 is the proportion coefficient of third-level images.

4. A fault warning system based on intelligent image recognition according to claim 3, characterized in that: The image processing module also includes a key monitoring unit, which is configured with a key monitoring strategy. The key monitoring strategy includes: dividing the image of the equipment operation into key areas, then acquiring the first-level image ratio of the key areas in the infrared images acquired for the first time, and marking them as the first-level key area ratio, and bringing the first-level key area ratio acquired each time in the first time into the key monitoring formula to obtain the key monitoring value.

5. A fault warning system based on intelligent image recognition according to claim 4, characterized in that: The key monitoring formula configuration is: Among them, Pzd is the key monitoring value, Z1b is the first-level key area ratio, and α is the key monitoring ratio coefficient.

6. A fault warning system based on intelligent image recognition according to claim 5, characterized in that: The fault warning module is configured with a fault warning strategy, and the fault warning strategy includes: when the emergency monitoring reference value is greater than or equal to the first warning threshold, outputting a device stop operation signal; When the emergency monitoring reference value is greater than the second warning threshold and less than the first warning threshold, a maintenance check signal is output; When the emergency monitoring reference value is less than the second warning threshold, a normal monitoring signal is output; When the key monitoring value is greater than or equal to the first key monitoring threshold, a device stop operation signal is output; When the key monitoring value is greater than or equal to the second key monitoring threshold and less than the first key monitoring threshold, a maintenance check signal is output; When the key monitoring value is less than the second key monitoring threshold, a normal monitoring signal is output.

7. The application of a fault warning system based on intelligent image recognition according to any one of claims 1 to 6, characterized in that: The fault warning system is applied to the warning monitoring of large-scale automation equipment in enterprise factories.

Images