Speed reducer oil leakage detection method based on machine vision

Abstract

A speed reducer oil leakage detection method based on machine vision is mainly composed of four parts including an oil spot display device, engine oil detection test paper and a base. The first camera is an image acquisition unit, the industrial camera is used for shooting and acquiring conditions of a speed reducer and test paper under the speed reducer, the third camera is an image processing unit integrating an image preprocessing module and a yolov8 model, and the early warning judgment unit is used for judging whether the oil spot area change exceeds a preset threshold value or not by comparing the change conditions of oil spots in continuously acquired images. And early warning is triggered through a sound-light alarm device. Laying oil leakage detection test paper at the bottom of the speed reducer, acquiring an image of an area with the test paper in real time by using an industrial-grade camera, transmitting the image to an image processing unit for preprocessing, and performing image segmentation on oil spots to obtain position and area information of the oil spots; whether the preset early warning threshold value is reached or not is judged according to the change condition of the oil spot area in the continuous image, so that the real-time monitoring and early warning of the oil spot change are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of oil leakage detection, and relates to a method for detecting oil leakage of a speed reducer based on machine vision. Background Art

[0002] The maintenance of mechanical equipment has become particularly important, especially for key components such as speed reducers. During the operation of a speed reducer, oil leakage may occur due to various reasons. This not only causes waste of lubricating oil but may also lead to equipment failures and even safety accidents. Therefore, studying effective methods for detecting oil leakage of speed reducers has become an important topic.

[0003] Traditional methods for detecting oil leakage of speed reducers mainly include regular inspections and empirical judgments. These methods usually rely on the skills and experience of operators, and visually observe the appearance of the speed reducer to determine whether there is oil leakage. However, this method has many limitations, such as strong subjectivity, low efficiency, and it is easy to miss initial and minor oil leakage signs. In recent years, with the progress of sensor technology and machine vision technology, various automated detection methods have emerged to improve the efficiency and accuracy of oil leakage detection of speed reducers.

[0004] Currently, traditional image processing techniques still play a role in oil leakage detection. For example, in the petrochemical industry, by using machine vision and digital image processing techniques, the oil leakage situation in cooling water can be effectively detected. This method usually includes converting the original image of the on-site water body into a 256-color grayscale image to highlight the edges of oil droplets and performing necessary image enhancement. Subsequently, the edges of the oil droplets are depicted by an edge detection operator, and the object and the background are segmented to form a binary image with distinct contrast. The segmented binary image has obvious features such as area and perimeter, and can achieve the purpose of recognition through simple within-class discrimination. In addition, the entire image can be scanned by template matching to judge the similarity, and a multi-recognizer integration method can be used to improve the reliability of recognition.

[0005] With the development of deep learning technology, oil leakage detection methods based on neural networks have become a research hotspot. These methods usually involve improving existing models to enhance detection accuracy and efficiency. For example, the improved YOLOv5 model adds a 4-fold sampling layer on the basis of the Feature Pyramid Network (FPN) structure to enable cross-layer fusion of feature information and improve the detection accuracy of the model. In addition, the Wise-IoU bounding box loss function with a dynamic non-monotonic focusing mechanism is introduced to accelerate the training and inference of the network, and the overall performance of the model is further improved by weighing the learning of low-quality samples and high-quality samples. Finally, by using the EfficientViT model as the backbone network, although a small part of the detection performance is sacrificed, it still maintains performance superior to the original model, and at the same time significantly reduces the number of parameters of the model, which is beneficial for engineering deployment.

[0006] In addition to image processing and deep learning methods, sensor monitoring technology has also been widely used in oil spill detection. For example, for the oil spill detection of a scraper conveyor reducer, oil storage tanks can be designed outside the seals at the input and output ends of the reducer to introduce the leaked oil from the seals into the oil storage holes at a certain height from the bottom of the oil storage tank, and a monitoring pressure sensor can be installed at the bottom of the oil storage hole to monitor the change of the oil pressure inside the oil storage hole. This technology realizes the data interaction between the detection result and the PLC controller through wireless transmission, can realize the real-time monitoring of the oil spill situation of the reducer, timely discover the oil spill problem, and avoid equipment damage caused by oil spill.

[0007] At present, the research on the oil spill detection of reducers is relatively less. Among the three detection methods mentioned above, the traditional method based on image processing is often very sensitive to lighting conditions and background changes, and is easily interfered by environmental factors, resulting in unstable detection accuracy. And the false detection rate is relatively high. In a complex environment, non-oily objects are easily misjudged as oily, or real oil stains are missed. The detection method based on sensors is easily affected by environmental changes. For example, changes in factors such as temperature and humidity will affect the detection accuracy. There is also the problem of low reliability, because the sensor may give false alarms or missed alarms due to aging or other reasons. In addition, some sensors are only applicable to specific environmental conditions and have poor adaptability to complex and changeable environments. The original deep learning methods often have problems such as large data requirements and high consumption of computing resources, which may be difficult to obtain for some application scenarios in specific fields. And the instability of the image quality of the training set of deep learning also has a great impact on the model prediction accuracy, posing a great challenge to the recognition accuracy in a complex industrial environment such as the oil spill of a reducer. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method for detecting oil spill of a reducer based on machine vision, which can monitor the real-time change of the oil stain on the test paper at the bottom of the reducer, timely discover the abnormal oil stain caused by the oil spill of the reducer and send out a warning signal to prevent potential environmental or safety risks.

[0009] To solve the above technical problem, the technical solution adopted by the present invention is: a method for detecting oil spill of a reducer based on machine vision, including the following steps: S1, hardware selection, select a machine vision system composed of a light source, a lens, a camera, an image collection device, an image analysis device and an interaction software; S2, image acquisition, install two cameras above the reducer, and there is no interference between the cameras and other moving mechanisms in the application environment, and the field of view is clear and unobstructed; collect from the front whether there is engine oil dripping and oil stain generation on the bottom detection test paper, and assist in collecting images from the rear to detect whether the change of the oil stain area is abnormal; S3, Grayscale processing, using grayscale processing based on color features to improve the quality of the acquired images; S4, Image filtering, using Gaussian filtering for image preprocessing to remove noise while retaining the edges and other important features of the image; S5, Image mean normalization, using preprocessing techniques to reduce the mean of the image data, making the image features have zero mean and unit variance, thereby accelerating the training process and improving the convergence speed of the model; S6, Image segmentation, using yolov8 to classify each pixel in the image to identify specific object instances; S7, Oil spot extraction, after extracting the mask of the oil spot area from the output of YOLOv8, using the findContours function in OpenCV to detect the contour of the oil spot area and using the contourArea function to calculate the area of the contour; S8, Image preprocessing, performing a series of preprocessing to reduce the interference of various influencing factors; including grayscale processing based on color features, image filtering, and image mean normalization; S9, Feature recognition and post-processing, using the object detection and instance segmentation model of YOLOv8 to provide fast and accurate predictions, identifying the object categories and positions in the image, and generating a segmentation mask for each object, that is, a pixel-level label; including loading the YOLOv8 model, image detection, extracting the image mask, contour detection, and oil spot area extraction; S10, Oil leakage anomaly alarm, using the yolov8 model for image segmentation and then obtaining the area change information of the image through opencv, and transmitting the detected visual information to the industrial control computer and controlling the alarm to alarm through the Siemens PLC.

[0010] In S1, the factor for lens selection is the focal length. Assuming the size of the field of view to be , with the horizontal and vertical directions being and , and the horizontal and vertical sizes of the camera chip target surface being and , the detection distance being , then the focal length The calculation formula is: ; .

[0011] In S3, the grayscale processing includes the following steps: S3-1, Color space conversion, converting the original image from the RGB color space to another color space, including HSV hue, saturation, value or Lab luminance, a channel, b channel; S3-2, Color feature extraction. In the converted color space, extract the channels related to specific color features; S3-3, Color threshold processing. Set a threshold according to the extracted color features to divide the pixel points in the image into foreground and background; S3-4, Binarization processing. Perform binarization processing on the image after color threshold processing to obtain a binary image, where the oil stain area is the foreground and the rest of the area is the background; S3-5, Post-processing. Perform morphological operations on the binary image to remove noise or thin the contour; perform connected component analysis on the binary image to extract the oil stain area.

[0012] In S4, image filtering includes the following steps: S4-1, Select the Gaussian kernel size. Select an appropriate Gaussian kernel size according to the type of noise to be removed; S4-2, Construct the Gaussian kernel. Construct a two-dimensional Gaussian kernel, which is a discrete approximation of a two-dimensional Gaussian function; S4-3, Normalize the Gaussian kernel. To ensure that the sum of the filter is 1, normalize the Gaussian kernel; calculate the sum of all elements, and then divide each element by this sum; S4-4, Image convolution. Use the normalized Gaussian kernel to perform a convolution operation on the image. For each pixel in the image, multiply the Gaussian kernel by the values of the pixel and its surrounding pixels, and sum to obtain a new pixel value.

[0013] In S5, image mean normalization includes the following steps: S5-1, Calculate the mean file. Calculate the average image using all the images in the training set, that is, calculate the average value of the pixel values of all training images at each pixel position; S5-2, Subtract the mean. For each input image, subtract the corresponding mean value from each pixel position; S5-3, Standardize. To further improve the convergence speed of the model, choose to perform standardization processing on the image after subtracting the mean, that is, divide by the standard deviation or a fixed value to make the features have unit variance.

[0014] In S6, image segmentation includes the following steps: S6-1, Input the image. Send the input image into the YOLOv8 network; S6-2, Feature extraction. Extract multi-scale feature maps of the image through the backbone network and the neck network; S6-3, Segmentation prediction. Perform predictions on the feature maps. Each prediction unit corresponds to a position in the image. For instance segmentation, each prediction unit outputs a class prediction and a segmentation mask; S6-4, Post-processing, perform NMS (Non-Maximum Suppression) on the prediction results to remove duplicate detection results and select the final detection results according to the confidence threshold.

[0015] In S7, the contourArea function is used to calculate the area of a given contour. This function returns the area of the region enclosed by the contour, and the specific steps are as follows: S7-1, YOLOv8 instance segmentation output. First, the YOLOv8 model segments the input image and generates a segmentation mask for each detected oil spot. S7-2, Extract the mask. Extract the segmentation mask of each oil spot from the output of YOLOv8. S7-3, Contour detection. Use the findContours function in OpenCV to detect the contours in each oil spot mask. S7-4, Calculate the area. Use the contourArea function to calculate the area of each contour.

[0016] In S8, image mean normalization adjusts the image data by subtracting a mean vector to make the input data have zero mean. It is performed in the preprocessing stage of the input data of the deep learning model. By mean normalization, the redundancy in the training data is reduced, making it easier for the model to converge. The core calculation process includes the following steps: S8-1, Calculate the mean vector. Set a two-dimensional array M to represent the pixel value matrix of all images in the training set. M is a four-dimensional tensor, and its dimensions are N, H, W, and C respectively. Here, N is the number of images, H is the height of the image, W is the width of the image, and C is the number of channels of the image. For grayscale images, C = 1; for RGB images, C = 3. Then calculate the mean vector μ. S8-2, Subtract the mean. For a new input image I, convert it to a three-dimensional array. For each pixel position (i, j, c), subtract the corresponding mean. S8-3, To further improve the convergence speed of the model, choose to perform normalization on the image after subtracting the mean, that is, divide by the standard deviation or a fixed value to make the features have unit variance.

[0017] In S9, based on the segmentation mask, the findContours function in OpenCV is used to detect the contours within each mask; contour detection helps to understand the specific shape and distribution of the oil spots. Contours are a series of continuous boundary points that enclose a closed shape. For the segmentation mask, the contour refers to the boundary line of the oil spot; the specific steps are as follows: S9-1, Convert the mask to a binary image. If the segmentation mask is not given in the form of a binary image, first convert it to a black-and-white binary image. S9-2, Find the contours. Use the findContours function to find the boundaries formed by all non-zero pixels in the image. S9-3, Contour sorting. If there are multiple contours, sort them according to area size or other criteria to select the largest contour as the main target. S9-4, Contour drawing. Draw the detected contours on the original image to visually display the segmentation result.

[0018] In S10, the oil leakage anomaly alarm specifically includes the following steps: S10-1, Image acquisition and processing. First, use OpenCV to obtain real-time images from the camera, and perform image processing and target detection; preprocess the images, including grayscale conversion, filtering, and contrast enhancement, to improve the detection accuracy; finally, apply the YOLOv8 image segmentation model to analyze the images and identify and segment the oil stain areas. S10-2, Data transmission. Establish a TCP / IP connection through Python's socket library, and send the detection results, including the position coordinates and area of the oil stain, from the industrial control computer to the PLC. S10-3, PLC program design. Configure the PLC in TIAPortal to ensure that the Ethernet communication module of the PLC can receive data from the industrial control computer. Write the PLC program using the TCP / IP communication library in TIAPortal to receive and parse the coordinate and area data sent by the industrial control computer; write the logic in the PLC to parse the received data and determine whether to trigger the alarm condition according to the detection results, including the oil stain area exceeding the set threshold. S10-4, Alarm system control. When the PLC detects that the oil stain area exceeds the predetermined threshold, trigger the alarm signal; the PLC controls the external alarm through a relay or output module.

[0019] The main beneficial effects of the present invention are as follows: Judge whether the preset warning threshold is reached by the change of the oil stain area in consecutive images. Once the change of the oil stain area exceeds the threshold, the system will trigger a warning signal and execute corresponding emergency measures through the control system, so as to realize the real-time monitoring and warning of the change of the oil stain.

[0020] An end-to-end complete system is established, which can realize the automation of the full detection process; the advantage of YOLOv8 in the segmentation task makes it one of the ideal choices for dealing with image segmentation problems.

[0021] It can not only provide high-precision segmentation results, but also has characteristics such as fast inference speed, simple model structure, and wide applicability. Using the oil stain on the test paper after oil leakage as the visual detection object is non-invasive. By checking the oil stain left on the test paper, it is possible to avoid disassembling the reducer for inspection, which not only saves time and cost, but also reduces the possible additional damage to the equipment. And this method only requires simply placing the test paper and checking it regularly, without the need for complex tools or technologies.

[0022] The method of using a test paper to detect oil leakage and then performing visual recognition designed in the present invention is a relatively economical method. The cost of the test paper is low, and there is no need for expensive detection equipment. And by regularly checking the test paper, information on whether there is oil leakage can be obtained quickly, so as to take measures in a timely manner. Compared with directly detecting the reducer itself through vision, detecting oil leakage through the test paper has advantages such as easy training of the model and high detection accuracy. Brief Description of the Drawings

[0023] The present invention will be further described below in conjunction with the drawings and embodiments.

[0024] Figure 1 It is the hardware block diagram of the present invention.

[0025] Figure 2 It is the camera layout diagram of the present invention.

[0026] Figure 3 It is the flow chart of the present invention. Detailed Embodiment

[0027] As Figures 1 to 3 in, a method for detecting oil leakage of a reducer based on machine vision includes the following steps: S1, Hardware selection, select a machine vision system composed of a light source, a lens, a camera, an image acquisition device, an image analysis device, and an interactive software; S2, Image acquisition, set up two cameras above the reducer. The cameras have no interference with other moving mechanisms in the application environment, and the field of view is clear and unobstructed; collect from the front whether there is oil dripping and oil stain generation on the bottom detection test paper, and assist in collecting images from the rear to detect whether the change in the oil stain area is abnormal; S3, Grayscale processing, adopt grayscale processing based on color features to improve the quality of the acquired images; S4, Image filtering, perform image preprocessing using Gaussian filtering to remove noise while retaining the edges and other important features of the image; S5, Image equalization, adopt a preprocessing technique to reduce the mean value of the image data, making the image features have zero mean and unit variance, thereby accelerating the training process and improving the convergence speed of the model; S6, Image segmentation, using YOLOv8 to classify each pixel in the image to identify specific object instances; S7, Oil stain extraction, after extracting the mask of the oil stain area from the output of YOLOv8, use the findContours function in OpenCV to detect the contour of the oil stain area, and use the contourArea function to calculate the area of the contour; S8, Image preprocessing, perform a series of preprocessing to reduce the interference of various influencing factors; including grayscale processing based on color features, image filtering, and image equalization; S9, Feature recognition and post-processing, using the object detection and instance segmentation models of YOLOv8 to provide fast and accurate predictions, identify the object categories and positions in the image, and generate a segmentation mask for each object, that is, a pixel-level label; including loading the YOLOv8 model, image detection, extracting the image mask, contour detection, and oil stain area extraction; S10, Oil leakage anomaly alarm, use the YOLOv8 model for image segmentation and then obtain the area change information of the image through OpenCV. The detected visual information is transmitted to the industrial control computer and the alarm is controlled by the Siemens PLC. In the above method, it is used for the real-time monitoring of the oil stain change on the test paper at the bottom of the reducer, aiming to timely detect the abnormal oil stain caused by the oil leakage of the reducer and send out a warning signal to prevent potential environmental or safety risks. The present invention mainly consists of four parts. One part is the oil stain display device, mainly composed of an oil detection test paper and a base. The second is the image acquisition unit, with the main body being an industrial camera that captures and collects the situation of the reducer and the test paper below it. The third is the image processing unit integrated with the image preprocessing module and the YOLOv8 model. Finally, it is the warning determination unit. By comparing the changes in the oil stain in continuously acquired images, when the change in the oil stain area exceeds the preset threshold, it is determined as abnormal and a warning is triggered through the sound and light alarm device.

[0028] Lay an oil leakage detection test paper at the bottom of the reducer, and then use an industrial camera to continuously collect images of the area where the test paper is placed. Transmit the images to the image processing unit for preprocessing, and then use the YOLOv8 deep learning model to perform image segmentation on the oil stain. Then, obtain the position and area information of the oil stain through post-processing. Determine whether the preset warning threshold is reached based on the change in the oil stain area in consecutive images. Once the change in the oil stain area exceeds the threshold, the system will trigger a warning signal and execute corresponding emergency measures through the control system, thereby realizing the real-time monitoring and warning of the oil stain change.

[0029] Example 1 As Figure 1In it, machine vision imitates the human visual system through an optical system and an image processing device, obtains and processes signals in the acquired target image, thereby obtaining the information required by the target, then classifies and evaluates the detected target, and transmits the final conclusion to the hardware device to guide the next step of the device's work. It can be seen from this that a complete machine vision system is composed of multiple devices, generally mainly divided into optics, including light sources, lenses and cameras, image collection devices, image analysis devices and interactive software, etc.

[0030] (1) Light source and lighting method In the entire machine vision system, the light source is a decisive factor in the imaging quality of the camera. First of all, the light source can not only illuminate the target object, but also increase the contrast between the target object and the interference object, further highlighting the characteristics of the object to be measured to achieve the best effect; if the object to be measured can be clearly separated from the background, the complexity of the image algorithm and development can be reduced, and the overall detection speed can be shortened.

[0031] For the lighting method of the light source, the commonly used ones at present mainly include front lighting, low-angle oblique lighting, diffuse reflection lighting, backlighting and lighting based on coaxial light sources. The principle of front lighting is to detect the surface of the object through the reflected light of the object to be measured, and it is generally used to detect objects with non-reflective surfaces. Low-angle oblique lighting is generally used to detect objects with uneven surfaces, concavities and convexities, and obvious texture changes. Diffuse reflection lighting uses a spherical integral light source to irradiate the object surface with light at different angles and is used to detect curved objects. Backlighting can separate the object edge from the background and is suitable for detecting phenomena such as the contour and transparency of the object.

[0032] The coaxial light source detects the object through uniform direct light and is suitable for detecting objects with strong reflectivity.

[0033] This project will conduct test research according to the on-site use environment and finally select the light source with the best environmental adaptability as the light source used in this project.

[0034] (2) Selection of camera and lens Industrial cameras are mainly used to obtain image information. According to the image sensor, they can be divided into many types. Among them, CMOS cameras have the advantages of high integration, low power consumption, clearer output images and low cost. Therefore, considering speed, image effect and cost factors, CMOS cameras are selected. In addition, for the parameter selection of industrial cameras, the following key factors are mainly considered: 1) Resolution of the camera The resolution is calculated by the following formula: Resolution in a single direction = Field of view in a single direction / Detection accuracy; In the detection scenario of this project, the minimum resolution requires 2 to 3 times the length and width. Only in this way can the collected features be more obvious, facilitating subsequent detection. Therefore, a CMOS industrial camera with 20 million pixels is selected. For the camera's transmission method, the USB3.0 interface form is chosen. The final determined camera parameters are as follows in the table:

[0035] 2) Selection of the lens Regarding the selection of the lens, the main factor considered is the focal length. Assume the size of the field of view to be , with the horizontal and vertical directions being and respectively, the horizontal and vertical sizes of the camera chip's target surface being and respectively, and the detection distance being . Then the calculation formulas for the focal length are as follows in the formulas respectively.

[0036]

[0037]

[0038] After comparison, an industrial zoom lens from Hikvision is finally selected to collect images. Its specific parameters are as follows in the table:

[0039] 3) Image processing platform According to the requirements of the detection system and the actual on-site situation, the selected image processing platform is an industrial control computer. The performance parameters of this industrial control computer are shown in the following table.

[0040]

[0041] Example 2, As in Figure 2 , two cameras are installed above the reducer. It is required that there is no interference with other moving mechanisms in the application environment, and the field of view is clear and unobstructed. The device mainly consists of two industrial cameras, an image processing unit, an early warning device, and an oil leakage detection test paper.

[0042] The first camera collects from the front whether there is oil dripping on the bottom detection test paper and the generation of oil spots; the second camera assists in collecting images from the rear to avoid visual dead angles and detect whether the change in the oil spot area is abnormal.

[0043] Decide whether to install an artificial light source and the type of light source according to the on-site environment.

[0044] The specific working principle and working process are as follows: (1) The industrial camera collects images of the reducer oil seal area at regular intervals or according to a trigger signal.

[0045] (2) Use OpenCV to preprocess the collected images, including grayscaling, filtering, etc., to improve the image quality.

[0046] (3) Adopt object detection algorithms such as YOLOv8 to identify oil spot features in the images.

[0047] (4) Compare the area changes of the oil spots in continuously collected images to determine whether the preset threshold is exceeded. (5) When the area change of the oil spot exceeds the threshold, the system triggers a warning signal.

[0048] (6) The control system executes emergency measures according to the warning signal, such as stopping the production line, starting the oil absorption equipment, etc.

[0049] The following specifications need to be followed for this oil leakage visual recognition warning system: (1) Operator training: All operators must undergo professional training to understand the operation process and safety specifications of the system.

[0050] (2) Regular maintenance: Regularly maintain the system, including cleaning the camera lens, checking the line connections, updating the software, etc.

[0051] (3) Emergency stop button: Set an emergency stop button near the system to quickly stop the system operation in case of an emergency.

[0052] (4) Data backup: Regularly back up the system data to prevent data loss or damage.

[0053] (5) Permission management: Conduct strict permission management on the system, and only authorized personnel can access and modify the system settings.

[0054] (6) Security audit: Conduct regular security audits to check whether there are security hazards in the system and take corresponding measures to eliminate the hazards.

[0055] (7) Emergency plan: Develop an emergency plan, including emergency evacuation routes, emergency contact lists, etc., to ensure a quick response in case of an accident.

[0056] (8) Software upgrade: Regularly update the software version to ensure that the system uses the latest security patches and functional improvements. (9) Hardware inspection: Regularly check the operating status of hardware devices, such as industrial cameras, PLCs, etc., to ensure their normal operation.

[0057] (10) System calibration: Regularly conduct system calibration, including camera calibration, sensor calibration, etc., to maintain the accuracy and reliability of the system.

[0058] Example 3 Algorithm Selection and Basic Principle (1) Grayscale Processing Based on Color Features The invention uses grayscale processing based on color features to improve the quality of the acquired images. Grayscale processing based on color features is an image preprocessing technique mainly used to highlight specific color features in images. This method is particularly suitable for application scenarios that require highlighting a specific color area, such as the oil spot detection application scenario of the present invention.

[0059] The basic idea of grayscale processing based on color features is to extract specific color areas through color space conversion and color threshold processing and convert them into grayscale images. The purpose of doing this is to highlight specific color features for subsequent image processing tasks such as segmentation and detection.

[0060] The specific operation steps are as follows: (1) Color Space Conversion: Convert the original image from the RGB color space to another color space, such as the HSV (Hue, Saturation, Value) or Lab (Luminance, a-channel, b-channel). The HSV and Lab color spaces are usually more suitable for processing color-related information.

[0061] (2) Color Feature Extraction: In the converted color space, extract the channels related to specific color features. For example, in the HSV color space, the V (Value) channel can be used to process grayscale images, or the H (Hue) and S (Saturation) channels can be used to extract information within a specific color range. In the Lab color space, the L (Luminance) channel can be used for grayscale processing, or the a and b channels can be used to extract specific color information.

[0062] (3) Color Threshold Processing: Set a threshold according to the extracted color features to divide the pixel points in the image into foreground and background. For example, for oil spot detection, a threshold can be set according to the color features of the oil spot, such as a dark color, to mark the oil spot area.

[0063] (4) Binarization Processing: Perform binarization processing on the image after color threshold processing to obtain a binary image, where the oil spot area is the foreground, usually white, and the remaining area is the background, usually black.

[0064] (5) Post-Processing: Morphological operations such as dilation and erosion can be performed on the binary image to remove noise or refine the contour. Perform connected component analysis on the binary image, such as extracting the oil spot area in the present invention.

[0065] (2) Gaussian Image Filtering The present invention uses Gaussian filtering for image preprocessing. Gaussian filtering is a linear smoothing filter, commonly used in image processing to remove noise while trying to retain the edges and other important features of the image. The Gaussian filter is implemented by convolving the image with a Gaussian kernel. This kernel has the property that the center is the largest and it gradually decreases towards the edges, which can effectively smooth the image and reduce high-frequency noise. The basic steps for its implementation are as follows: (1) Select the Gaussian kernel size: Select an appropriate Gaussian kernel size according to the type of noise to be removed. A smaller kernel can remove fine noise, while a larger kernel can remove larger noise, but may also result in more detail loss.

[0066] (2) Construct the Gaussian kernel: Construct a two-dimensional Gaussian kernel, which is a discrete approximation of a two-dimensional Gaussian function. The size of the kernel is usually odd, such as 3x3, 5x5, etc. (3) Normalize the Gaussian kernel: To ensure that the sum of the filter is 1, the Gaussian kernel needs to be normalized. Calculate the sum of all elements, and then divide each element by this sum.

[0067] (4) Image convolution: Perform a convolution operation on the image using the normalized Gaussian kernel. For each pixel in the image, multiply the Gaussian kernel with the values of this pixel and its surrounding pixels, and sum them to obtain the new pixel value. The final result of the convolution is a smoothed image with noise removed.

[0068] (III) Image mean normalization, Image mean normalization is a preprocessing technique used to reduce the mean of image data, making the image features have zero mean and unit variance, thereby accelerating the training process and improving the convergence speed of the model. Mean normalization is usually used in the input data preprocessing stage of deep learning models.

[0069] The specific implementation steps are as follows: (1) Calculate the mean file: Calculate the average image using all the images in the training set, that is, calculate the average value of the pixel values of all training images at each pixel position.

[0070] (2) Subtract the mean: For each input image, subtract the corresponding mean value from each pixel position. This can be achieved by pixel-by-pixel subtraction.

[0071] (3) Standardization (optional): To further improve the convergence speed of the model, it is possible to choose to perform standardization on the image after subtracting the mean, that is, divide by the standard deviation or a fixed value, so that the features have unit variance.

[0072] (IV) yolov8 image segmentation, YOLOv8, namely You Only Look Once version 8, is one of the latest versions of the YOLO series of object detection algorithms. It not only supports object detection but also extends functions such as instance segmentation and semantic segmentation. In YOLOv8, image segmentation refers to classifying each pixel point in an image to identify specific object instances. YOLOv8 is developed based on YOLOv7. By introducing some improvements and technological updates, it has improved detection accuracy and efficiency. The design focus of YOLOv8 is on simplifying the architecture and improving flexibility, making it a very powerful and general object detection framework. YOLOv8 supports multiple tasks, including: object detection, instance segmentation, semantic segmentation, pose estimation, etc. The instance segmentation used in this invention aims to identify and segment each individual object instance in an image. The instance segmentation model of YOLOv8 generates a bounding box and a segmentation mask for each detected object to represent the exact contour of the object.

[0073] The specific implementation steps are as follows: (1) Input image: Send the input image into the YOLOv8 network.

[0074] (2) Feature extraction: Extract multi-scale feature maps of the image through the backbone network and the neck network.

[0075] (3) Segmentation prediction: Make predictions on the feature map. Each prediction unit corresponds to a position in the image. For instance segmentation, each prediction unit outputs a class prediction and a segmentation mask.

[0076] (4) Post-processing: Perform NMS (Non-Maximum Suppression) processing on the prediction results to remove duplicate detection results and screen out the final detection results according to the confidence threshold.

[0077] (5) Oil stain area extraction, After extracting the mask of the oil stain area from the output of YOLOv8, use the findContours function of OpenCV to detect the contour of the oil stain area and use the contourArea function to calculate the area of the contour.

[0078] The findContours function is used to find contours from a binary image. It can identify the contours composed of all boundary pixels and return a list of these contours The contourArea function is used to calculate the area of a given contour. This function returns the area of the region enclosed by the contour.

[0079] The following are the specific steps: (1)YOLOv8 instance segmentation output: First, the YOLOv8 model segments the input image and generates a segmentation mask for each detected oil stain.

[0080] (2)Extract the mask: Extract the segmentation mask of each oil stain from the output of YOLOv8.

[0081] (3)Contour detection: Use the findContours function in OpenCV to detect the contours in each oil stain mask.

[0082] (4)Calculate the area: Use the contourArea function to calculate the area of each contour.

[0083] Example 4 Image preprocessing: In the process of image processing, the extraction of image feature information is greatly affected by image quality. In actual image acquisition, due to a series of factors such as light, vibration, and noise, the collected image data cannot be directly detected and requires a series of preprocessing to minimize the interference of various influencing factors.

[0084] Gray processing based on color features: The color image data collected by the camera is usually based on the RGB color space, that is, the color is expressed through three components: R:Red, G:Green, and B:Blue. Each component channel is generally composed of 8 bits, and the value range is 0~255. In traditional digital image processing, various feature detection algorithms are basically designed based on grayscale images. To improve the subsequent feature recognition accuracy, it is necessary to select an appropriate gray transformation method to transform the three-channel color RGB image into a single-channel grayscale image with the same value range. Set the gray value at pixel point P( x, y ) to Gray( x, y ). The common gray transformation methods are as follows: (1)Maximum gray processing method: Take the component with the largest value in the RGB three channels as the gray value of the corresponding pixel point after gray change. Usually, the gray image obtained based on the maximum value method has high brightness. The calculation principle is shown in the following formula.

[0085]

[0086] (2)Average gray processing method: Take the average value of the RGB three-channel component values as the gray value of the corresponding pixel point after gray transformation. The calculation principle is shown in the formula.

[0087]

[0088] (3)Weighted average gray-scale processing method: This method is equivalent to the generalization of the average gray-scale processing method. After assigning appropriate weights (the sum of each weight is 1) to the RGB three-channel component values respectively, the gray-scale value of the corresponding pixel point of the gray-scale transformation is calculated. The calculation principle is as follows.

[0089]

[0090] Image filtering: Image filtering can eliminate invalid information in the input data. For image data, its filtering process usually refers to image smoothing, making the gray-scale changes between each pixel point in the image tend to be gentle and trying to retain the original information in the image data as much as possible. The image smoothing process is often implemented based on the spatial domain or the frequency domain, and its most intuitive manifestation is that the image will become blurred. When collecting and transmitting image data, due to various influences such as the hardware of the image acquisition system, the working environment, and the data transmission medium, various noises of different shapes and sizes will randomly appear in the image. Analyzing the image data from the frequency domain, the noise usually appears as high-frequency components. Therefore, a low-pass filter is often used in the image filtering process to minimize the impact of noise on feature extraction.

[0091] Comparing spatial domain and frequency domain filtering, the former has an absolute advantage in terms of processing speed. To ensure the real-time performance of image processing, Gaussian filtering algorithm is used for image filtering in the image preprocessing of this project.

[0092] Gaussian filtering can effectively reduce the impact of Gaussian noise on feature extraction. It uses the convolution operation of the Gaussian kernel and the image in the image smoothing process to achieve the effect of removing image noise. The calculation of the Gaussian kernel needs to be based on the Gaussian distribution function. The Gaussian distribution functions in one-dimensional space and two-dimensional space are shown in the following two formulas.

[0093]

[0094] Image mean normalization: Image mean normalization is a common image preprocessing technique used to reduce the mean of image data, making the image features have zero mean and unit variance, thereby accelerating the training process and improving the convergence speed of the model. The main purpose of image mean normalization is to adjust the image data by subtracting a mean vector so that the input data has zero mean. This step is usually performed in the input data preprocessing stage of the deep learning model. By mean normalization, the redundancy in the training data can be reduced, making it easier for the model to converge.

[0095] Its core calculation process is as follows: (1)Calculate the mean vector: Set a two-dimensional array M to represent the pixel value matrix of all images in the training set. Here, M is a four-dimensional tensor, and its dimensions are N, H, W, and C respectively. Among them, N is the number of images, H is the height of the image, W is the width of the image, and C is the number of channels of the image (for grayscale images, C = 1; for RGB images, C = 3). Then calculate the mean vector μ.

[0096] (2)Subtract the mean: For a new input image I, convert it into a three-dimensional array, and subtract the corresponding mean for each pixel position (i, j, c).

[0097] To further improve the convergence speed of the model, it can be chosen to perform normalization processing on the image after subtracting the mean, that is, divide by the standard deviation or a fixed value to make the features have unit variance.

[0098] Example 5, Feature recognition segmentation and area extraction post-processing: (1)Load the YOLOv8 model, YOLOv8, that is, YouOnlyLookOnce version 8, is an advanced object detection and instance segmentation model, and its design aims to provide fast and accurate predictions. YOLOv8 can not only identify the object categories and positions in the image, but also generate a segmentation mask for each object, that is, a pixel-level label, which is very important for subsequent analysis.

[0099] The specific operations are as follows: 1) Install the ultralytics library: First, ensure that the ultralytics library is installed. This is the main implementation library of YOLOv8. It can be installed by running pip install ultralytics.

[0100] 2) Pre-train the model: Take pictures of relevant leaked oil spots on-site, label them using labelling, and then input them into the yolo model for pre-training.

[0101] 3) Load the model: Use the YOLO class to load this pre-trained model file.

[0102] (2)Image detection, YOLOv8 can directly accept the original image and output the detection results. To improve the detection accuracy, image preprocessing is usually performed, such as resizing the image to meet the input size requirements of the model. The specific operations are as follows: 1) Load the image: Read the image file to be detected, for example, use the imread function of OpenCV.

[0103] 2) Preprocess the image: Operations such as scaling and cropping may be required to meet the model's requirements. YOLOv8 usually requires input images of a fixed size, so the image size may need to be adjusted.

[0104] 3) Input the image: Input the preprocessed image into the YOLOv8 model.

[0105] 4) Obtain the detection results: YOLOv8 will return a list of results, which contains information about the location, category, and segmentation mask of each detected object.

[0106] (3) Extract the image mask, The output of YOLOv8 contains the segmentation mask information of each detected object. The segmentation mask is a pixel-level label representing the contour of the object, usually composed of a set of polygon vertices. Extracting the mask is a necessary step for subsequent obtaining of the oil stain area, and the specific implementation steps are as follows: 1) Access the mask data: The results of YOLOv8 contain a masks attribute, which provides information about the segmentation mask.

[0107] 2) Extract the mask: For each detected object, masks.data can be used to obtain the mask data, which is a two-dimensional array representing the pixel values of the mask.

[0108] 3) Visualize the mask: The mask data can be visualized on the original image to better understand the detection results.

[0109] (4) Contour detection, First, perform contour detection. Based on the segmentation mask, the findContours function in OpenCV can be used to detect the contours within each mask. Contour detection can help us better understand the specific shape and distribution of the oil stain. A contour is a series of continuous boundary points that enclose a closed shape. For the segmentation mask, the contour refers to the boundary line of the oil stain.

[0110] The implementation steps are as follows: 1) Convert the mask to a binary image: If the segmentation mask is not given in the form of a binary image, then it needs to be converted to a black-and-white binary image first.

[0111] 2) Find the contours: Use the findContours function, which can find the boundaries composed of all non-zero pixels in the image.

[0112] 3) Sort the contours: If there are multiple contours, they can be sorted according to the area size or other criteria to select the largest contour as the main target.

[0113] 4) Contour drawing: The detected contours can be drawn on the original image to visually display the segmentation results.

[0114] (5) Oil spot area extraction, Following the contour detection in the previous step, the area of each contour can then be calculated. This can be done using the contourArea function. The area calculation is achieved by counting the pixels within the contour. For irregular shapes, OpenCV uses the polygon area formula for calculation.

[0115] The steps for calculating the area are as follows: 1) Calculate the area of a single contour: For each contour, use the contourArea function to calculate the area.

[0116] Statistical area of all contours: If there are multiple contours, calculate their areas separately and summarize the results according to requirements.

[0117] Example 6, Oil leakage anomaly alarm: In terms of program control, the present invention uses the yolov8 model for image segmentation and then obtains the area change information of the image through opencv. The detected visual information is transmitted to the industrial control computer and the alarm is controlled through the Siemens PLC. The specific technical solution is as Figure 3 shown: (1) Environment preparation, Hardware preparation: Camera: Used to capture real-time images. It is recommended to use a high-resolution industrial camera to obtain clear images.

[0118] Industrial control computer, i.e., industrial control PC: Runs image processing and object detection algorithms, such as YOLOv8, and is responsible for communication with the PLC.

[0119] Siemens PLC: Such as the S7-1200 or S7-1500 series, used to receive the detection results and control the alarm system.

[0120] Communication interface: Used for data transmission between the PLC and the industrial control computer, such as an Ethernet interface.

[0121] Software preparation: OpenCV library: Used for image acquisition and preprocessing.

[0122] YOLOv8 image segmentation model: Used to detect and segment the oil spot area.

[0123] Siemens TIAPortal: Used for PLC programming and configuration.

[0124] Communication protocol: such as ModbusTCP or Profinet, used for data exchange between the industrial computer and the PLC (2)Image acquisition and processing, First, use OpenCV to obtain real-time images from the camera and perform image processing and object detection. Preprocess the images, such as grayscale conversion, filtering, contrast enhancement, etc., to improve the detection accuracy. Finally, apply the YOLOv8 image segmentation model to analyze the images and identify and segment the oil stain areas.

[0125] (3)Data transmission, Establish a TCP / IP connection through Python's socket library and send the detection results (such as the position coordinates and area of the oil stain) from the industrial computer to the PLC. Optionally, use the ModbusTCP or Profinet protocol to ensure the stability and reliability of data transmission.

[0126] (4)PLC programming, Configure the PLC in TIAPortal to ensure that the Ethernet communication module of the PLC can receive data from the industrial computer. Write the PLC program using the TCP / IP communication library in TIAPortal to receive and parse the coordinate and area data sent by the industrial computer. Write the logic in the PLC to parse the received data and determine whether to trigger the alarm condition based on the detection results, such as when the area of the oil stain exceeds the set threshold.

[0127] (5)Alarm system control, When the PLC detects that the area of the oil stain exceeds the predetermined threshold, trigger an alarm signal. The PLC can control an external alarm, such as an audible and visual alarm, through a relay or other output module. If there is a recording function in the system, the time and other relevant information of the alarm event can also be recorded.

[0128] In the above method, a complete end-to-end system is established, which can realize the automation of the full detection process; the advantages of YOLOv8 in the segmentation task make it one of the ideal choices for dealing with image segmentation problems. It can not only provide high-precision segmentation results, but also has characteristics such as fast inference speed, simple model structure, and wide applicability. Using the oil stain on the test paper after oil leakage as the visual detection object is non-invasive. By checking the oil stain left on the test paper, it is possible to avoid disassembling and inspecting the reducer, which saves time and cost and reduces the possible additional damage to the equipment. And this method only requires simple placement of the test paper and regular inspection, without the need for complex tools or technologies.

[0129] The method of using test strips to detect oil leakage and then through visual recognition designed by the present invention is a relatively economical method. The cost of the test strips is low, and expensive detection equipment is not required. Moreover, by regularly checking the test strips, information on whether there is oil leakage can be obtained quickly, so that measures can be taken in a timely manner. Compared with directly visually detecting the reducer itself, detecting oil leakage through test strips has advantages such as easy model training and high detection accuracy.

[0130] The above embodiments are only the preferred technical solutions of the present invention and should not be regarded as limitations on the present invention. The embodiments in this application and the features in the embodiments can be arbitrarily combined with each other without conflict. The protection scope of the present invention should be the technical solutions recorded in the claims, including the equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, the equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A reducer oil leakage detection method based on machine vision, characterized in that: The steps include: S1, hardware selection, select a machine vision system consisting of light source, lens, camera, image collection device, image analysis device and interactive software; S2, image acquisition, two cameras are set up above the reducer. The cameras have no interference with other motion mechanisms in the application environment, and the field of view is clear and unobstructed; Collect images from the front to detect whether there is oil dripping from the bottom of the test paper and whether there is oil spot formation. Collect auxiliary images from the back to detect whether the change in the area of ​​the oil spot is abnormal. S3, grayscale processing, uses grayscale processing based on color features to improve the quality of the acquired image; S4, image filtering, uses Gaussian filtering for image preprocessing to remove noise while retaining the edges and other important features of the image; S5, image averaging, uses preprocessing technology to reduce the mean of image data so that image features have zero mean and unit variance, thereby accelerating the training process and improving the convergence speed of the model; S6, image segmentation, uses yolov8 to classify each pixel in the image to identify specific object instances; S7, oil spot extraction, after extracting the mask of the oil spot area from the output of YOLOv8, the findContours function of OpenCV is used to detect the contour of the oil spot area, and the contourArea function is used to calculate the area of ​​the contour; S8, image preprocessing, a series of preprocessing is performed to reduce the interference of various influencing factors; Including grayscale processing, image filtering and image averaging based on color features; S9, feature recognition and post-processing, uses YOLOv8 for target detection and instance segmentation models, provides fast and accurate prediction, identifies the object category and location in the image, and generates segmentation masks for each object, i.e. pixel-level labels; including loading the YOLOv8 model, image detection, extracting image masks, contour detection, and oil spot area extraction; S10, oil leakage abnormal alarm, uses yolov8 model to segment the image and then obtains the area change information of the image through opencv. The detected visual information is transmitted to the industrial computer and the alarm is controlled by Siemens PLC.

2. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S1, the factor for lens selection is focal length. Assuming the field of view is , the horizontal and vertical directions are and , the horizontal and vertical sizes of the camera chip target surface are and , the detection distance is , then the focal length The calculation formula is: ； 。 3. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S3, grayscale processing includes the following steps: S3-1, color space conversion, converting the original image from RGB color space to another color space, including HSV hue, saturation, brightness or Lab brightness, a channel, b channel; S3-2, color feature extraction, extracting channels related to specific color features in the converted color space; S3-3, color threshold processing, setting a threshold according to the extracted color features to divide the pixels in the image into foreground and background; S3-4, binarization processing, binarization processing is performed on the image after color threshold processing to obtain a binary image, in which the oil spot area is the foreground and the rest of the area is the background; S3-5, post-processing, performs morphological operations on the binary image to remove noise or refine the contour; performs connected component analysis on the binary image to extract the oil spot area.

4. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S4, image filtering includes the following steps: S4-1, select the Gaussian kernel size, and select the appropriate Gaussian kernel size according to the type of noise to be removed; S4-2, construct a Gaussian kernel, construct a two-dimensional Gaussian kernel, which is a discrete approximation of a two-dimensional Gaussian function; S4-3, normalized Gaussian kernel, to ensure that the sum of the filter is 1, the Gaussian kernel is normalized; the sum of all elements is calculated, and then each element is divided by this sum; S4-4, image convolution, uses the normalized Gaussian kernel to perform convolution operation with the image. For each pixel in the image, the Gaussian kernel is multiplied by the value of the pixel and its surrounding pixels, and the sum is obtained to obtain a new pixel value.

5. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S5, image averaging includes the following steps: S5-1, calculate the mean file, use all the images in the training set to calculate the average image, that is, calculate the average value of the pixel values ​​of all training images for each pixel position; S5-2, subtract the mean, for each input image, subtract the corresponding mean value from each pixel position; S5-3, standardization, in order to further improve the convergence speed of the model, the image after subtracting the mean is standardized, that is, divided by the standard deviation or a fixed value so that the feature has unit variance.

6. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S6, image segmentation includes the following steps: S6-1, input image, send the input image to the YOLOv8 network; S6-2, feature extraction, extracting multi-scale feature maps of the image through the backbone network backbone and the neck network neck; S6-3, segmentation prediction, prediction is performed on the feature map, each prediction unit corresponds to a position in the image, and for instance segmentation, each prediction unit outputs a category prediction and a segmentation mask; S6-4, post-processing, performs NMS non-maximum suppression processing on the prediction results, removes duplicate detection results, and screens out the final detection results based on the confidence threshold.

7. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S7, the contourArea function is used to calculate the area of ​​a given contour. The function returns the area of ​​the region enclosed by the contour. Specifically, it includes the following steps: S7-1, YOLOv8 instance segmentation output,First, the YOLOv8 model segments the input image and generates a segmentation mask for each detected oil spot; S7-2, extract mask, extract the segmentation mask of each oil spot from the output of YOLOv8; S7-3, contour detection, using OpenCV's findContours function to detect the contours in each oil spot mask; S7-4, Calculate Area, use the contourArea function to calculate the area of ​​each contour.

8. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S8, image averaging is to adjust the image data by subtracting a mean vector so that the input data has zero mean. It is performed in the input data preprocessing stage of the deep learning model. By averaging, the redundancy in the training data is reduced, making the model easier to converge. Its core calculation process includes the following steps: S8-1, calculate the mean vector, set a two-dimensional array M to represent the pixel value matrix of all images in the training set, where M is a four-dimensional tensor with dimensions N, H, W and C, where N is the number of images, H is the height of the image, W is the width of the image, and C is the number of channels of the image. For grayscale images, C=1; for RGB images, C=3; then calculate the mean vector μ; S8-2, subtract the mean. For a new input image I, convert it into a three-dimensional array, and for each pixel position (i, j, c), subtract the corresponding mean. S8-3, in order to further improve the convergence speed of the model, the image after subtracting the mean is selected to be standardized, that is, divided by the standard deviation or a fixed value so that the feature has unit variance.

9. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S9, contour detection uses OpenCV's findContours function to detect contours within each mask based on the segmentation mask. Contour detection helps understand the specific shape and distribution of the oil spot. The contour is a series of continuous boundary points that form a closed shape. For the segmentation mask, the contour refers to the boundary line of the oil spot. It specifically includes the following steps: S9-1, converting the mask into a binary image. If the segmentation mask is not given in the form of a binary image, first convert it into a black and white binary image; S9-2, find contours, use the findContours function to find the boundaries of all non-zero pixels in the image; S9-3, contour sorting, if there are multiple contours, sort them according to area size or other criteria so that the largest contour can be selected as the primary target; S9-4, contour drawing, draws the detected contours on the original image to intuitively display the segmentation results.

10. The reducer oil leakage detection method based on machine vision according to claim 1 is characterized in that: In S10, the oil leakage abnormal alarm specifically includes the following steps: S10-1, image acquisition and processing, first use OpenCV to obtain real-time images from the camera, and perform image processing and target detection; preprocess the image, including grayscale, filtering, and contrast enhancement to improve detection accuracy; finally, use the YOLOv8 image segmentation model to analyze the image, identify and segment the oil spot area; S10-2, data transmission, establish a TCP / IP connection through the Python socket library, and send the detection results, such as the location coordinates and area of ​​the oil spot, from the industrial computer to the PLC; S10-3, PLC program design, configure PLC in TIA Portal, ensure that the Ethernet communication module of PLC can receive data from the industrial computer, use the TCP / IP communication library in TIA Portal to write PLC program, receive and parse the coordinate and area data sent by the industrial computer; write logic in PLC, parse the received data, and determine whether to trigger alarm conditions based on the detection results, including whether the oil spot area exceeds the set threshold; S10-4, alarm system control, when the PLC detects that the area of ​​the oil spot exceeds the preset threshold, an alarm signal is triggered; the PLC controls the external alarm through a relay or output module.

Images