Method, device, computer equipment and storage medium for identifying foreign matter inside equipment

Abstract

The present application relates to a method, device, computer equipment and storage medium for identifying foreign objects inside a device. The method comprises: obtaining an initial filtered image; filtering and purifying the initial filtered image to obtain a purified image; performing morphological analysis on the purified image to obtain a foreign object contour information image; performing morphological recognition based on the foreign object contour information image to obtain a foreign object recognition result. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign object detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment and storage medium for identifying foreign matter inside a device. Background Art

[0002] Camera is a video input device, which is widely used in video conferencing, telemedicine and real-time monitoring. Camera includes lens, image sensor and power pack. At present, foreign objects such as dust, debris and hair are easily introduced into the camera during the assembly process, which reduces the image quality. Therefore, it is necessary to detect foreign objects inside the camera.

[0003] The traditional method of detecting foreign objects inside a camera is to obtain the coordinates of the highest peak by detecting the filtered image after collecting the image inside the camera. For a camera without foreign objects, the image waveform is a smooth transition without sudden changes in the peak. When there are foreign objects, the foreign objects, as high-frequency noise, will cause the peak to suddenly change to the highest coordinates. The coordinates with output peaks higher than the threshold are output to indicate the location and shape of the foreign object. The specific value of the threshold can be selected according to the actual situation. However, if the threshold is set unreasonably, for example, the threshold is set too high, when the peak mutation caused by the foreign object is low, it cannot be identified. Defects in the camera itself can also cause the highest peak value to appear, resulting in misjudgment. The traditional foreign object detection method has the disadvantage of low detection reliability. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for identifying foreign objects inside a device, which can improve the reliability of foreign object detection, in order to address the above problems.

[0005] A method for identifying foreign matter inside a device, the method comprising:

[0006] Get the initial filtered image;

[0007] Filtering and purifying the initial filtered image to obtain a purified image;

[0008] Performing morphological analysis on the purified image to obtain a foreign body contour information image;

[0009] Morphological recognition is performed based on the foreign body contour information image to obtain a foreign body recognition result.

[0010] The above-mentioned method for identifying foreign objects inside the device suppresses the background imaging in the initial filtered image by filtering and purifying the image, thereby highlighting the foreign object imaging to facilitate the subsequent extraction of foreign object contour information. The purified image is subjected to morphological analysis, and the image is subjected to shape extraction to complete image analysis and identification. Finally, the foreign object-related information in the image is extracted based on the obtained foreign object contour information image as the foreign object identification result, so as to facilitate the judgment of the foreign object's morphology. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign object detection.

[0011] In one embodiment, filtering and purifying the initial filtered image to obtain a purified image includes:

[0012] Performing median filtering on the initial filtered image to remove background noise, thereby obtaining an image after the background noise is removed;

[0013] Performing first-order derivative and superposition processing on the image after background noise removal to obtain a derived image;

[0014] The closed areas in the derived image whose gradient amplitude is less than a preset amplitude are suppressed to obtain a purified image.

[0015] First, median filtering is performed on the initial filtered image to remove the noise of the image while maintaining the original clear outline of the image. Then, the image after removing the background noise is subjected to first-order derivative and superposition processing. The grayscale values ​​of different points in the image are derived to obtain the first-order gradient of the image for image edge detection. Finally, closed area suppression is performed to complete the image purification for subsequent morphological analysis to ensure that the image filtering and purification is accurate and reliable.

[0016] In one embodiment, the foreign body contour information image is a foreign body contour corner point image, and the morphological analysis of the purified image to obtain the foreign body contour information image includes:

[0017] Performing a morphological closing operation on the purified image, removing black spots inside the closed area of ​​the purified image and restoring and retaining the external area, to obtain a restored image;

[0018] Performing a morphological closing operation on the restored image to smooth the edge of the image to obtain a smoothed image;

[0019] Corner point solving is performed on the smoothed image to obtain a foreign body contour corner point image.

[0020] The purified image is subjected to morphological closing operation, and the black spots in the image are removed and the image edge is smoothed to reduce interference information. Then, the corner point solution is used to effectively reduce the amount of information data while retaining the important features of the image graphics, which effectively improves the calculation speed and is conducive to reliable image matching.

[0021] In one embodiment, the performing morphological recognition according to the foreign body contour information image to obtain a foreign body recognition result includes:

[0022] Performing closed curve fitting on the corner points in the contour corner point image to obtain a curve equation;

[0023] Performing two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information;

[0024] The curve equation and the foreign object information are drawn into the initial filtered image to obtain a foreign object recognition image as a foreign object recognition result.

[0025] By fitting the corner points in the contour corner point image with a closed curve and simulating the shape contour equation of the foreign body, the relevant information of the foreign body can be accurately extracted. Then the curve equation and foreign body information are drawn into the initial filtered image, which makes it convenient for the inspector to view the relevant information of the foreign body in combination with the original image, and also improves the observation convenience of foreign body detection.

[0026] In one embodiment, the foreign object information includes at least one of the foreign object type, foreign object location and foreign object area.

[0027] Inspectors can choose the content of foreign body information according to actual needs to ensure that the test results are consistent with actual needs, avoid inconvenience to inspectors' analysis due to too much or too little foreign body information detected, and improve the operational convenience of foreign body detection.

[0028] In one embodiment, the obtaining of the initial filtered image includes: receiving an original image acquired by a device, and filtering the original image to obtain the initial filtered image.

[0029] It can be the original image received from its own device or the original image sent by an external image processing device. The received original image is filtered to obtain an initial filtered image for subsequent image analysis. By filtering the original image, the accuracy of image analysis can be improved.

[0030] A device for identifying foreign matter inside a device, comprising:

[0031] An image acquisition module, used for acquiring an initial filtered image;

[0032] An image filtering module, used for filtering and purifying the initial filtered image to obtain a purified image;

[0033] An image analysis module, used for performing morphological analysis on the purified image to obtain a foreign body contour information image;

[0034] The morphology recognition module is used to perform morphology recognition according to the foreign body contour information image to obtain a foreign body recognition result.

[0035] The foreign body identification device inside the above-mentioned device suppresses the background imaging in the initial filtered image through image filtering and purification, thereby highlighting the foreign body imaging to facilitate the subsequent extraction of foreign body contour information. The purified image is subjected to morphological analysis, and the image is subjected to shape extraction to complete image analysis and identification. Finally, the foreign body related information in the image is extracted based on the obtained foreign body contour information image as the foreign body identification result, so as to facilitate the judgment of the foreign body's morphology. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign body detection.

[0036] In one of the embodiments, the foreign object recognition result is obtained by drawing the curve equation and the foreign object information into the initial filtered image.

[0037] The curve equation and foreign body information are drawn into the initial filtered image, which makes it convenient for inspectors to view the foreign body related information in combination with the original image, and also improves the observation convenience of foreign body detection.

[0038] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] Get the initial filtered image;

[0040] Filtering and purifying the initial filtered image to obtain a purified image;

[0041] Performing morphological analysis on the purified image to obtain a foreign body contour information image;

[0042] Morphological recognition is performed based on the foreign body contour information image to obtain a foreign body recognition result.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Get the initial filtered image;

[0045] Filtering and purifying the initial filtered image to obtain a purified image;

[0046] Performing morphological analysis on the purified image to obtain a foreign body contour information image;

[0047] Morphological recognition is performed based on the foreign body contour information image to obtain a foreign body recognition result.

[0048] The above-mentioned computer device and computer-readable storage medium suppress background imaging in the initial filtered image by filtering and purifying the image, thereby highlighting the foreign body imaging to facilitate the subsequent extraction of foreign body contour information. The purified image is subjected to morphological analysis, and shape extraction is performed on the image to complete image analysis and recognition. Finally, the foreign body-related information in the image is extracted based on the obtained foreign body contour information image as the foreign body recognition result, so as to facilitate the judgment of the morphology of the foreign body. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign body detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a method for identifying foreign matter inside a device in one embodiment;

[0050] Figure 2 A flowchart of filtering and purifying an initial filtered image to obtain a purified image in an embodiment;

[0051] Figure 3 A flowchart of performing morphological analysis on a purified image to obtain a foreign body contour information image in an embodiment;

[0052] Figure 4 It is a flow chart of performing morphological recognition according to a foreign body contour information image to obtain a foreign body recognition result in an embodiment;

[0053] Figure 5 is a flow chart of a method for identifying foreign matter inside a device in another embodiment;

[0054] Figure 6 is a schematic diagram of an initial filtered image in one embodiment;

[0055] Figure 7 is a schematic diagram of an image after purification in one embodiment;

[0056] Figure 8 is a schematic diagram of a foreign body contour information image in an embodiment;

[0057] Fig. 9 is a schematic diagram of a foreign body recognition image in one embodiment;

[0058] Fig.10 is a structural block diagram of a device for identifying foreign matter inside a device in an embodiment;

[0059] Fig.11is a structural block diagram of an image filtering module in one embodiment;

[0060] Fig.12 is a structural block diagram of an image analysis module in one embodiment;

[0061] Fig.13 is a structural block diagram of a morphology recognition module in an embodiment;

[0062] Fig.14 is a structural block diagram of a device for identifying foreign matter inside a device in another embodiment;

[0063] Fig.15 1 is a diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] In one embodiment, a method for identifying foreign objects inside a device is provided, which is applicable to identifying foreign objects inside a camera, and the foreign objects may specifically be dust, etc. Figure 1 As shown, the method includes:

[0066] Step S110: Acquire an initial filtered image.

[0067] The controller can obtain an initial filtered image obtained by filtering the original image captured by the device. Specifically, the controller can be connected to an image acquisition device, and the image acquisition device can be used to capture images inside the device, and the captured original image inside the device is sent to the controller. The controller filters the original image to obtain an initial filtered image for subsequent image processing. Alternatively, the controller and the image acquisition device can be integrated into a processing device, and the integrated processing device can be used to capture the original image inside the device and filter it to obtain an initial filtered image. In addition, the controller can also be connected to an external image processing device. Specifically, the external image processing device first captures the original image inside the device, and then filters the original image to obtain an initial filtered image, and finally sends the initial filtered image to the controller so that the controller can perform subsequent image processing on the initial filtered image.

[0068] In one embodiment, step S110 includes: receiving an original image acquired by a device, and filtering the original image to obtain an initial filtered image. The original image may be received from the device itself, or may be received from an external image processing device. The received original image is filtered to obtain an initial filtered image for subsequent image analysis. By filtering the original image, the accuracy of image analysis can be improved.

[0069] Step S120: filtering and purifying the initial filtered image to obtain a purified image.

[0070] After acquiring the initial filtered image, the controller filters and purifies the initial filtered image to remove the image background noise, thereby improving the accuracy of the subsequent recognition of foreign matter information in the image. There is no unique way to filter and purify the image, and the image can be purified by median filtering, Gaussian filtering, or mean filtering. Among them, the median filter replaces the pixel at the center of the window with the median of each point in a sliding window containing an odd number of points, and its effect is closely related to the selected window size. Different image contents and different application requirements often require the use of different window shapes and sizes. In one embodiment, Figure 2 As shown, step S120 includes steps S122 to S126.

[0071] Step S122: Perform median filtering on the initial filtered image to remove background noise and obtain an image after removing background noise. As a nonlinear signal processing method, median filtering sets the grayscale value of each pixel to the median of the grayscale values ​​of all pixels in a neighborhood window of the point. The value of a point in the image is replaced by the median of the values ​​of each point in a neighborhood of the point, so that the surrounding pixel values ​​are close to the true value, thereby eliminating isolated noise points. Median filtering can overcome the blurring of image details caused by linear filtering (such as smoothing filtering), and is most effective in filtering out pulse interference and image scanning noise. Image contour information is a very important feature in image processing and pattern recognition. The outstanding feature of using median filtering is that it removes the noise of the image while maintaining the original clear contour of the image.

[0072] Step S124: Perform first-order derivative and superposition processing on the image after background noise removal to obtain the derived image. The specific method of performing first-order derivative and superposition processing on the image is not unique, and the image may be first-order derivative and superposition processing using the Sobel algorithm, the Canny algorithm or the Laplacian algorithm. In this embodiment, the Sobel algorithm is used to perform first-order derivatives in the X and Y directions of the image after background noise removal and superposition to obtain the derived image. Among them, the X and Y directions refer to the horizontal and vertical directions of the image, respectively. The Sobel algorithm combines Gaussian smoothing and differential derivation to calculate the approximate gradient of the image grayscale function. The grayscale values ​​of different points in the image are derived along the X and Y directions to obtain the first-order gradient of the image for image edge detection. The Sobel algorithm weights the influence of the position of the pixel, which can reduce the degree of edge blur and has a good detection effect.

[0073] Step S126: Suppress the closed areas in the image after the derivation where the gradient amplitude is less than the preset amplitude to obtain a purified image. The specific value of the preset amplitude is not unique and can be adjusted according to actual needs. In the image after the derivation using the Sobel algorithm, suppress the closed areas where the gradient amplitude is less than the preset amplitude to complete the purification of the image for subsequent morphological analysis.

[0074] First, median filtering is performed on the initial filtered image to remove the noise of the image while maintaining the original clear outline of the image. Then, the image after removing the background noise is subjected to first-order derivative and superposition processing. The grayscale values ​​of different points in the image are derived to obtain the first-order gradient of the image for image edge detection. Finally, closed area suppression is performed to complete the image purification for subsequent morphological analysis to ensure that the image filtering and purification is accurate and reliable.

[0075] Please continue to refer to Figure 1 , Step S130: Perform morphological analysis on the purified image to obtain a foreign body contour information image.

[0076] Image morphological analysis is a nonlinear transformation that marks the contour of foreign objects in the image by corroding and dilating the image. The contour of foreign objects can be the corner points or other features of the foreign objects. The specific method of image morphological analysis is not unique, and can include two operations: opening operation and closing operation. The opening operation refers to corroding the image first and then dilating it, and the closing operation refers to dilating the image first and then corroding it. In one embodiment, the foreign object contour information image is a foreign object contour corner point image, such as Figure 3 As shown, step S130 includes steps S132 to S136.

[0077] Step S132: Perform morphological closing operation on the purified image, remove the black spots inside the closed area in the purified image and restore the external area to obtain a restored image. Using morphological closing operation, first perform dilation processing on the image, remove the black spots inside the closed area in the image and restore the external area. The dilation processing can include three methods: horizontal dilation, vertical dilation and omnidirectional dilation.

[0078] Step S134: Perform morphological closing operation on the restored image, and smooth the image edge to obtain a smoothed image. Using morphological closing operation, perform erosion processing on the image after dilation processing, and smooth the image edge. Similarly, the erosion processing can also include three methods: horizontal erosion, vertical erosion and omnidirectional erosion.

[0079] Step S136: Solve the corner points of the smoothed image to obtain the foreign body contour corner point image. Perform corner point detection on the smoothed image to generate the foreign body contour corner point image. There is no unique method for corner point detection. The Harris corner point detection algorithm can be used for corner point detection. The corner points in the image can be considered as the connection points of the object contour lines. When the shooting angle changes, these feature points can still maintain stable properties well. While retaining the important features of the image graphics, the corner points can effectively reduce the amount of information data, making its information content very high, effectively improving the speed of calculation, facilitating reliable matching of images, and making real-time processing possible.

[0080] The purified image is subjected to morphological closing operation, and the black spots in the image are removed and the image edge is smoothed to reduce interference information. Then, the corner point solution is used to effectively reduce the amount of information data while retaining the important features of the image graphics, which effectively improves the calculation speed and is conducive to reliable image matching.

[0081] Please continue to refer to Figure 1 , step S140: performing morphological recognition according to the foreign body contour information image to obtain a foreign body recognition result.

[0082] The foreign object is identified according to the foreign object contour information image to determine the general composition, height position, area and other information of the foreign object. Specifically, in one embodiment, Figure 4 As shown, step S140 includes steps S142 to S146.

[0083] Step S142: Perform closed curve fitting on the corner points in the contour corner point image to obtain a curve equation. Perform closed curve fitting on the corner points of the foreign body contour, eliminate repeated roots, and obtain the curve equation of the foreign body closed curve. Specifically, the contour corner point image can be imported into the Matlab tool, and the closed curve fitting function of the Matlab tool can be used to obtain the curve equation of the foreign body contour corner point.

[0084] Step S144: Perform two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information. Perform two-dimensional derivation on the image area surrounded by the curve equation to obtain image gradient, and obtain foreign body information based on image gradient analysis. The two-dimensional derivation method is not unique. Specifically, the Plath operator can be used to perform two-dimensional derivation of the image. The specific content of the foreign body information is not unique. In this embodiment, the foreign body information includes at least one of the foreign body type, foreign body location and foreign body area. The detection personnel can select the content of the foreign body information according to actual needs, so as to facilitate subsequent information viewing and calling, and improve the convenience of operation.

[0085] Step S146: Plot the curve equation and foreign body information into the initial filtered image to obtain a foreign body recognition image as a foreign body recognition result. Plot the obtained curve equation and foreign body information into the initial filtered image, and use the final foreign body recognition image as a recognition result for subsequent image display, storage, and abnormality analysis.

[0086] The corner points in the contour corner point image are fitted with a closed curve to simulate the shape contour equation of the foreign object, and the relevant information of the foreign object can be accurately extracted. Then the curve equation and the foreign object information are drawn into the initial filtered image, which is convenient for the inspection personnel to view the relevant information of the foreign object in combination with the original image, and also improves the observation convenience of foreign object detection. It can be understood that in other embodiments, the foreign object information can also be directly used as the foreign object recognition result.

[0087] Further, in one embodiment, if Figure 5 As shown, after step S140, the method further includes step S150.

[0088] Step S150: Output the foreign body identification result to the display for display. The foreign body identification result obtained by the detection is displayed for the detection personnel to view, and the cause of the equipment abnormality is found according to the displayed result so as to quickly solve the problem, thereby improving the convenience of foreign body identification detection.

[0089] In order to better understand the above-mentioned method for identifying foreign objects inside a device, the following is a detailed explanation using the example of identifying foreign objects inside a camera.

[0090] The foreign body recognition and analysis technology inside the camera equipment uses the initial filtered image ( Figure 6 ), first further filter and purify to obtain the purified image ( Figure 7 ), and then use morphological operations to obtain the foreign body contour information image ( Figure 8 ), mark the foreign body contour, and obtain the foreign body recognition image by identifying the foreign body contour ( Fig. 9 ) to determine the approximate composition, height, position and area of ​​the foreign matter. Figure 6-Figure 9 As shown, the background image 10 is the part outside the foreign body image 20, which is mainly composed of low-frequency noise and clutter, while the foreign body image 20 is the image on the image sensor after the foreign body blocks the light path, which is mainly composed of high-frequency noise. Figure 6 Further filtering and purification can suppress the background image 10 to be a suppressed background image 101, thereby highlighting the foreign matter image purification body 30. Figure 7 By performing morphological analysis, the foreign body contour corner points 40 of the foreign body pure body 30 can be calculated. By completing the foreign body contour corner points 40 with closed curve fitting, the foreign body closed curve 201 can be calculated. By identifying and analyzing its morphology, the location information of the camera device component where the foreign body is located and the approximate foreign body type and area 202 can be obtained. The various stages of foreign body detection are explained in detail as follows:

[0091] Filtering and purification: the original image img( Figure 6 ) Repeatedly use median filtering to remove background noise and obtain the processed image imgA; perform Sobel algorithm on imgA to obtain the first-order derivative in the X and Y directions and superimpose them to obtain image imgB; suppress all closed areas less than the threshold value to obtain the final image imgC of this stage ( Figure 7 ).

[0092] Morphological analysis operation: perform morphological closing operation on imgC, remove the black spots inside the closed area and restore the external area to obtain the processed image imgD; perform morphological closing operation on imgD to smooth the image edge and obtain imgE; solve the skeleton points (corner points) of imgF to obtain the final image imgG ( Figure 8 ).

[0093] Foreign body morphology recognition and analysis: Fit the corner points of imgG with a closed curve, exclude repeated roots, and obtain the curve equation F1; perform two-dimensional (several orders) derivatives on the area surrounded by F1, perform machine learning, and then obtain the foreign body type and height position F2; draw F1 and F2 on the original image img to obtain the final image imgH ( Fig. 9 ).

[0094] The above-mentioned method for identifying foreign objects inside the device suppresses the background imaging in the initial filtered image by filtering and purifying the image, thereby highlighting the foreign object imaging to facilitate the subsequent extraction of foreign object contour information. The purified image is subjected to morphological analysis, and the image is subjected to shape extraction to complete image analysis and identification. Finally, the foreign object-related information in the image is extracted based on the obtained foreign object contour information image as the foreign object identification result, so as to facilitate the judgment of the foreign object's morphology. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign object detection.

[0095] It should be understood that although Figure 1-5The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-5 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0096] In one embodiment, a device for identifying foreign objects inside a device is provided, which is suitable for identifying foreign objects inside a camera, and the foreign objects may specifically be dust, etc. Fig.10 As shown, the device includes an image acquisition module 110 , an image filtering module 120 , an image analysis module 130 and a morphology recognition module 140 .

[0097] The image acquisition module 110 is used to acquire an initial filtered image. Specifically, in one embodiment, the image acquisition module 110 receives the original image acquired by the device, and filters the original image to obtain the initial filtered image. The original image may be received from the device itself, or may be received from an external image processing device. The received original image is filtered to obtain the initial filtered image for subsequent image analysis. By filtering the original image, the accuracy of image analysis can be improved.

[0098] The image filtering module 120 is used to filter and purify the initial filtered image to obtain a purified image. After obtaining the initial filtered image, the initial filtered image is filtered and purified to remove image background noise, thereby improving the accuracy of subsequent recognition of foreign matter information in the image. In one embodiment, Fig.11 As shown, the image filtering module 120 includes an image filtering unit 122 , an image processing unit 124 and a region suppression unit 126 .

[0099] The image filtering unit 122 is used to perform median filtering on the initial filtered image to remove background noise, thereby obtaining an image after the background noise is removed.

[0100] The image processing unit 124 is used to perform first-order derivative and superposition processing on the image after the background noise is removed to obtain a derived image.

[0101] The region suppression unit 126 is used to suppress closed regions in the derived image whose gradient amplitude is less than a preset amplitude to obtain a purified image.

[0102] First, median filtering is performed on the initial filtered image to remove the noise of the image while maintaining the original clear outline of the image. Then, the image after removing the background noise is subjected to first-order derivative and superposition processing. The grayscale values ​​of different points in the image are derived to obtain the first-order gradient of the image for image edge detection. Finally, closed area suppression is performed to complete the image purification for subsequent morphological analysis to ensure that the image filtering and purification is accurate and reliable.

[0103] The image analysis module 130 is used to perform morphological analysis on the purified image to obtain a foreign body contour information image. The specific method of image morphological analysis is not unique, and may include two operations: opening operation and closing operation. The opening operation refers to corroding the image first and then dilating it, and the closing operation refers to dilating the image first and then corroding it. In one embodiment, the foreign body contour information image is a foreign body contour corner point image, such as Fig.12 As shown, the image analysis module 130 includes a region processing unit 132 , an image smoothing unit 134 and an image analysis unit 136 .

[0104] The region processing unit 132 is used to perform a morphological closing operation on the purified image, remove the black spots inside the closed region of the purified image, and restore and retain the external region to obtain a restored image.

[0105] The image smoothing unit 134 is used to perform a morphological closing operation on the restored image to smooth the image edge to obtain a smoothed image.

[0106] The image analysis unit 136 is used to perform corner point detection on the smoothed image to obtain a foreign body contour corner point image. The image analysis unit 136 is used to perform corner point detection on the smoothed image to generate a foreign body contour corner point image.

[0107] The purified image is subjected to morphological closing operation, and the black spots in the image are removed and the image edge is smoothed to reduce interference information. Then, the corner point solution is used to effectively reduce the amount of information data while retaining the important features of the image graphics, which effectively improves the calculation speed and is conducive to reliable image matching.

[0108] The morphological recognition module 140 is used to perform morphological recognition based on the foreign body contour information image to obtain a foreign body recognition result. The foreign body contour is recognized based on the foreign body contour information image to determine the approximate composition, height position, area and other information of the foreign body. In one embodiment, the foreign body recognition result is obtained by drawing the curve equation and foreign body information into the initial filtered image. Drawing the curve equation and foreign body information into the initial filtered image makes it convenient for the inspection personnel to view the foreign body related information in combination with the original image.

[0109] Specifically, in one embodiment, Fig.13As shown, the morphology recognition module 140 includes a curve fitting unit 142 , a foreign object recognition unit 144 and an information generation unit 146 .

[0110] The curve fitting unit 142 is used to perform closed curve fitting on the corner points in the contour corner point image to obtain a curve equation.

[0111] The foreign body identification unit 144 is used to perform two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information.

[0112] The information generating unit 146 is used to draw the curve equation and the foreign object information into the initial filtered image to obtain a foreign object recognition image as a foreign object recognition result.

[0113] The corner points in the contour corner point image are fitted with a closed curve to simulate the shape contour equation of the foreign object, and the relevant information of the foreign object can be accurately extracted. Then the curve equation and the foreign object information are drawn into the initial filtered image, which is convenient for the inspection personnel to view the relevant information of the foreign object in combination with the original image, and also improves the observation convenience of foreign object detection. It can be understood that in other embodiments, the foreign object information can also be directly used as the foreign object recognition result.

[0114] Further, in one embodiment, if Fig.14 As shown, the device also includes an information output module 150.

[0115] The information output module 150 is used to output the foreign body identification result to the display for display after the morphology recognition module 140 performs morphology recognition based on the foreign body contour information image and obtains the foreign body identification result. The foreign body identification result obtained by the detection is displayed for the inspection personnel to view, and the cause of the equipment abnormality is found according to the displayed result to quickly solve the problem, thereby improving the convenience of foreign body identification detection.

[0116] The foreign body identification device inside the above-mentioned device suppresses the background imaging in the initial filtered image by filtering and purifying the image, thereby highlighting the foreign body imaging to facilitate the subsequent extraction of foreign body contour information. The purified image is subjected to morphological analysis, and the image is subjected to shape extraction to complete image analysis and identification. Finally, the foreign body related information in the image is extracted based on the obtained foreign body contour information image as the foreign body identification result, so as to facilitate the judgment of the foreign body's morphology. Compared with traditional detection methods, the results are more intuitive, and the situation of incorrect detection due to inaccurate threshold setting is avoided, thereby improving the reliability of foreign body detection.

[0117] For the specific definition of the device for identifying foreign objects inside the device, please refer to the definition of the method for identifying foreign objects inside the device above, which will not be repeated here. Each module in the above-mentioned device for identifying foreign objects inside the device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0118] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.15 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying foreign objects inside the device is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0119] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program: acquiring an initial filtered image; filtering and purifying the initial filtered image to obtain a purified image; performing morphological analysis on the purified image to obtain a foreign body contour information image; and performing morphological recognition based on the foreign body contour information image to obtain a foreign body recognition result.

[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing median filtering on the initial filtered image to remove background noise, thereby obtaining an image after the background noise is removed; performing first-order derivative and superposition processing on the image after the background noise is removed, thereby obtaining a derived image; and suppressing closed areas in the derived image whose gradient amplitude is less than a preset amplitude, thereby obtaining a purified image.

[0122] In one embodiment, the foreign body contour information image is a foreign body contour corner point image, and the processor also implements the following steps when executing the computer program: performing a morphological closing operation on the purified image, removing black spots inside the closed area of ​​the purified image and restoring and retaining the external area to obtain a restored image; performing a morphological closing operation on the restored image, smoothing the image edges to obtain a smoothed image; performing corner point solving on the smoothed image to obtain a foreign body contour corner point image.

[0123] In one embodiment, when the processor executes the computer program, the following steps are also implemented: performing closed curve fitting on the corner points in the contour corner point image to obtain the curve equation; performing two-dimensional derivative on the image area surrounded by the curve equation to obtain foreign body information; and drawing the curve equation and foreign body information into the initial filtered image to obtain a foreign body recognition image as a foreign body recognition result.

[0124] In one embodiment, when the processor executes the computer program, the following steps are further implemented: receiving an original image acquired by the device, and filtering the original image to obtain an initial filtered image.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining an initial filtered image; filtering and purifying the initial filtered image to obtain a purified image; performing morphological analysis on the purified image to obtain a foreign body contour information image; performing morphological recognition based on the foreign body contour information image to obtain a foreign body recognition result.

[0126] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: performing median filtering on the initial filtered image to remove background noise, thereby obtaining an image after the background noise is removed; performing first-order derivative and superposition processing on the image after the background noise is removed, thereby obtaining a derived image; and suppressing closed areas in the derived image whose gradient amplitude is less than a preset amplitude, thereby obtaining a purified image.

[0127] In one embodiment, the foreign body contour information image is a foreign body contour corner point image, and the computer program, when executed by the processor, also implements the following steps: performing a morphological closing operation on the purified image, removing black spots inside the closed area of ​​the purified image and restoring and retaining the external area to obtain a restored image; performing a morphological closing operation on the restored image, smoothing the image edges to obtain a smoothed image; performing corner point solving on the smoothed image to obtain a foreign body contour corner point image.

[0128] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: closed curve fitting is performed on the corner points in the contour corner point image to obtain the curve equation; two-dimensional derivative is performed on the image area surrounded by the curve equation to obtain foreign body information; the curve equation and foreign body information are drawn into the initial filtered image to obtain a foreign body recognition image as a foreign body recognition result.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: receiving an original image acquired by the device, and filtering the original image to obtain an initial filtered image.

[0130] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0131] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying foreign matter inside a device, characterized in that: The method includes: Get the initial filtered image; Filtering and purifying the initial filtered image to obtain a purified image; Performing morphological analysis on the purified image to obtain a foreign body contour information image; Performing morphological recognition according to the foreign body contour information image to obtain a foreign body recognition result; The foreign body contour information image is a foreign body contour corner point image, and the morphological analysis of the purified image to obtain the foreign body contour information image includes: performing a morphological closing operation on the purified image, removing black spots inside the closed area of ​​the purified image and restoring the external area to obtain a restored image; performing a morphological closing operation on the restored image, smoothing the image edge to obtain a smoothed image; performing corner point solving on the smoothed image to obtain a foreign body contour corner point image, wherein the corner point is a connection point of the object contour line in the image; The method of performing morphological recognition based on the foreign object contour information image to obtain a foreign object recognition result includes: performing closed curve fitting on the corner points in the contour corner point image to obtain a curve equation; performing two-dimensional differentiation on the image area surrounded by the curve equation to obtain foreign object information; and drawing the curve equation and the foreign object information into the initial filtered image to obtain a foreign object recognition image as a foreign object recognition result.

2. The method according to claim 1, characterized in that The filtering and purifying the initial filtered image to obtain a purified image includes: Performing median filtering on the initial filtered image to remove background noise, thereby obtaining an image after the background noise is removed; Performing first-order derivative and superposition processing on the image after background noise removal to obtain a derived image; The closed areas in the derived image whose gradient amplitude is less than a preset amplitude are suppressed to obtain a purified image.

3. The method according to claim 1, characterized in that The performing two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information comprises: Performing two-dimensional derivation on the image area surrounded by the curve equation to obtain an image gradient; Foreign body information is obtained according to the image gradient analysis.

4. The method according to claim 1, characterized in that: The performing two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information comprises: The Plath operator is used to perform two-dimensional derivation on the image area surrounded by the curve equation to obtain foreign body information.

5. The method according to claim 4, characterized in that The foreign object information includes at least one of the foreign object type, the foreign object position and the foreign object area.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the initial filtered image comprises: An original image acquired by a device is received, and the original image is filtered to obtain an initial filtered image.

7. A device for identifying foreign matter inside a device, characterized in that: include: An image acquisition module, used for acquiring an initial filtered image; An image filtering module, used for filtering and purifying the initial filtered image to obtain a purified image; An image analysis module, used for performing morphological analysis on the purified image to obtain a foreign body contour information image; A morphology recognition module, used to perform morphology recognition based on the foreign body contour information image to obtain a foreign body recognition result; The foreign body contour information image is a foreign body contour corner point image, and the image analysis module is specifically used to: perform a morphological closing operation on the purified image, remove the black spots inside the closed area in the purified image, and restore the external area to obtain a restored image; perform a morphological closing operation on the restored image, and smooth the image edge to obtain a smoothed image; perform corner point solving on the smoothed image to obtain a foreign body contour corner point image, wherein the corner point is a connection point of the object contour line in the image; The morphological recognition module is specifically used to: perform closed curve fitting on the corner points in the contour corner point image to obtain the curve equation; perform two-dimensional derivative on the image area surrounded by the curve equation to obtain foreign body information; draw the curve equation and the foreign body information into the initial filtered image to obtain a foreign body recognition image as a foreign body recognition result.

8. The device according to claim 7, characterized in that The foreign body recognition result is obtained by drawing the curve equation and the foreign body information into the initial filtered image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images