Foreign matter detection method and device based on image processing and related medium

Abstract

The invention discloses a foreign matter detection method and device based on image processing and a related medium. The method comprises the following steps: acquiring a target image to be recognized; performing segmentation processing on the target image by using an image segmentation model to obtain a segmentation mask graph corresponding to the target image; and carrying out logic processing on the segmented mask graph, and judging whether foreign matters exist in the target image or not according to a logic processing result. According to the embodiment of the invention, the target image is segmented to extract the segmented mask image of the target image, and then the segmented mask image is logically processed to judge whether foreign matters exist in the segmented mask image, so that the problem of insufficient recognition precision caused by lack of foreign matter materials can be solved, the foreign matter detection precision is improved, and the detection efficiency is improved. And the foreign matter detection method can be suitable for detection of different foreign matters in different scenes, the problem of excessive customization is avoided, and the generalization ability of foreign matter detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer software technology, and in particular to a foreign body detection method and device based on image processing and related media. Background Art

[0002] In places such as electric power, petrochemicals, rail transit, and production workshops, it is necessary to monitor equipment and other targets in order to detect foreign objects in time and then generate alarms to ensure the operation and production safety of equipment. Traditional foreign object monitoring methods mainly rely on manual or detection instruments. With the popularization and deployment of monitoring equipment, foreign object detection intelligent analysis solutions are gradually relying on image recognition technology to replace manual or detection instruments. In addition, with the development of deep learning technology, methods such as target detection, classification, and segmentation are also gradually applied to the field of foreign object detection. However, at present, the foreign object monitoring solutions in the existing technology have certain defects. For example, due to the large number of foreign objects in the actual environment, the types are not limited, and once foreign objects appear, they will be dealt with in time, so this leads to very few real materials with foreign objects, which in turn affects the accuracy of recognition and its application effect.

[0003] For example, patent document 201811358337.4 discloses a method for detecting foreign objects in power transmission lines based on an improved Faster RCNN. The key points of the design are: preprocessing the collected foreign object images, increasing the size of the training set, improving the shared convolutional network part of Faster RCNN; adjusting the anchor size ratio in the RPN network in Faster RCNN; end-to-end joint training of the improved network; using the trained detection model to detect foreign objects in the image, frame the location of the foreign object and identify the type of foreign object. However, the defect of this solution is that when there is a lack of foreign object materials on site, it will directly lead to a decrease in the recognition effect, that is, the generalization ability of this solution is poor. For example, patent document 201810226499.6 discloses a method and device for detecting foreign objects on rails based on multi-level depth features. The key points of its design are: obtaining the image to be processed, which is a picture of rails taken by a drone; splitting the image to be processed into N sub-images; extracting the multi-level depth features of the N sub-images; judging whether there are foreign objects on the rails in the image to be processed based on the multi-level depth features of the N sub-images. The defect of this scheme is also that when there is a lack of foreign material on site, it will directly lead to a decrease in the recognition effect, that is, the generalization ability of this scheme is poor. For example, patent document 202110430795.X discloses a method and system for identifying foreign objects in a substation based on image registration and target detection. The key points of its design are: S1: input image; S2: image registration; S3: find inconsistent areas; S4: non-foreign object target detection; S5: obtain foreign object areas. This invention first finds the area where the objects appear, and then determines which ones are not foreign objects in the area. The rest are foreign objects. There are not many types of non-foreign objects, and there are many samples. The trained non-foreign object target detection model has a better effect. In addition, for the detected non-foreign objects, after manual re-judgment, the filtered non-foreign object categories can quickly supplement samples to train the non-foreign object target detection model. The defect of this solution is that it belongs to the non-foreign object target detection model, and it is necessary to continuously update and optimize the model according to the actual situation, resulting in the effect of foreign object monitoring being not reliable enough, and occupying more resources, and the search for the difference area is seriously dependent on the accuracy of image registration, that is, a fixed camera needs to be used in actual application, otherwise the effect is not good. At the same time, if the "foreign object" target in the image is large, it will affect the accuracy of image registration.

[0004] Therefore, for those skilled in the art, improving the accuracy of foreign body detection and improving the generalization ability of foreign body detection are issues that need to be addressed. Summary of the invention

[0005] The embodiments of the present invention provide a foreign body detection method, apparatus, computer equipment and storage medium based on image processing, aiming to improve the accuracy of foreign body detection and improve the generalization ability of foreign body detection.

[0006] In a first aspect, an embodiment of the present invention provides a foreign body detection method based on image processing, comprising:

[0007] Acquire a target image to be identified;

[0008] Using an image segmentation model to segment the target image, and obtaining a segmentation mask image corresponding to the target image;

[0009] The segmentation mask image is logically processed, and whether there is a foreign object in the target image is determined according to the result of the logical processing.

[0010] In a second aspect, an embodiment of the present invention provides a foreign body detection device based on image processing, comprising:

[0011] An acquisition unit, used for acquiring a target image to be identified;

[0012] A segmentation unit, used to perform segmentation processing on the target image using an image segmentation model to obtain a segmentation mask image corresponding to the target image;

[0013] The judging unit is used to perform logic processing on the segmentation mask image and judge whether there is a foreign object in the target image according to the result of the logic processing.

[0014] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image processing-based foreign body detection method as described in the first aspect is implemented.

[0015] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the foreign body detection method based on image processing as described in the first aspect is implemented.

[0016] The embodiment of the present invention provides a foreign body detection method, device, computer equipment and storage medium based on image processing, the method comprising: obtaining a target image to be identified; performing segmentation processing on the target image using an image segmentation model to obtain a segmentation mask map corresponding to the target image; performing logical processing on the segmentation mask map, and judging whether the target image contains foreign bodies according to the result of the logical processing. The embodiment of the present invention extracts the segmentation mask map of the target image by performing segmentation processing on the target image, and then performs logical processing on the segmentation mask map to judge whether there are foreign bodies therein, which can solve the problem of insufficient recognition accuracy due to lack of foreign body material, improve the accuracy of foreign body detection, and the foreign body detection method can be applied to different foreign body detections in different scenes, avoiding the problem of over-customization and improving the generalization ability of foreign body detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0019] Figure 2 A schematic block diagram of a foreign body detection device based on image processing provided by an embodiment of the present invention;

[0020] Figure 3 An example diagram of a first image in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of target detection in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0022] Figure 5 A first segmentation example diagram in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0023] Figure 6 An example diagram of a first segmentation mask in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0024] Figure 7 An example diagram of a second image in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0025] Figure 8A second segmentation example diagram in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0026] Fig. 9 An example diagram of a second segmentation mask in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0027] Fig.10 A first detection example diagram in a foreign body detection method based on image processing provided by an embodiment of the present invention;

[0028] Fig.11 This is a second detection example diagram in a foreign matter detection method based on image processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0031] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0032] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0033] See below Figure 1 , an embodiment of the present invention provides a foreign body detection method based on image processing, which specifically includes: steps S101 to S103.

[0034] Step S101, obtaining a target image to be identified;

[0035] Step S102: segmenting the target image using an image segmentation model to obtain a segmentation mask image corresponding to the target image;

[0036] Step S103: performing logic processing on the segmentation mask image, and judging whether there is a foreign object in the target image according to the result of the logic processing.

[0037] In this embodiment, for the target image to be identified, it is segmented by an image segmentation model to segment and extract a segmentation mask map, and then the segmentation mask map is logically processed, for example, to determine whether the number of contours in the segmentation mask map is normal, whether the device position is reasonable, etc., to determine whether there are foreign objects in the segmentation mask map, and then determine whether there are foreign objects in the target image.

[0038] This embodiment extracts the segmentation mask of the target image by segmenting the target image, and then performs logical processing on the segmentation mask to determine whether there is a foreign object therein, which can solve the problem of insufficient recognition accuracy due to the lack of foreign material, improve the accuracy of foreign object detection, and the foreign object detection method can be applied to different foreign object detections in different scenes, avoiding the problem of over-customization and improving the generalization ability of foreign object detection. It should also be noted that the foreign object detection method provided in this embodiment does not directly study the foreign object as in the prior art, but analyzes the location where the foreign object may appear to analyze whether the location and other targets have changed.

[0039] In some optional embodiments, before the step of segmenting the target image using the image segmentation model to obtain a segmentation mask image corresponding to the target image, the step includes:

[0040] Performing target detection on the target image using a target detection model to extract a detection area of ​​the target image;

[0041] Cropping the target image according to the detection area to obtain a cropped image of the target image;

[0042] The cropped image is input into an image segmentation model for segmentation processing.

[0043] In this embodiment, before the target image is segmented, target detection is performed on the target image through a target detection model, and the target image is cropped according to the result of the target detection to obtain a corresponding cropped image, and then the obtained cropped image is segmented through the image segmentation image, which can further improve the final foreign body detection result.

[0044] It is understandable that when the final foreign body detection result is expected to have a higher accuracy, the target detection model and the image segmentation model can be combined to perform preliminary processing on the target image in sequence, and then the target image can be logically processed. When the overall foreign body detection efficiency is expected to be higher, only the image segmentation model can be used to perform preliminary processing on the target image, and then the target image can be logically processed.

[0045] In actual application scenarios, the target detection model can be pre-trained. The network structure of the target detection model is not limited. For example, detection networks such as YoloV4, etc., and the training data is the actual application scene diagram. Specifically, the material of the actual scene (without foreign objects) can be used as the training data. The training method is to use a certain area in the application scene diagram as the target for training.

[0046] In some optional embodiments, the step of performing segmentation processing on the target image using an image segmentation model to obtain a segmentation mask image corresponding to the target image includes:

[0047] The target image is segmented using the RVOS segmentation network to obtain a segmentation mask image corresponding to the target image.

[0048] In this embodiment, when segmenting the target image, the RVOS segmentation network (ReferringVOS) can be used. The RVOS segmentation network is designed to segment the object referred to by the text from a video sequence based on the reference text. Compared with semi-supervised video object segmentation, RVOS only relies on abstract language descriptions rather than pixel-level reference masks.

[0049] In actual application scenarios, other segmentation networks can also be selected to segment the target image, such as DeepLab V3, Unet and other segmentation networks. When training the image segmentation network, the training data is the actual application scene map. Specifically, the material of the actual scene (without foreign objects) can be used as the training data. The training method is to use a certain type of target or multiple types of targets in the application scene map as the foreground, and the remaining targets as the background for training.

[0050] In some optional embodiments, performing logical processing on the segmentation mask image and judging whether there is a foreign object in the target image according to a result of the logical processing includes:

[0051] Acquire parameter information of the foreground area of ​​the segmentation mask image; wherein the parameter information includes shape information and size information;

[0052] Normal scene information of the target image is obtained, and foreign body detection is performed on the segmentation mask image in combination with the normal scene information and parameter information.

[0053] When performing logical processing on the segmentation mask image, this embodiment first obtains parameter information such as shape information and size information of the foreground area of ​​the segmentation mask image, and also obtains normal scene information when there is no foreign object, and then combines the normal scene information and parameter information to determine whether there is a foreign object. In actual application scenarios, it is also possible to combine scene prior knowledge to determine whether there is a foreign object.

[0054] Further, in some optional embodiments, after the step of acquiring normal scene information of the target image and performing foreign body detection on the segmentation mask image in combination with the normal scene information and parameter information, the step further comprises:

[0055] When the target image is in a frame in the video stream, acquiring timing information of the video stream;

[0056] A foreign body detection operation is performed on the segmentation mask image in combination with the timing information, the normal scene information and the parameter information.

[0057] In this embodiment, if the target image to be identified is a single image, the segmentation mask image of the target image can be directly combined with the normal scene information and parameter information to perform foreign object detection. If the target image to be identified is a frame image in a video stream, the segmentation mask image of the target image can be further combined with the video timing information to perform foreign object detection. In this way, the accuracy of foreign object detection can be further improved with the assistance of timing information.

[0058] In a specific embodiment, the acquiring normal scene information of the target image and performing foreign body detection on the segmentation mask image in combination with the normal scene information and parameter information includes:

[0059] Acquire the number, shape and area of ​​contours in the segmentation mask image in combination with the parameter information;

[0060] Determining whether the number, shape and area of ​​contours in the segmentation mask image have changed according to the normal scene information;

[0061] If the number of contours, the shape of contours and the area of ​​contours in the segmentation mask image do not change, it is determined that no foreign matter exists;

[0062] If any one or more of the number of contours, the shape of contours and the area of ​​contours in the segmentation mask image changes, it is determined that a foreign object exists.

[0063] When the present embodiment combines normal scene information and parameter information to detect foreign matter, the number of contours, contour shape and contour area (i.e., the number of pixels in the contour) contained in the segmentation mask map is first extracted according to the parameter information, and then the number of contours, contour shape and contour area under the normal scene are determined according to the normal scene information, and then the number of contours, contour shape and contour area under the normal scene determined by the normal scene information are respectively compared with the number of contours, contour shape and contour area contained in the segmentation mask map to determine which one or several of the three items of the number of contours, contour shape and contour area have changed. If it is finally determined that none of the three items of the number of contours, contour shape and contour area have changed, it is further determined that there is no foreign matter in the segmentation mask map; if it is finally determined that any one or more of the three items of the number of contours, contour shape and contour area have changed, it is further determined that there is a foreign matter in the segmentation mask map.

[0064] For example, if the number of contours in the normal scene information is 6, and the number of contours in the parameter information obtained from the segmentation mask is 7, then the number of contours in the parameter information is greater than the number of contours in the normal scene information, and it can be determined that there are foreign objects in the segmentation mask. For another example, if the number of contours in the normal scene information and the number of contours in the segmentation mask are both 6, but the shape of the contours in the normal scene information and the shape of the contours in the segmentation mask are different, it can also be determined that there are foreign objects in the segmentation mask.

[0065] It is understandable that in actual applications, how to use the three data of contour quantity, contour shape and contour area can be set according to actual conditions. They can be set after "foreign matter-free" image analysis, or they can be determined by averaging continuous video frames, etc.

[0066] Specifically, judging whether the number, shape and area of ​​contours in the segmentation mask image have changed according to the normal scene information includes:

[0067] Performing concave hull detection on the contour shape in the segmentation mask image using a concave hull detection algorithm to obtain a corresponding concave hull detection result;

[0068] When the hull detection result is that there is no hull, it is determined that the contour shape has not changed;

[0069] When the hull detection result indicates that a hull exists, it is determined that the contour shape has changed.

[0070] In this embodiment, when detecting foreign objects based on whether the contour shape has changed, a concave hull detection algorithm can be used to detect whether there is a concave hull in the segmentation mask image. If there is a concave hull, it is considered that the contour shape has changed, and then the foreign object is determined to exist. If there is no concave hull, it is considered that the contour shape has not changed, and then the foreign object is determined to exist. It should be noted that the concave hull detection described in this embodiment is only one of the methods for identifying the contour shape of the segmentation mask image. However, in actual applications, whether the contour shape has changed is not limited to the concave hull detection, and for example, the convex hull detection method can also be used.

[0071] In a practical application scenario, the target detection model is combined to detect foreign objects in the contact network of the railway scene. Figure 3 As shown, first obtain the railway scene image to be identified, then use the YoloV4 network as the target detection model, and perform target detection on the railway scene image to obtain the target position, and then crop the target image according to the position to obtain the cropped image cut_image, as shown in Figure 4 Then the RVOS network is used as the image segmentation model to segment the cropped image cut_image and obtain the segmentation mask map mask, as shown in Figure 5 As shown. Then the segmentation mask is processed logically, that is, the segmentation mask is processed morphologically to reduce noise. Then the segmentation mask is scaled to a uniform scale to extract all contours. Because different components are trained as different categories, only the largest contour is retained for the same category (larger near and smaller far), as shown in Figure 6 As shown, the red outline in the lower right corner is filtered out, and the circumference and area of ​​the retained outline are calculated.

[0072] Here, if the railway scene image is a single image, it can be directly judged whether there is a foreign body. For example, after scaling to a uniform scale, the circumference and area of ​​each contour can be calculated. For example, the circumference of the red part is within a certain range [c1, c2], and the area area also varies within a certain range [a1, a2]. When it is detected that the circumference and / or area area of ​​the red part are not within the corresponding range, it can be determined that there is a foreign body. If the railway scene image is a frame in a video stream, then within a period of time, the information of the previous and next frames is continuous, that is, within a normal (no foreign body) time period, it can be counted that the circumference is within a certain range [c1, c2], and the area area also varies within a certain range [a1, a2]. When the circumference is not within the range or the area is not within the range, and it lasts for T time, it can be determined that there is a foreign body.

[0073] In another practical application scenario, there is no need to combine the target detection model to detect foreign objects on the rails in the railway scene. Figure 7 As shown, first obtain the railway scene image to be identified, and then use the RVOS network as the image segmentation model to segment the cropped image cut_image to obtain the segmentation mask map mask, as shown in Figure 8 and Fig. 9 As shown. Then the segmentation mask is processed logically, that is, the segmentation mask is processed morphologically to reduce noise. Then all contours in the segmentation mask are extracted, and the circumference and area of ​​the contours are calculated. Here, if the railway scene image is a single image, it can be directly analyzed by the number and shape of the contours, such as Fig.10 As shown, there are two rails, so the number of contours is equal to 2. If the number is not equal to 2, there is a foreign object in the output. In addition, according to practical experience, the rail contour should be continuous and smooth, such as Fig.11 As shown, concave hull detection can be performed, and when a concave hull is detected, it can be determined that there is a foreign object. If the railway scene image is a frame in the video stream, the information of the previous and next frames is continuous within a period of time, that is, in the normal (no foreign object) time period, the number of contours contours equals c_num, the number of concave hulls equals bag_num, the circumference is within a certain range [c1, c2], and the area area also changes within a certain range [a1, a2]. If the number of contours changes, the number of concave hulls changes, the circumference is not within the range, or the area is not within the range, and it lasts for T time, then the output is that there is a foreign object.

[0074] Of course, it is understandable that the foreign object detection provided by this embodiment is not limited to railway scenarios, but can also be applied to other scenarios. As long as the training pictures are replaced, the method can be used to detect foreign objects.

[0075] Figure 2 A schematic block diagram of a foreign body detection device 200 based on image processing provided by an embodiment of the present invention, the device 200 includes:

[0076] An acquisition unit 201 is used to acquire a target image to be identified;

[0077] The segmentation unit 202 is used to perform segmentation processing on the target image using an image segmentation model to obtain a segmentation mask image corresponding to the target image;

[0078] The judging unit 203 is used to perform logic processing on the segmentation mask image, and judge whether there is a foreign object in the target image according to the result of the logic processing.

[0079] In some optional embodiments, the foreign body detection device 200 based on image processing further includes:

[0080] A target detection unit, used to perform target detection on the target image using a target detection model to extract a detection area of ​​the target image;

[0081] An image cropping unit, used for cropping the target image according to the detection area to obtain a cropped image of the target image;

[0082] The image input unit is used to input the cropped image into the image segmentation model for segmentation processing.

[0083] In some optional embodiments, the segmentation unit 202 includes:

[0084] The network segmentation unit is used to use the RVOS segmentation network to perform segmentation processing on the target image to obtain a segmentation mask image corresponding to the target image.

[0085] In some optional embodiments, the judging unit 203 includes:

[0086] A parameter acquisition unit, used to acquire parameter information of the foreground area of ​​the segmentation mask image; wherein the parameter information includes shape information and size information;

[0087] The foreign body detection unit is used to obtain normal scene information of the target image, and perform foreign body detection on the segmentation mask image in combination with the normal scene information and parameter information.

[0088] In some optional embodiments, the foreign matter detection unit includes:

[0089] A contour extraction unit, used to obtain the number, shape and area of ​​contours in the segmentation mask image in combination with the parameter information;

[0090] A contour judgment unit, configured to judge whether the number, shape and area of ​​contours in the segmentation mask image have changed according to the normal scene information;

[0091] A first determination unit, configured to determine that no foreign matter exists if the number of contours, the shape of contours, and the area of ​​contours in the segmentation mask image do not change;

[0092] The second determination unit is configured to determine that a foreign object exists if any one or more of the number of contours, the shape of contours, and the area of ​​contours in the segmentation mask image changes.

[0093] In some optional embodiments, the contour judgment unit includes:

[0094] a concave hull detection unit, configured to perform concave hull detection on the contour shape in the segmentation mask image using a concave hull detection algorithm to obtain a corresponding concave hull detection result;

[0095] A third determination unit, configured to determine that the contour shape has not changed when the concave hull detection result indicates that there is no concave hull;

[0096] The fourth determination unit is configured to determine that the contour shape has changed when the hull detection result indicates that a hull exists.

[0097] In some optional embodiments, the foreign body detection device 200 based on image processing further includes:

[0098] A timing acquisition unit, used for acquiring timing information of the video stream when the target image is in a frame in the video stream;

[0099] The second combining unit is used to perform a foreign body detection operation on the segmentation mask image by combining the timing information, the normal scene information and the parameter information.

[0100] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0101] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0102] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0103] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A foreign body detection method based on image processing, characterized in that: include: Acquire a target image to be identified; Using an image segmentation model to segment the target image, and obtaining a segmentation mask image corresponding to the target image; The segmentation mask image is logically processed, and whether there is a foreign object in the target image is determined according to the result of the logical processing.

2. The foreign body detection method based on image processing according to claim 1, characterized in that: Before the step of using the image segmentation model to segment the target image to obtain a segmentation mask image corresponding to the target image, the method further comprises: Performing target detection on the target image using a target detection model to extract a detection area of ​​the target image; Cropping the target image according to the detection area to obtain a cropped image of the target image; The cropped image is input into an image segmentation model for segmentation processing.

3. The foreign body detection method based on image processing according to claim 1, characterized in that: The step of performing segmentation processing on the target image by using the image segmentation model to obtain a segmentation mask image corresponding to the target image includes: The target image is segmented using the RVOS segmentation network to obtain a segmentation mask image corresponding to the target image.

4. The foreign body detection method based on image processing according to claim 1, characterized in that: The logical processing of the segmentation mask image and judging whether the target image contains foreign matter according to the result of the logical processing include: Acquire parameter information of the foreground area of ​​the segmentation mask image; wherein the parameter information includes shape information and size information; Normal scene information of the target image is obtained, and foreign body detection is performed on the segmentation mask image in combination with the normal scene information and parameter information.

5. The foreign body detection method based on image processing according to claim 4 is characterized in that: The acquiring normal scene information of the target image and performing foreign body detection on the segmentation mask image in combination with the normal scene information and parameter information includes: Acquire the number, shape and area of ​​contours in the segmentation mask image in combination with the parameter information; Determining whether the number, shape and area of ​​contours in the segmentation mask image have changed according to the normal scene information; If the number of contours, the shape of contours and the area of ​​contours in the segmentation mask image do not change, it is determined that no foreign matter exists; If any one or more of the number of contours, the shape of contours and the area of ​​contours in the segmentation mask image changes, it is determined that a foreign object exists.

6. The foreign body detection method based on image processing according to claim 5, characterized in that: The determining, according to the normal scene information, whether the number, shape and area of ​​contours in the segmentation mask image have changed includes: Performing concave hull detection on the contour shape in the segmentation mask image using a concave hull detection algorithm to obtain a corresponding concave hull detection result; When the hull detection result is that there is no hull, it is determined that the contour shape has not changed; When the hull detection result indicates that a hull exists, it is determined that the contour shape has changed.

7. The foreign body detection method based on image processing according to claim 1, characterized in that: After the step of acquiring the normal scene information of the target image and performing foreign body detection on the segmentation mask image in combination with the normal scene information and parameter information, the method further comprises: When the target image is in a frame in the video stream, acquiring timing information of the video stream; A foreign body detection operation is performed on the segmentation mask image in combination with the timing information, the normal scene information and the parameter information.

8. A foreign body detection device based on image processing, characterized in that: include: An acquisition unit, used for acquiring a target image to be identified; A segmentation unit, used to perform segmentation processing on the target image using an image segmentation model to obtain a segmentation mask image corresponding to the target image; The judging unit is used to perform logic processing on the segmentation mask image and judge whether there is a foreign object in the target image according to the result of the logic processing.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the foreign body detection method based on image processing as claimed in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the foreign matter detection method based on image processing according to any one of claims 1 to 7 is implemented.

Images