Plastic packaging film defect detection method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a plastic packaging film defect detection method and device, electronic equipment and storage medium, relating to the technical field of object detection, which determines the detection frame of the damaged area in the plastic packaging film image and the damage type of the damaged area by performing object detection on the plastic packaging film image to be detected; performs contour detection based on the size information of the detection frame, the damage type and the plastic packaging film image to determine the damaged contour area in the plastic packaging film image; and determines the defect detection result of the plastic packaging film image based on the damaged contour area. The embodiments of the present application can reduce the interference of small area damage, scratches, wrinkles and bumps, improve the accuracy of the defect detection result, and thus reduce the misjudgment rate of the defect detection result by locating the position and damage type of the damaged area in the plastic packaging film image and determining the defect detection result of the plastic packaging film image based on the damaged contour area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to a plastic packaging film defect detection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] After the production and assembly of the miniature circuit breaker, the miniature circuit breaker is generally packaged with a transparent plastic film. The plastic film has certain moisture-proof and waterproof effects, good wear resistance and tear resistance, and can effectively protect the product from external damage. At the same time, it has good visibility, and consumers can clearly see the product through the film. However, during the automatic packaging process of the miniature circuit breaker, the hot melt machine may not be properly sealed, or the temperature may be too high to cause local film burning, and the packaged film may be damaged due to mutual collision during the product flow on the assembly line. Therefore, the packaged product needs to be removed before packaging to improve the appearance quality of the product packaging. Therefore, after the miniature circuit breaker is packaged, the packaged miniature circuit breaker needs to be damaged.

[0003] The existing miniature circuit breaker packaging damage detection method mainly locates the damage position through contour detection and feature judgment. However, since the packaging material is a transparent plastic film, the white area or the tightly fitted part has high transparency, the damage position is not obvious, the hot melt glue joint state is complex, and the small area damage is not obvious. The scratch, wrinkle and bulge have strong interference, which makes the existing miniature circuit breaker packaging damage detection method difficult to adapt to all damage features and environments, so that the detection result is misjudged. SUMMARY

[0004] The embodiments of the present application provide a plastic packaging film defect detection method and device, electronic equipment and a storage medium to improve the problem of high misjudgment probability of the existing miniature circuit breaker packaging damage detection method.

[0005] In one aspect, the embodiments of the present application provide a plastic packaging film defect detection method, which comprises:

[0006] Performing target detection on a plastic packaging film image to be detected to determine a detection frame of a damaged area in the plastic packaging film image and a damage type of the damaged area;

[0007] Performing contour detection based on size information of the detection frame, the damage type and the plastic packaging film image to determine a damaged contour area in the plastic packaging film image;

[0008] Determining a defect detection result of the plastic packaging film image based on the damaged contour area.

[0009] In another aspect, the embodiments of the present application provide a plastic packaging film defect detection device, which comprises:

[0010] a breakage detection module configured to perform target detection on the plastic packaging film image to be detected, determine a detection frame of a breakage area in the plastic packaging film image and a breakage type of the breakage area;

[0011] a contour detection module configured to perform contour detection based on size information of the detection frame, the breakage type and the plastic packaging film image, and determine a breakage contour area in the plastic packaging film image;

[0012] a defect detection module configured to determine a defect detection result of the plastic packaging film image based on the breakage contour area.

[0013] In another aspect, an electronic device is provided, including a memory and a processor; the memory stores an application program, and the processor is configured to run the application program in the memory to perform the operations in the above plastic packaging film defect detection method.

[0014] In another aspect, a storage medium is provided, which stores a plurality of instructions suitable for being loaded by a processor to perform the steps in the above plastic packaging film defect detection method.

[0015] The embodiments of the present application provide a plastic packaging film defect detection method and device, an electronic device and a storage medium, and relate to the technical field of target detection. The position of a breakage area in a plastic packaging film image and a breakage type of the breakage area are determined by performing target detection on the plastic packaging film image to be detected. A breakage contour area in the plastic packaging film image is determined by performing contour detection based on size information of a detection frame, the breakage type and the plastic packaging film image. A defect detection result of the plastic packaging film image is determined based on the breakage contour area. The embodiments of the present application can reduce the interference of small-area breakage, scratches, wrinkles and bumps, improve the accuracy of the defect detection result, and thus reduce the misjudgment rate of the defect detection result. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0017] Figure 1 is an application scenario diagram of the plastic packaging film defect detection method provided by the embodiments of the present application;

[0018] Figure 2FIG. 1 is a flowchart of a plastic packaging film defect detection method provided by an embodiment of the present application;

[0019] Figure 3 FIG. 2 is a schematic diagram of a plastic packaging film image to be detected by different damage types provided by an embodiment of the present application;

[0020] Figure 4 FIG. 3 is a flowchart of a target detection method provided by an embodiment of the present application;

[0021] Figure 5 FIG. 4 is a structural diagram of a feature extractor provided by an embodiment of the present application;

[0022] Figure 6 FIG. 5 is a structural diagram of a 3*3 inverted linear bottleneck network provided by an embodiment of the present application;

[0023] Figure 7 FIG. 6 is a structural diagram of a feature extraction layer provided by an embodiment of the present application;

[0024] Figure 8 FIG. 7 is a schematic diagram of image feature fusion of different scales provided by an embodiment of the present application;

[0025] Figure 9 FIG. 8 is another schematic diagram of image feature fusion of different scales provided by an embodiment of the present application;

[0026] Figure 10 FIG. 9 is a structural diagram of a detection model provided by an embodiment of the present application;

[0027] Figure 11 FIG. 10 is another structural diagram of a detection model provided by an embodiment of the present application;

[0028] Figure 12 FIG. 11 is a flowchart of damage contour area calculation provided by an embodiment of the present application;

[0029] Figure 13 FIG. 12 is a flowchart of a contour detection method provided by an embodiment of the present application;

[0030] Figure 14 FIG. 13 is a flowchart of another plastic packaging film defect detection method provided by an embodiment of the present application;

[0031] Figure 15 FIG. 14 is a structural diagram of a plastic packaging film defect detection device provided by an embodiment of the present application;

[0032] Figure 16 FIG. 15 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0033] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0034] Moreover, described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware- specific instructions, user inputs, etc., to provide a thorough understanding of examples of the present application. One skilled in the relevant art will recognize, however, that the

[0035] The block diagrams in the drawings show only the functionality of the embodiments and do not imply any particular physical or architectural arrangement of the devices, systems, or methods. No inference should be drawn regarding the

[0036] The flow diagrams depicted herein merely provide an example of the methods that can be implemented in connection with the embodiments. There can be many variations to the logic depicted in these flow diagrams, including implementing the variations in parallel, or varying the order of the logic steps. Further, not all of the steps can be required, and in some instances, additional steps can be provided. Also, the steps can be combined or further subdivided into additional steps.

[0037] It should be noted that "a plurality" refers to two or more. The conjunction "and / or" describes an associative relationship between associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0038] As described in the background, in the process of automatic packaging of small circuit breakers, due to the fact that the hot melt machine is not sealed well, or the temperature is too high, causing the local film to be burned through, and the packaged film is damaged due to mutual collision during the process of flowing on the assembly line, and due to the fact that the packaging material plastic film is transparent, the white area or the tightly fitted part has high permeability, the damaged position is not obvious, the state of the hot melt glue joint is complex, the small area damage is not obvious, the scratch, wrinkle and bulge are strong interference, the existing detection method can identify large area damage, but for small area damage and hot melt glue joint damage, there is false detection and missed detection, so that the detection result exists false judgment.

[0039] Based on this, in order to improve the high false positive rate of existing miniature circuit breaker packaging damage detection methods and enhance the applicability of defect detection methods, this application provides a plastic sealing film defect detection method, device, electronic device, and storage medium. By locating the position and damage type of the damaged area in the plastic sealing film image and determining the defect detection result of the plastic sealing film image based on the damaged contour area, the interference of small-area damage, scratches, wrinkles, and bulges can be reduced, improving the accuracy of defect detection results and thus reducing the false positive rate of defect detection results.

[0040] To facilitate understanding of the technical solution of this application, the method for detecting defects in plastic sealing films provided in the embodiments of this application will be introduced below in conjunction with actual application scenarios.

[0041] For example, taking the plastic-encapsulated film of a miniature circuit breaker as an example, the application scenario of the plastic-encapsulated film defect detection method is provided. Please refer to [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic diagram of an application scenario for the plastic film defect detection method provided in this application embodiment. The application scenarios shown include a conveyor belt, a sorting device, a plastic-sealed circuit breaker, and a plastic film defect detection system.

[0042] The system includes a conveyor belt for transporting the plastic-encapsulated circuit breakers; a plastic film defect detection system equipped with the plastic film defect detection method provided in this application, used to detect defects in the plastic film of the plastic-encapsulated circuit breakers on the conveyor belt and obtain defect detection results; and a sorting device for sorting the plastic-encapsulated circuit breakers, for example, sorting plastic-encapsulated circuit breakers with defects as indicated by the defect detection results to a defective product conveyor belt for resealing, and sorting plastic-encapsulated circuit breakers without defects as indicated by the defect detection results to a good product conveyor belt. The defective plastic-encapsulated circuit breaker may have at least one of the following defects: small-area damage, large-area damage, hot melt adhesive seam damage, or elongated damage on the plastic film.

[0043] For example, such as Figure 1 As shown, the plastic sealing film defect detection system is installed on the conveyor belt to detect defects in the plastic sealing film of the circuit breakers already sealed on the conveyor belt. Figure 1 As shown, the plastic sealing film defect detection system includes an image sensor, a detection unit, and a communication unit. The image sensor is used to acquire images of the plastic sealing film of a circuit breaker already sealed on a conveyor belt, obtaining an image of the plastic sealing film to be detected. The detection unit is equipped with the plastic sealing film defect detection method provided in this embodiment, which is used to execute the plastic sealing film defect detection method provided in this embodiment on the image of the plastic sealing film to be detected, obtaining a defect detection result for the plastic sealing film image. The communication unit is used to transmit the defect detection result of the plastic sealing film image to a sorting device.

[0044] In some implementations, such as Figure 1 As shown, the detection unit includes a target detection subunit, a contour detection subunit, and a result output subunit. The target detection subunit performs target detection on the plastic wrapping film image to be inspected, determining the detection bounding box of the damaged area and the damage type of the damaged area. The contour detection subunit performs contour detection based on the size information of the detection bounding box, the damage type, and the plastic wrapping film image, determining the area of ​​the damaged contour in the plastic wrapping film image. The result output subunit determines the defect detection result of the plastic wrapping film image based on the area of ​​the damaged contour.

[0045] Specifically, in this application scenario, the defect detection steps for the molding film include:

[0046] (1) The encapsulated circuit breaker is transported to the encapsulation film defect detection system via a conveyor belt. When the encapsulation film defect detection system detects that the circuit breaker is in place, it uses the image sensor in the encapsulation film defect detection system to collect the image of the encapsulated circuit breaker, obtains the image of the encapsulation film to be detected, and transmits the image of the encapsulation film to be detected to the detection unit.

[0047] (2) The target detection subunit in the detection unit performs target detection on the plastic film image to be detected, determines the detection box of the damaged area in the plastic film image and the damage type of the damaged area, and transmits the detection box of the damaged area in the plastic film image and the damage type of the damaged area to the contour detection subunit; the contour detection subunit performs contour detection based on the size information of the detection box, the damage type and the plastic film image, determines the damaged contour area in the plastic film image, and transmits the damaged contour area to the result output subunit; the result output subunit determines the defect detection result of the plastic film image based on the damaged contour area, and transmits the defect detection result of the plastic film image to the communication unit.

[0048] (3) The communication unit sends the defect detection results to the sorting device.

[0049] (4) Based on the defect detection results, the sorting device sorts circuit breakers with defects in the plastic film to the defective product conveyor belt, and sorts circuit breakers without defects in the plastic film to the good product conveyor belt.

[0050] Optionally, the image sensor can be a CCD sensor or an industrial camera.

[0051] The application scenario provided in this application provides that by locating the position and type of damage in the damaged area of ​​the plastic film image, and determining the defect detection result of the plastic film image based on the damaged contour area, the interference of small-area damage, scratches, wrinkles and bulges can be reduced, the accuracy of defect detection results can be improved, and the misjudgment rate of defect detection results can be reduced.

[0052] based on Figure 1 As shown in the application scenario, this application provides a method for detecting defects in plastic sealing films, such as... Figure 2 As shown, Figure 2 This is a schematic flowchart of a plastic sealing film defect detection method provided in an embodiment of this application. The plastic sealing film defect detection method shown can be performed by... Figure 1 The plastic film defect detection system shown can also be executed by electronic devices with data processing capabilities, such as servers, server clusters, cloud servers, etc. This application embodiment does not specifically limit this. Specifically, Figure 2 The method for detecting defects in plastic sealing films shown includes at least steps 210 to 230, which are described in detail below:

[0053] Step 210: Perform target detection on the plastic film image to be detected, and determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

[0054] The image of the plastic wrapping film to be inspected can be an image of the plastic wrapping film of a product that has already been plastic-wrapped. These products include, but are not limited to, electronic products, food, and logistics products. Electronic products include, but are not limited to, circuit breakers, electricity meters, mobile terminals, sensors, and wearable devices.

[0055] In some embodiments, damage to the plastic film of a sealed product includes large-area damage, small-area damage, cracks at the hot melt adhesive seams, and elongated tears. The types of damage include, but are not limited to, large-area damage, small-area damage, cracks at the hot melt adhesive seams, and elongated tears. For example, taking a circuit breaker as an example... Figure 3 As shown, Figure 3 These are schematic diagrams of images of plastic sealant films to be inspected for different types of damage, provided in embodiments of this application. Figure 3 Figure (a) shows extensive damage. Figure 3 Figure (b) shows a small area of ​​damage. Figure 3 Figure (c) shows the crack at the hot melt adhesive seam. Figure 3 Figure (d) shows a slender tear. Considering that slender tears include both large-area and small-area slender tears, in order to improve the detection efficiency of the plastic sealing film defect detection method, the tear type can include large-area tears, small-area tears, and cracks at the hot melt adhesive seams.

[0056] In some implementations, the image of the plastic wrapping film to be detected can be a real-time captured image of the plastic wrapping film of the plastic-wrapped product, or it can be an image of the plastic wrapping film of the plastic-wrapped product sent by other devices.

[0057] In some implementations, the damaged area in the plastic wrapping film image to be inspected can be determined by contour detection. Based on the position information of the damaged area in the plastic wrapping film image to be inspected, the image area corresponding to the damaged area can be obtained. The damaged area can then be identified to determine the damage type of the damaged area.

[0058] Optionally, an edge detection operator can be used to perform contour detection on the image of the plastic wrapping film to be detected, identifying the damaged areas in the image. A minimum bounding box containing the damaged area is then drawn, centered on the center point of the damaged area, to obtain the detection bounding box for the damaged area. The center point can be either the geometric center or the actual center of the damaged area. Edge detection operators include, but are not limited to, the Sobel operator, Isotropic Sobel operator, Roberts operator, Prewitt operator, Laplacian operator, and Canny operator.

[0059] Optionally, image features can be extracted from the image region to obtain its image features. Based on these image features, a preset feature library can be queried to determine the damage type of the damaged region. The preset feature library includes multiple damage types and corresponding image feature clusters for each damage type, with each image feature cluster containing at least one image feature. These image features include, but are not limited to, edge features, grayscale features, contrast features, and texture features.

[0060] For example, the image features of the image region can be compared with image feature clusters in a preset feature library to determine the target image feature cluster with the highest similarity to the image features of the image region, and the damage type corresponding to the target image feature cluster can be determined as the damage type of the damaged region.

[0061] In some implementations, the image of the plastic wrapping film to be detected can be input into a preset detection model for target detection, determining the detection bounding box of the damaged area in the plastic wrapping film image and the type of damage to the damaged area. The detection model can be a neural network-based detection model or a machine learning-based detection model.

[0062] Step 220: Based on the size information of the detection frame, the damage type, and the image of the plastic film, perform contour detection to determine the area of ​​the damaged contour in the plastic film image.

[0063] Considering that existing damage detection methods have low accuracy in identifying small damaged areas, wrinkles, and areas with weak exposure, resulting in a high probability of false positives or false negatives, the accuracy of defect detection results is reduced. Therefore, to improve the accuracy of defect detection results, in some implementations, it can be determined whether the damaged area meets preset contour area calculation conditions based on the damage type. When the damaged area meets the preset contour area calculation conditions, the plastic film image is cropped according to the size information of the detection frame to obtain a cropped image. Contour recognition is performed on the cropped image to obtain the damaged contour and its area. The detection results of the target detection are then filtered using the preset contour area calculation conditions. Contour calculation is performed on the filtered damaged areas to obtain the damaged contour area. The defect detection result is determined based on the damaged contour area. In this way, through two identifications—the contour area calculation conditions and the damaged contour area—the probability of false positives and false negatives is reduced.

[0064] Optionally, the conditions for calculating the contour area can be that the damage type is small-area damage or a tear at the hot melt adhesive seam. Understandably, when the damage type is small-area damage or a tear at the hot melt adhesive seam, the plastic film image is cropped according to the size information of the detection frame to obtain the cropped image. Contour recognition is then performed on the cropped image to obtain the damage contour and its area. When the damage type is large-area damage, the defect detection result of the plastic film image is determined to indicate the presence of a defect.

[0065] Optionally, the cropped image can be contour-recognized using an edge detection operator to obtain the damaged contour.

[0066] Optionally, considering that the damaged contour may be irregular in shape and the area of ​​the contour is difficult to calculate, in order to facilitate the calculation of the area of ​​the contour, after obtaining the damaged contour, the damaged contour can be fitted, and the area of ​​the fitted damaged contour can be calculated to obtain the area of ​​the damaged contour in the plastic sealing film image. For example, the damaged contour can be fitted with an ellipse, a circle, or a rectangle, etc., and this application embodiment does not specifically limit this.

[0067] Step 230: Determine the defect detection result of the plastic film image based on the damaged contour area.

[0068] In some implementations, the area of ​​the damaged contour can be compared with a preset area threshold; if the area of ​​the damaged contour is greater than or equal to the preset area threshold, the defect detection result of the plastic wrapping film image is determined to be that there is a defect in the plastic wrapping film image; if the area of ​​the damaged contour is less than the preset area threshold, the defect detection result of the plastic wrapping film image is determined to be that there is no defect in the plastic wrapping film image.

[0069] The plastic film defect detection method provided in this application can reduce interference from small-area damage, scratches, wrinkles and bulges, improve the accuracy of defect detection results, and thus reduce the misjudgment rate of defect detection results by locating the position and type of damage area in the plastic film image and determining the defect detection result based on the area of ​​the damage contour.

[0070] In some implementations, considering that existing defect detection methods have a high probability of missing or false detections of small-area damage and cracks at hot melt adhesive seams, multi-scale feature extraction is performed on the plastic film image to be detected to obtain image features of different receptive fields. These features are then fused and used for feature calculation to determine the detection bounding box and damage type of the damaged area in the plastic film image. This increases the richness of the features, ensuring the accuracy of the final recognition result and reducing the probability of missing or false detections of small-area damage and cracks at hot melt adhesive seams. Here, scale refers to the resolution of the plastic film image.

[0071] Specifically, such as Figure 4 As shown, Figure 4 This is a flowchart illustrating the target detection method provided in an embodiment of this application. The target detection method shown includes steps 211 to 213:

[0072] Step 211: Perform multi-scale feature extraction on the image of the plastic sealing film to be detected to obtain image features at different scales.

[0073] In some implementations, the resolution of the plastic wrapping film image to be detected can be adjusted to obtain plastic wrapping film images at different resolutions. Feature extraction can then be performed on the plastic wrapping film images at different resolutions to obtain image features at different scales. For example, an edge detection operator can be used to extract contour features from the plastic wrapping film images at different resolutions to obtain contour features at different scales.

[0074] In some implementations, to extract as many image features as possible from the image of the plastic sealing film to be detected, a preset feature extractor can be used to perform multi-scale feature extraction on the image of the plastic sealing film to be detected, obtaining image features at different scales. The feature extractor can be a neural network-based feature extractor, such as a convolutional network-based feature extractor, a YOLO-based feature extractor, or a depthwise separable convolution-based feature extractor.

[0075] Optionally, the feature extractor can include feature extraction layers of different sizes. These layers allow for multi-scale feature extraction of the plastic film image to be detected, yielding image features at different scales. For example, the feature extractor may include seven feature extraction layers of different sizes.Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of the feature extractor provided in the embodiment of this application. The feature extractor shown includes a series of input layer, first downsampling layer, second downsampling layer, third feature extraction layer, fourth feature extraction layer, fifth feature extraction layer, sixth feature extraction layer and seventh feature extraction layer.

[0076] The resolutions of the first, second, third, fourth, fifth, sixth, and seventh feature extraction layers are 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128, respectively. The first and second downsampling layers are used to downsample the image of the plastic sealing film to be detected, while the third, fourth, fifth, sixth, and seventh feature extraction layers are used for feature extraction.

[0077] In some implementations, the third, fourth, fifth, sixth, and seventh feature extraction layers have similar structures, all including convolutional networks and inverted linear bottleneck networks. For example, an inverted linear bottleneck network with 3x3 convolutional kernels is used. Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a 3*3 inverted linear bottleneck network provided in an embodiment of this application. The inverted linear bottleneck network shown includes a first convolutional unit 601, a depthwise separable convolutional unit 602, and a second convolutional unit 603. The first convolutional unit 601 includes a cascaded 1*1 convolutional network, a normalization module, and an activation function. The depthwise separable convolutional unit 602 includes a cascaded 3*3 depthwise separable convolutional network, a normalization module, and an activation function. The second convolutional unit 603 includes a cascaded 1*1 convolutional network and a normalization module. When performing feature extraction, the inverted linear bottleneck network inputs the image features into the first convolutional unit 601 for convolution calculation, and then inputs them into the depthwise separable convolutional unit 602. The depthwise separable convolutional unit 602 calculates the input features and then inputs them into the second convolutional unit 603. The second convolutional unit 603 performs convolution calculation on the features output by the depthwise separable convolutional unit 602 to obtain the output features. The inverted linear bottleneck network then superimposes the output features of the second convolutional unit 603 with the input features of the first convolutional unit 601 as the output of the inverted linear bottleneck network.

[0078] like Figure 7 As shown, Figure 7This is a schematic diagram of the feature extraction layer provided in the embodiments of this application. The feature extraction layer shown includes a first convolutional network, a first inverted linear bottleneck network, a second inverted linear bottleneck network, a third inverted linear bottleneck network, a fourth inverted linear bottleneck network, a fifth inverted linear bottleneck network, a sixth inverted linear bottleneck network, a seventh inverted linear bottleneck network, a second convolutional network, a pooling network, and a fully connected network, all connected in series.

[0079] Step 212: Perform feature fusion on image features at different scales to obtain fused image features.

[0080] In some implementations, image features at different scales can be fused layer by layer to obtain fused image features corresponding to each scale.

[0081] Optionally, the highest-scale image features can be used as the first fused image features at the highest scale. Starting from the second-highest scale, the image features at each scale and the first fused image features at the previous scale adjacent to that scale are fused layer by layer to obtain the first fused image features at each scale. The first fused image features at the highest scale can be used as the second fused image features at the highest scale. Starting from the second-highest scale, the first fused image features at each scale and the second fused image features at the previous scale adjacent to that scale are fused layer by layer to obtain the second fused image features at each scale. The second fused image features at each scale are determined as the fused image features.

[0082] For example, let's take image features at five scales as an example for illustration, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of image feature fusion at different scales provided in the embodiments of this application, such as... Figure 8 As shown in Figure (a), the image features at the P7 scale are used as the first fusion feature at the P7 scale. The first fusion image features at the P7 scale are fused with the image features at the P6 scale to obtain the first fusion image features at the P6 scale. The first fusion image features at the P6 scale are fused with the image features at the P5 scale to obtain the first fusion image features at the P5 scale. The first fusion image features at the P5 scale are fused with the image features at the P4 scale to obtain the first fusion image features at the P4 scale. The first fusion image features at the P4 scale are fused with the image features at the P3 scale to obtain the first fusion image features at the P3 scale.

[0083] Optionally, image features at each scale can be fused with image features at the remaining scales to obtain first fused image features at each scale. The first fused image features at each scale can be fused with the first fused image features at the remaining scales to obtain second fused image features at each scale. The second fused image features at each scale can be determined as fused image features.

[0084] For example, let's take image features at five scales as an example for illustration, such as... Figure 8 As shown in Figure (b), the image features at scale P7 are fused with the corresponding image features at scales P6, P5, P4, and P3 to obtain the first fused image feature at scale P7; the image features at scale P6 are fused with the corresponding image features at scales P7, P5, P4, and P3 to obtain the first fused image feature at scale P6; the image features at scale P5 are fused with the corresponding image features at scales P7, P6, P4, and P3 to obtain the first fused image feature at scale P5; the image features at scale P4 are fused with the corresponding image features at scales P7, P6, P5, and P3 to obtain the first fused image feature at scale P4; and the image features at scale P3 are fused with the corresponding image features at scales P7, P6, P5, and P4 to obtain the first fused image feature at scale P3.

[0085] Optionally, starting from the highest scale, the image features of each scale and the image features of the previous scale adjacent to each scale can be fused layer by layer to obtain the first fused image features of each scale. Starting from the lowest scale, the second fused image features of each scale can be fused layer by layer to obtain new image features of each scale.

[0086] For example, let's take image features at five scales as an example for illustration, such as... Figure 8 As shown in Figure (c), image features at scale P6 are fused with image features at scale P7 to obtain a first fused image feature at scale P6. This first fused image feature at scale P6 is then fused with image features at scale P5 to obtain a first fused image feature at scale P5. The first fused image feature at scale P4 is then fused with image features at scale P3 to obtain a first fused image feature at scale P3. This first fused image feature at scale P3 is then used as a second fused image feature at scale P3. The first fused image feature at scale P3 is then fused with the first fused image feature at scale P4 to obtain a second fused image feature at scale P4. The second fused image feature at scale P4 is then fused with the first fused image feature at scale P5 to obtain a second fused image feature at scale P5. The second fused image feature at scale P5 is then fused with the first fused image feature at scale P6 to obtain a second fused image feature at scale P6. Finally, the second fused image feature at scale P6 is fused with image features at scale P7 to obtain a second fused image feature at scale P7.

[0087] In some implementations, to further enhance the feature richness of the fused image features, a residual concept can be introduced into feature fusion. In feature fusion at each scale, starting from the second-highest scale, the image features of each scale are fused layer by layer with the first fused image features of the adjacent previous scale to obtain the first fused image features of each scale. The image features of each intermediate scale are fused with the first fused image features of that scale to obtain the second fused image features of each intermediate scale. The intermediate scales are the remaining scales excluding the highest and lowest scales. The image features of the highest scale are determined as the second fused image features of the highest scale, and the first fused image features of the lowest scale are determined as the second fused image features of the lowest scale. Starting from the lowest scale, the second fused image features of the lowest scale, the second fused image features of each intermediate scale, and the second fused image features of the highest scale are fused layer by layer to obtain new image features at each scale. Starting from the second-highest scale, the new image features of each scale are fused layer by layer with the new first fused image features of the adjacent previous scale to obtain new first fused image features at each scale. This process is repeated until the number of times the image features are updated is greater than or equal to a preset threshold. At this point, the new image features at each scale are determined as fused image features.

[0088] For example, taking an image feature with a preset threshold of 1 and 5 scales as an example, the explanation is as follows: Figure 9 As shown, Figure 9 This is a schematic diagram of another image feature fusion at different scales provided in this application embodiment. Image features at scale P7 are used as the first fused image feature at scale P7. The image features at scale P7 are fused with image features at scale P6 to obtain the first fused image feature at scale P6. The first fused image feature at scale P6 is fused with image features at scale P5 to obtain the first fused image feature at scale P5. The first fused image feature at scale P5 is fused with image features at scale P4 to obtain the first fused image feature at scale P4. The first fused image feature at scale P4 is fused with image features at scale P3 to obtain the first fused image feature at scale P3. The image features at scale P6 are fused with the first fused image features at scale P6... The image features are fused to obtain the second fused image features at the P6 scale. The image features at the P5 scale are fused with the first fused image features at the P5 scale to obtain the second fused image features at the P5 scale. The image features at the P4 scale are fused with the first fused image features at the P4 scale to obtain the second fused image features at the P4 scale. The first fused image features at the P3 scale are used as the second fused image features at the P3 scale, and the image features at the P7 scale are determined as the second fused image features at the P7 scale. Starting from the P3 scale, the second fused image features at each scale (P3, P4, P5, P6, P7) are fused layer by layer to obtain new image features at each scale (P3, P4, P5, P6, P7).

[0089] In some implementations, image features at different scales can be input into a preset feature fusion unit for feature fusion to obtain fused image features. The feature fusion unit can be a fully connected network.

[0090] Step 213: Perform feature calculation based on the fused image features to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

[0091] In some implementations, the fused image features can be input into a preset locator and classifier for feature calculation to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

[0092] The preset locator can be a regression network, and the classifier can be a classification network.

[0093] In some implementations, to facilitate target detection, the image of the plastic film to be detected can be input into a preset detection model for target detection, thereby obtaining the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

[0094] The preset detection model is obtained by training the EfficientDet network based on sample images.

[0095] For example, such as Figure 10 As shown, Figure 10 This is a schematic diagram of the detection model provided in the embodiments of this application. The detection model shown includes an input layer, a feature extractor, a feature fusionist, a locator, a classifier, and an output layer.

[0096] The process involves inputting the image of the plastic film to be detected into the input layer, inputting the output of the input layer into the feature extractor, inputting the output of the feature extractor into the feature fusion unit, inputting the output of the feature fusion unit into the locator and classifier respectively, and inputting the outputs of the locator and classifier into the output layer respectively.

[0097] In some implementations, the feature fusion unit includes multiple feature fusion units. Each feature fusion unit performs top-down feature fusion on image features at each scale to obtain first fused image features at each scale, and then performs bottom-up feature fusion on the first fused image features at each scale to obtain new image features at each scale, such as... Figure 9 As shown.

[0098] In some implementations, to reduce network complexity while improving the accuracy of target detection results, the feature extraction layers in the feature extractor input the extracted image features to the next feature extraction layer and the feature fusion unit, respectively. Understandably, the highest-scale feature extraction layer inputs its extracted image features to the feature fusion unit. For example, consider a feature fusion unit comprising three feature fusion units, such as... Figure 11 As shown, Figure 11 This is another structural diagram of the detection model provided in the embodiments of this application. The outputs of the third feature extraction layer, the fourth feature extraction layer, the fifth feature extraction layer, and the sixth feature extraction layer are respectively input to the first feature fusion unit and the next feature extraction layer. The output of the seventh feature extraction layer is input to the first feature fusion unit. The first feature fusion unit performs feature fusion to obtain new image features at each scale. After the new image features at each scale are input to the second feature fusion unit for fusion processing, the output of the second feature fusion unit is input to the third feature fusion unit. The output of the third feature fusion unit is input to the classifier and the localizer respectively.

[0099] In some implementations, to improve the detection performance of the preset detection model and ensure the accuracy of the detection bounding box and damage type of the damaged area in the plastic film image, it is necessary to train the EfficientDet network based on sample images to obtain the detection model. The EfficientDet network and... Figure 10 The detection model shown, or Figure 11 The detection models shown have similar structures.

[0100] Optionally, both damaged and undamaged sample images can be collected. The damaged sample images are labeled with the actual damaged areas and the actual damaged types. The undamaged sample images and the labeled damaged sample images are used as a sample dataset. The EfficientDet network is trained using the sample dataset to obtain the detection model.

[0101] Considering the significant differences in the degree of damage to the plastic film and the large differences in the aspect ratio of the detection box, in order to detect the damaged area more accurately, in some implementations, prior boxes of different sizes can be set in the feature extractor of the detection model. In target detection, the cross-union ratio of each prior box is determined, and the prior box with the highest cross-union ratio is used as the detection box for the damaged area in the plastic film image.

[0102] Optional, such as Figure 11As shown, the feature extractor includes network layers of different scales. From P1 to P7, as the receptive field gradually increases, the feature extractor becomes more and more sensitive to large-area damage and less and less sensitive to small-area damage. Therefore, in order to further improve the accuracy of the detection model, different sizes of prior boxes are set for network layers of different scales.

[0103] For example, the aspect ratios of the prior boxes in the first and second downsampling layers can be 1:1 and 2:1, respectively; the aspect ratios of the prior boxes in the third, fourth, and fifth feature extraction layers can be 1:1, 1:2, 2:1, 3:1, and 1:3; and the aspect ratios of the prior boxes in the sixth and seventh feature extraction layers can be 1:1, 3:1, 1:3, 4:1, and 5:1. It should be noted that the aspect ratios of the prior boxes in the above network layers are merely illustrative and do not constitute a method for detecting defects in plastic-sealed films provided in this application. The aspect ratios of the prior boxes in each network layer can be set based on actual application scenarios.

[0104] In some implementations, after determining the detection frame of the damaged area and the damage type of the damaged area in the plastic wrapping film image, it can be determined whether the damaged area meets the preset contour area calculation conditions based on the size information of the detection frame and the damage type. When the damaged area meets the preset contour area calculation conditions, contour detection is performed on the plastic wrapping film image to determine the damaged contour area in the plastic wrapping film image.

[0105] Specifically, such as Figure 12 As shown, Figure 12 This is a schematic flowchart illustrating the calculation of the damaged contour area according to an embodiment of this application. The method for calculating the damaged contour area includes steps 221 to 225:

[0106] Step 221: Based on the size information and damage type of the detection frame, determine whether the damaged area meets the preset contour area calculation conditions.

[0107] In some implementations, considering that for large-area damage, the damaged area is already very large and area screening is unnecessary, while for small-area damage that is larger than the actual area (the detection frame does not completely fit the damaged area but extends outward by 3-5 pixels, especially for long and thin damaged areas tilted at 45°, where the circumscribed rectangle is much larger than the actual damaged area), using this area as the damage area for screening would lead to many false positives. Therefore, to improve the accuracy of defect detection results and reduce the probability of missed and false positives, the size information of the detection frame can be compared with a preset size threshold. When the size information of the detection frame is less than or equal to the preset size threshold, and the damage type is a preset target type, it indicates that the damaged area may be a small-area damage or a crack in the hot melt adhesive seam. Further screening based on the damaged outline area is needed to improve the accuracy of defect detection results and reduce the probability of missed and false positives. The preset target types include small-area damage and cracks in the hot melt adhesive seam.

[0108] Specifically, step 221 includes steps a1 to a3:

[0109] Step a1: Compare the size information of the detection frame with the preset size threshold.

[0110] Step a2: If the size information of the detection box is less than or equal to the preset size threshold, and the damage type is the preset target type, then the damaged area is determined to meet the preset contour area calculation conditions.

[0111] Step a3: If the size information of the detection box is greater than the preset size threshold, or the damage type is not the preset target type, then it is determined that the damaged area does not meet the preset contour area calculation conditions.

[0112] Step 222: When the damaged area meets the preset contour area calculation conditions, the plastic film image is processed according to the size information of the detection frame to obtain the processed image.

[0113] In some implementations, image processing may be image cropping.

[0114] Optionally, the image of the plastic film can be cropped according to the size information of the detection frame to obtain the processed image.

[0115] Optionally, the detection frame can be enlarged based on its size information to obtain an enlarged detection frame. The plastic film image can then be cropped based on the enlarged detection frame to obtain the processed image.

[0116] For example, the detection frame can be enlarged based on its size information and a preset magnification ratio to obtain an enlarged detection frame.

[0117] Step 223: Perform contour detection on the processed image to obtain candidate damaged contours in the processed image.

[0118] In some implementations, contour detection can be performed on the processed image based on an edge detection operator to obtain candidate damaged contours.

[0119] In some implementations, to improve the accuracy of the calculated area of ​​the damaged contour, the processed image can be preprocessed to obtain a preprocessed image, and contour detection can be performed on the preprocessed image to obtain candidate damaged contours in the processed image.

[0120] Image preprocessing includes, but is not limited to, image enhancement and grayscale transformation. Image enhancement includes grayscale enhancement, contrast enhancement, and gamma enhancement.

[0121] Considering that small damaged areas, wrinkles, and other similar damaged areas, as well as areas with weak exposure, are prone to missed detection or false detection, in order to improve the accuracy of defect detection results, gamma transformation can be performed on the processed image in contour detection. By increasing the exposure value of the processed image, the image quality of the processed image is improved, thereby improving the accuracy of contour detection and thus improving the accuracy of defect detection results.

[0122] Specifically, such as Figure 13 As shown, Figure 13 This is a schematic flowchart of the contour detection method provided in the embodiments of this application. The contour detection method shown includes steps 2231 to 2234:

[0123] Step 2231: Perform a gamma transform on the processed image to obtain the transformed image.

[0124] Step 2232: Perform edge detection on the transformed image to obtain the edge contour image of the transformed image.

[0125] In some implementations, edge detection operators can be used to perform edge detection on the transformed image to obtain the edge contour image of the transformed image.

[0126] Step 2233: Perform dilation and erosion processing on the edge contour image to obtain a binary image.

[0127] In some implementations, the edge contour image can be dilated to fill the holes in the edge contour image, resulting in a dilated image. The dilated image can then be eroded to restore the damaged areas, resulting in a binary image.

[0128] Step 2234: Based on binary image row ellipse fitting, candidate damaged contours are obtained in the processed image.

[0129] In some implementations, ellipse fitting can be performed on pixels with preset pixel values ​​in the binary image to obtain candidate damaged contours in the processed image.

[0130] Step 224: Determine the damaged contour from the candidate damaged contours in the processed image based on the area and center of the circle, and determine the area of ​​the damaged contour as the damaged contour area in the plastic film image.

[0131] In some implementations, since there are noisy contours among the candidate damaged contours, i.e. there is at least one candidate damaged contour, in order to ensure the accuracy of the damaged contour area, it is necessary to filter the candidate damaged contours according to the area and center of each candidate damaged contour to obtain the damaged contour, and the area of ​​the damaged contour is determined as the damaged contour area in the plastic film image.

[0132] Optionally, the distance between the center of the processed image and the center of the candidate damaged contour, as well as the area of ​​each candidate damaged contour, can be calculated, and the candidate damaged contour with the smallest distance and the largest area can be determined as the damaged contour.

[0133] After steps 225 and 221, when the damaged area does not meet the preset contour area calculation conditions, the defect detection result of the plastic film image is determined to be that there is a defect in the plastic film image.

[0134] In some implementations, when the damaged area does not meet the preset contour area calculation conditions, it indicates that the damaged area is a large-area damage, or a crack in the hot melt adhesive seam with a large damaged area. Therefore, the defect detection result of the plastic sealing film image is determined to be that there is a defect in the plastic sealing film image.

[0135] This application embodiment filters damaged areas by damage type and detection frame size, filtering out small damaged areas, wrinkles and other similar damaged areas, and areas with weak exposure. It then performs a second inspection based on the damaged outline area, thereby reducing the probability of missed and false detections of small damaged areas and cracks at hot melt adhesive seams, and improving the accuracy of defect detection results.

[0136] Considering that when the detection frame of the damaged area is small, it indicates that there may be no damage in the plastic wrapping film image, or the damaged area is small, or the damage is a scratch, which will not affect the subsequent transportation and use of the product. If steps 220 and 230 are performed on all such damaged areas to determine the defect detection result, it will increase the data processing load of the plastic wrapping film defect detection method, thereby affecting the real-time performance of the plastic wrapping film defect detection method. Based on this, Figure 2The provided method for detecting defects in plastic wrapping films, in order to improve the real-time performance of the method, can, after determining the detection frame and the type of damage in the damaged area of ​​the plastic wrapping film image, filter the damaged areas based on the size information of the detection frame, eliminating damaged areas that do not require contour detection, and for damaged areas that do require contour detection, perform steps 220 and 230 to determine the defect detection result of the plastic wrapping film image. Thus, through two rounds of filtering, the amount of data is reduced, and the real-time performance of the plastic wrapping film defect detection method is improved. Specifically, as shown... Figure 14 As shown, Figure 14 This is a schematic flowchart of another method for detecting defects in plastic sealing films provided in this application embodiment. The method for detecting defects in plastic sealing films shown includes steps 1410 to 1460:

[0137] Step 1410: Perform target detection on the plastic film image to be detected, and determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

[0138] In some embodiments, target detection can be performed on the plastic film image to be detected according to step 210 above to determine the detection frame of the damaged area in the plastic film image and the damage type of the damaged area. The embodiments of this application will not be described in detail here.

[0139] In some implementation methods, reference may be made to Figure 4 The provided target detection method performs target detection on the plastic film image to be detected, determines the detection box of the damaged area in the plastic film image and the damage type of the damaged area, which will not be described in detail in the embodiments of this application.

[0140] In some implementations, it can be based on Figure 10 or Figure 11 The provided detection model performs target detection on the plastic film image to be detected, determines the detection box of the damaged area in the plastic film image and the damage type of the damaged area, which will not be described in detail in the embodiments of this application.

[0141] Step 1420: Determine the confidence level of the damaged area.

[0142] In some implementations, the confidence level of the damaged area can be obtained based on the confidence level of the detection box in the target detection results.

[0143] In some implementations, a pre-defined prediction model can be used to predict the probability of damage to the damaged area, thus determining the probability of damage in the area as the confidence level of the damaged area. The prediction model can be either a machine learning-based model or a neural network-based model.

[0144] Step 1430: Compare the size information of the damaged area with the preset minimum size threshold.

[0145] Step 1440: When the size information of the damaged area is greater than or equal to the preset minimum size threshold, and the confidence level is greater than or equal to the preset confidence threshold, contour detection is performed based on the size information of the damaged area, the damage type, and the plastic film image to determine the damaged contour area in the plastic film image.

[0146] In some embodiments, contour detection can be performed based on the size information of the detection frame, the damage type, and the plastic film image according to step 220 above to determine the area of ​​the damaged contour in the plastic film image. The embodiments of this application will not be described in detail here.

[0147] In some implementation methods, reference may be made to Figure 12 The provided method for calculating the area of ​​the damaged contour in the image of the plastic sealing film determines the area of ​​the damaged contour, which will not be described in detail in the embodiments of this application.

[0148] Step 1450: Determine the defect detection result of the plastic film image based on the damaged contour area.

[0149] In some embodiments, the defect detection results of the plastic film image can be determined according to step 230 above, which will not be described in detail in the embodiments of this application.

[0150] Step 1460: When the size information of the damaged area is less than the preset minimum size threshold and / or the confidence level is less than the preset confidence level threshold, the defect detection result of the plastic film image is determined to be that there is no defect in the plastic film image.

[0151] In this embodiment, after determining the detection frame and damage type of the damaged area in the plastic wrapping film image, the damaged area is screened based on the size information of the detection frame. Damaged areas that do not require contour detection are removed. For damaged areas that require contour detection, the defect detection result of the plastic wrapping film image is determined by the damaged contour area of ​​the damaged area. In this way, through two rounds of screening, the amount of data is reduced and the real-time performance of the plastic wrapping film defect detection method is improved.

[0152] The plastic film defect detection method provided in this application can reduce interference from small-area damage, scratches, wrinkles and bulges, improve the accuracy of defect detection results, and thus reduce the misjudgment rate of defect detection results by locating the position and type of damage area in the plastic film image and determining the defect detection result based on the area of ​​the damage contour.

[0153] To better implement the molding film defect detection method provided in the embodiments of this application, based on the embodiments of the molding film defect detection method, the embodiments of this application provide a molding film defect detection device, specifically, as follows: Figure 15 As shown, Figure 15 This is a schematic diagram of the structure of the plastic sealing film defect detection device provided in the embodiments of this application. The plastic sealing film defect detection device shown includes:

[0154] The damage detection module is used to perform target detection on the plastic film image to be inspected, and to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area;

[0155] The contour detection module is used to perform contour detection based on the size information of the detection frame, the damage type, and the plastic film image, and to determine the area of ​​the damaged contour in the plastic film image.

[0156] The defect detection module is used to determine the defect detection results of the plastic sealing film image based on the area of ​​the damaged contour.

[0157] In some implementations, the contour detection module is used for:

[0158] Based on the size information and damage type of the detection frame, determine whether the damaged area meets the preset contour area calculation conditions;

[0159] When the damaged area meets the preset contour area calculation conditions, the plastic film image is processed according to the size information of the detection frame to obtain the processed image;

[0160] Contour detection is performed on the processed image to obtain candidate damaged contours in the processed image;

[0161] Based on the area and center of the candidate damaged contour in the processed image, the damaged contour is determined from the candidate damaged contours in the processed image, and the area of ​​the damaged contour is determined as the area of ​​the damaged contour in the plastic film image.

[0162] In some implementations, the contour detection module is used for:

[0163] Perform a gamma transform on the processed image to obtain the transformed image;

[0164] Edge detection is performed on the transformed image to obtain the edge contour image of the transformed image;

[0165] Dilation and erosion processing is applied to the edge contour image to obtain a binary image;

[0166] Ellipse fitting is performed based on the binary image to obtain candidate damaged contours in the processed image.

[0167] In some implementations, the contour detection module is used for:

[0168] Compare the size information of the detection frame with a preset size threshold;

[0169] If the size information of the detection box is less than or equal to the preset size threshold, and the damage type is the preset target type, then the damaged area is determined to meet the preset contour area calculation conditions.

[0170] If the size of the detection frame is greater than the preset size threshold, or the damage type is not the preset target type, then the damaged area is determined not to meet the preset contour area calculation conditions.

[0171] In some implementations, the defect detection module is used for:

[0172] When the damaged area does not meet the preset contour area calculation conditions, the defect detection result of the plastic film image is determined to be that there is a defect in the plastic film image.

[0173] In some embodiments, the plastic film defect detection device further includes:

[0174] The filtering module is used to determine the confidence level of the damaged area; compare the size information of the damaged area with a preset minimum size threshold; when the size information of the damaged area is greater than or equal to the preset minimum size threshold, and the confidence level is greater than or equal to the preset confidence threshold, perform contour detection based on the size information of the damaged area, the damage type, and the plastic film image to determine the damaged contour area in the plastic film image; when the size information of the damaged area is less than the preset minimum size threshold, and / or the confidence level is less than the preset confidence threshold, determine that the defect detection result of the plastic film image is that there is no defect in the plastic film image.

[0175] In some implementations, the damage detection module is used for:

[0176] Multi-scale feature extraction is performed on the image of the plastic sealing film to be tested to obtain image features at different scales;

[0177] Feature fusion is performed on image features at different scales to obtain fused image features;

[0178] Feature calculations are performed based on fused image features to determine the detection bounding box of the damaged area in the plastic film image and the damage type of the damaged area.

[0179] In some implementations, the damage detection module is used for:

[0180] Starting from the second highest scale, the image features of each scale are fused layer by layer with the first fused image features of the previous scale adjacent to that scale to obtain the first fused image features of each scale.

[0181] The image features at each intermediate scale are fused with the first fused image features at that scale to obtain the second fused image features at each intermediate scale; the intermediate scales are the remaining scales excluding the highest and lowest scales.

[0182] The highest-scale image feature is determined as the highest-scale second fused image feature, and the lowest-scale first fused image feature is determined as the lowest-scale second fused image feature. Starting from the lowest scale, the lowest-scale second fused image feature, the second fused image features of each intermediate scale, and the highest-scale second fused image feature are fused layer by layer to obtain new image features at each scale.

[0183] Starting from the second highest scale, the new image features at each scale are fused layer by layer with the new first fused image features at the previous scale adjacent to that scale to obtain the new first fused image features at each scale. This process is repeated until the number of times the image features are updated is greater than or equal to a preset threshold. At this point, the new image features at each scale are determined as fused image features.

[0184] In some implementations, the defect detection module is used for:

[0185] The area of ​​the damaged contour is compared with a preset area threshold.

[0186] If the area of ​​the damaged contour is greater than or equal to the preset area threshold, the defect detection result of the plastic film image is determined to be that there is a defect in the plastic film image;

[0187] If the area of ​​the damaged contour is less than the preset area threshold, the defect detection result of the plastic film image is determined to be that there is no defect in the plastic film image.

[0188] In some implementations, the damage detection module is used for:

[0189] The image of the plastic film to be detected is input into the preset detection model for target detection, and the detection box of the damaged area in the plastic film image and the damage type of the damaged area are obtained. The preset detection model is trained on the EfficientDet network based on the sample image.

[0190] The plastic film defect detection device provided in this application can reduce interference from small-area damage, scratches, wrinkles and bulges, improve the accuracy of defect detection results, and thus reduce the misjudgment rate of defect detection results by locating the position and type of damage area in the plastic film image and determining the defect detection result based on the area of ​​the damage outline.

[0191] This invention also provides an electronic device, such as... Figure 16As shown, it illustrates a structural schematic diagram of the electronic device involved in an embodiment of the present invention, specifically:

[0192] The electronic device may include components such as a processor 1601 with one or more processing cores, a memory 1602 with one or more computer-readable storage media, a power supply 1603, and an input unit 1604. Those skilled in the art will understand that... Figure 16 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0193] The processor 1601 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1602, and by calling data stored in the memory 1602, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 1601 may include one or more processing cores; preferably, the processor 1601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1601.

[0194] The memory 1602 can be used to store software programs and modules. The processor 1601 executes various functional applications and data processing by running the software programs and modules stored in the memory 1602. The memory 1602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1602 may also include a memory controller to provide the processor 1601 with access to the memory 1602.

[0195] The electronic device also includes a power supply 1603 that supplies power to the various components. Preferably, the power supply 1603 can be logically connected to the processor 1601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0196] The electronic device may also include an input unit 1604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0197] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1601 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 1602 according to the following instructions, and the processor 1601 runs the applications stored in the memory 1602 to realize various functions, as follows:

[0198] Target detection is performed on the plastic film image to be inspected to determine the detection box of the damaged area in the plastic film image and the type of damage in the damaged area;

[0199] Contour detection is performed based on the size information of the detection frame, the type of damage, and the image of the plastic wrapping film to determine the area of ​​the damaged contour in the plastic wrapping film image.

[0200] Defect detection results of plastic film images are determined based on the area of ​​the damaged contour.

[0201] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0202] To this end, embodiments of the present invention provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the plastic encapsulation film defect detection methods provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0203] Target detection is performed on the plastic film image to be inspected to determine the detection box of the damaged area in the plastic film image and the type of damage in the damaged area;

[0204] Contour detection is performed based on the size information of the detection frame, the type of damage, and the image of the plastic wrapping film to determine the area of ​​the damaged contour in the plastic wrapping film image.

[0205] Defect detection results of plastic film images are determined based on the area of ​​the damaged contour.

[0206] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0207] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0208] Since the instructions stored in the storage medium can execute the steps in any of the plastic film defect detection methods provided in the embodiments of the present invention, the beneficial effects that any of the plastic film defect detection methods provided in the embodiments of the present invention can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0209] The above provides a detailed description of a method, apparatus, electronic device, and storage medium for detecting defects in plastic-encapsulated films according to embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting defects in plastic sealing films, characterized in that, The method includes: Target detection is performed on the plastic film image to be detected to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area; Based on the size information of the detection frame, the damage type, and the image of the plastic sealing film, contour detection is performed to determine the area of ​​the damaged contour in the image of the plastic sealing film. The defect detection result of the plastic sealing film image is determined based on the damaged contour area; The step of performing contour detection based on the size information of the detection frame, the damage type, and the image of the plastic wrapping film to determine the area of ​​the damaged contour in the plastic wrapping film image includes: determining whether the damaged area meets a preset contour area calculation condition based on the size information of the detection frame and the damage type; when the damaged area meets the preset contour area calculation condition, performing image processing on the plastic wrapping film image according to the size information of the detection frame to obtain a processed image; performing contour detection on the processed image to obtain candidate damaged contours in the processed image; determining the damaged contour from the candidate damaged contours in the processed image based on the area and center of the candidate damaged contours in the processed image, and determining the area of ​​the damaged contour as the area of ​​the damaged contour in the plastic wrapping film image; The step of determining whether the damaged area meets the preset contour area calculation conditions based on the size information of the detection frame and the damage type includes: comparing the size information of the detection frame with a preset size threshold; if the size information of the detection frame is less than or equal to the preset size threshold, and the damage type is a preset target type, then the damaged area is determined to meet the preset contour area calculation conditions; if the size information of the detection frame is greater than the preset size threshold, or the damage type is not a preset target type, then the damaged area is determined not to meet the preset contour area calculation conditions.

2. The method according to claim 1, characterized in that, The step of performing contour detection on the processed image to obtain candidate damaged contours in the processed image includes: Perform a gamma transform on the processed image to obtain the transformed image; Edge detection is performed on the transformed image to obtain the edge contour image of the transformed image; The edge contour image is subjected to dilation and erosion processing to obtain a binary image; Ellipse fitting is performed based on the binary image to obtain candidate damaged contours in the processed image.

3. The method according to claim 1, characterized in that, After determining whether the damaged area meets the preset contour area calculation conditions based on the size information of the detection frame and the damage type, the method further includes: When the damaged area does not meet the preset contour area calculation conditions, the defect detection result of the plastic sealing film image is determined to be that there is a defect in the plastic sealing film image.

4. The method according to claim 1, characterized in that, After performing target detection on the image of the plastic sealing film to be detected, and determining the detection bounding box of the damaged area in the plastic sealing film image and the damage type of the damaged area, the method further includes: Determine the confidence level of the damaged area; The size information of the damaged area is compared with a preset minimum size threshold; When the size information of the damaged area is greater than or equal to the preset minimum size threshold, and the confidence level is greater than or equal to the preset confidence level threshold, the step of performing contour detection based on the size information of the damaged area, the damage type, and the plastic film image to determine the damaged contour area in the plastic film image is executed. When the size of the damaged area is less than the preset minimum size threshold, and / or the confidence level is less than the preset confidence level threshold, the defect detection result of the plastic film image is determined to be that there is no defect in the plastic film image.

5. The method according to claim 1, characterized in that, The step of performing target detection on the image of the plastic wrapping film to be detected, and determining the detection bounding box of the damaged area in the plastic wrapping film image and the damage type of the damaged area, includes: Multi-scale feature extraction is performed on the image of the plastic sealing film to be tested to obtain image features at different scales; Feature fusion is performed on the image features at different scales to obtain fused image features; Feature calculations are performed based on the fused image features to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area.

6. The method according to claim 5, characterized in that, The process of fusing image features at different scales to obtain fused image features includes: Starting from the second highest scale, the image features of each scale and the first fused image features of the previous scale adjacent to that scale are fused layer by layer to obtain the first fused image features of each scale. The image features at each intermediate scale are fused with the first fused image features at that scale to obtain the second fused image features at each intermediate scale; the intermediate scales are the remaining scales excluding the highest and lowest scales. The highest-scale image feature is determined as the second fused image feature of the highest scale, and the lowest-scale first fused image feature is determined as the second fused image feature of the lowest scale. Starting from the lowest scale, the second fused image feature of the lowest scale, the second fused image features of each intermediate scale, and the second fused image feature of the highest scale are fused layer by layer to obtain new image features of each scale. Starting from the second highest scale, the new image features of each scale and the new first fused image features of the previous scale adjacent to the scale are fused layer by layer to obtain the new first fused image features of each scale. This process is repeated until the number of times the image features are updated is greater than or equal to a preset number threshold. Then, the new image features of each scale are determined as fused image features.

7. The method according to claim 1, characterized in that, The method of determining the defect detection result of the plastic film image based on the damaged contour area includes: The area of ​​the damaged contour is compared with a preset area threshold. If the area of ​​the damaged contour is greater than or equal to the preset area threshold, then the defect detection result of the plastic film image is determined to be that there is a defect in the plastic film image; If the area of ​​the damaged contour is less than the preset area threshold, then the defect detection result of the plastic film image is determined to be that there is no defect in the plastic film image.

8. The method according to any one of claims 1 to 7, characterized in that, The process involves target detection on the plastic film image to be detected, resulting in a detection bounding box for the damaged area in the plastic film image and the damage type of the damaged area, including: The image of the plastic film to be detected is input into a preset detection model for target detection, and the detection box of the damaged area in the plastic film image and the damage type of the damaged area are obtained; the preset detection model is obtained by training the EfficientDet network based on the sample image.

9. A device for detecting defects in plastic sealing films, characterized in that, The device includes: The damage detection module is used to perform target detection on the plastic film image to be detected, and to determine the detection box of the damaged area in the plastic film image and the damage type of the damaged area. The contour detection module is used to perform contour detection based on the size information of the detection frame, the damage type, and the plastic film image, and to determine the area of ​​the damaged contour in the plastic film image. A defect detection module is used to determine the defect detection result of the plastic sealing film image based on the damaged contour area; The step of performing contour detection based on the size information of the detection frame, the damage type, and the image of the plastic wrapping film to determine the area of ​​the damaged contour in the plastic wrapping film image includes: determining whether the damaged area meets a preset contour area calculation condition based on the size information of the detection frame and the damage type; when the damaged area meets the preset contour area calculation condition, performing image processing on the plastic wrapping film image according to the size information of the detection frame to obtain a processed image; performing contour detection on the processed image to obtain candidate damaged contours in the processed image; determining the damaged contour from the candidate damaged contours in the processed image based on the area and center of the candidate damaged contours in the processed image, and determining the area of ​​the damaged contour as the area of ​​the damaged contour in the plastic wrapping film image; The step of determining whether the damaged area meets the preset contour area calculation conditions based on the size information of the detection frame and the damage type includes: comparing the size information of the detection frame with a preset size threshold; if the size information of the detection frame is less than or equal to the preset size threshold, and the damage type is a preset target type, then the damaged area is determined to meet the preset contour area calculation conditions; if the size information of the detection frame is greater than the preset size threshold, or the damage type is not a preset target type, then the damaged area is determined not to meet the preset contour area calculation conditions.

10. An electronic device, characterized in that, It includes a memory and a processor; the memory stores an application program, and the processor runs the application program within the memory to perform the operations in the plastic sealing film defect detection method according to any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores a plurality of instructions adapted for loading by a processor to execute the steps of the plastic encapsulation film defect detection method according to any one of claims 1 to 8.

Images