Defect detection method and device for rubber product, electronic equipment and storage medium

Abstract

The application discloses a defect detection method and device of a rubber product, an electronic device and a storage medium, and belongs to the technical field of industrial detection. The method comprises the following steps: acquiring a video of a rubber product extrusion process; for each frame of image in the video, a plurality of local images in the image are intercepted according to a sliding window, wherein the size of the local image is the same as the size of the sliding window; the rubber product image in each local image is subjected to defect detection to obtain a candidate defect area of the current frame of image; and a classification model is used to perform defect detection on the candidate defect area to obtain a defect result of the rubber product. Through the application, the technical problem of low defect detection efficiency of the rubber product in the related art is solved.

Description

Technical Field

[0001] This application relates to the field of industrial testing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for detecting defects in rubber products. Background Technology

[0002] The deep integration of new-generation information technology and manufacturing has triggered a tremendous transformation in the industry, gradually shifting from quantity-based growth to quality-based improvement. Improving product quality to produce high-quality, high-profit products can significantly enhance product competitiveness. Strategies for improving product quality include strengthening product quality inspection, upgrading industrial standards, and standardizing production operations. Among these, quality inspection is the most commonly used method in manufacturing.

[0003] In related technologies, the inspection of rubber products mainly adopts manual inspection. However, when the defect area of ​​the rubber product is small and the defect characteristics are not obvious, manual inspection is prone to missed detection, and the accuracy of the inspection is difficult to guarantee. In addition, manual inspection requires long-term visual observation, which can easily cause eye fatigue and result in low inspection efficiency.

[0004] There are currently no effective solutions to the aforementioned problems in the relevant technologies. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for detecting defects in rubber products, in order to solve the technical problem of low efficiency in defect detection of rubber products in related technologies.

[0006] According to one aspect of the embodiments of this application, a method for detecting defects in rubber products is provided, comprising: acquiring a video of the rubber product being squeezed; for each frame of the video, cropping multiple local images from the image according to a sliding window, wherein the size of the local image is the same as the size of the sliding window; performing defect detection on the rubber product image in each local image to obtain candidate defect regions of the current frame image; and using a classification model to perform defect detection on the candidate defect regions to obtain the defect result of the rubber product.

[0007] Further, defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame image, including: using a first deep learning model to perform target detection on the current local image to obtain a first defect result, and using a second deep learning model to perform anomaly detection on the current local image to obtain a second defect result, wherein the first deep learning model is trained using negative samples and the second deep learning model is trained using positive samples; merging the first defect result and the second defect result to obtain the defect detection result of the current local image; and merging the defect detection results of all local images to obtain candidate defect regions for the current frame image.

[0008] Furthermore, before using the first deep learning model to perform target detection on the current local image, the method further includes: acquiring a negative sample image with defects, and extracting the defect region in the negative sample image; defining a sliding window on the negative sample image, and sliding the sliding window on the negative sample image in multiple directions with a preset interval, centered on the defect region; cropping a local image of the sliding window on the negative sample image at each sliding, and using the local image as a negative sample; and using the negative sample to train the target detection model to obtain the first deep learning model.

[0009] Further, training the target detection model using the negative samples to obtain the first deep learning model includes: training the target detection model using the negative samples and the following loss function to obtain the first deep learning model: FL(p t )=-α t (1-p t ) γ log(p t ), where p t Let (1-p) be the probability that the result predicted by the first deep learning model is a defect. t ) γ A modulation factor, used to reduce the contribution of easily distinguishable samples, a t γ is used to adjust the ratio between the loss of positive and negative samples, and is used to control the imbalance between easily distinguishable and difficult-to-distinguish samples.

[0010] Furthermore, before using the second deep learning model to perform anomaly detection on the current local image, the method further includes: acquiring a defect-free positive sample video of the rubber product being squeezed; extracting multiple keyframe images from the positive sample video; for each keyframe image, cropping multiple local images from the keyframe image according to a sliding window, and using the multiple local images as positive samples; using the positive samples to train the anomaly detection model to obtain the second deep learning model.

[0011] Further, extracting multiple keyframe images from the positive sample video includes: retaining the first frame image of the positive sample video as a keyframe image; calculating the image similarity between the current frame image and the previous frame image for the current frame image after the first frame image; if the image similarity is lower than a preset threshold, then the current frame image is used as a keyframe image.

[0012] Further, the defect detection of the candidate defect region using a classification model to obtain the defect result of the rubber product includes: expanding the candidate defect region to obtain the region to be detected; acquiring multiple consecutive historical frame images before the current frame image and determining the target region corresponding to the region to be detected in each historical frame image; inputting the candidate defect region image of the current frame image and the target region image corresponding to the historical frame images into the classification model respectively, and statistically analyzing the defect result of each frame image; if the number of frames with defect results is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect; if the number of frames with defect results is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

[0013] According to another aspect of the embodiments of this application, a defect detection device for rubber products is also provided, comprising: an acquisition module for acquiring a video of the rubber product being squeezed; a cropping module for cropping multiple local images of each frame of the video according to a sliding window, wherein the size of the local image is the same as the size of the sliding window; a first detection module for performing defect detection on the rubber product image in each local image to obtain a candidate defect region of the current frame image; and a second detection module for performing defect detection on the candidate defect region using a classification model to obtain the defect result of the rubber product.

[0014] Furthermore, the first detection module includes a first detection unit, used to perform target detection on the current local image using a first deep learning model to obtain a first defect result, and to perform anomaly detection on the current local image using a second deep learning model to obtain a second defect result, wherein the first deep learning model is trained using negative samples and the second deep learning model is trained using positive samples; merging the first defect result and the second defect result to obtain the defect detection result of the current local image; and merging the defect detection results of all local images to obtain the candidate defect region of the current frame image.

[0015] Furthermore, the defect detection device for rubber products also includes a first training unit, used to acquire negative sample images containing defects and extract defect regions from the negative sample images; define a sliding window on the negative sample images, and slide the sliding window on the negative sample images in multiple directions with a preset interval, centered on the defect region; capture a local image of the sliding window on the negative sample images at each slide, and use the local image as a negative sample; use the negative samples to train the target detection model to obtain a first deep learning model.

[0016] Furthermore, the first training unit is also used to train the object detection model using the negative samples and the following loss function to obtain a first deep learning model: FL(p t )=-α t (1-p t ) γ log(p t ), where p t Let (1-p) be the probability that the result predicted by the first deep learning model is a defect. t ) γ A modulation factor, used to reduce the contribution of easily distinguishable samples, a t γ is used to adjust the ratio between the loss of positive and negative samples, and is used to control the imbalance between easily distinguishable and difficult-to-distinguish samples.

[0017] Furthermore, the defect detection device for rubber products also includes a second training unit, used to acquire positive sample videos of rubber products without defects during the extrusion process; extract multiple key frame images from the positive sample videos; for each key frame image, extract multiple local images from the key frame image according to a sliding window, and use the multiple local images as positive samples; use the positive samples to train the anomaly detection model to obtain a second deep learning model.

[0018] Furthermore, the second training unit is also used to retain the first frame image of the positive sample video as a keyframe image; for the current frame image after the first frame image, calculate the image similarity between the current frame image and the previous frame image; if the image similarity is lower than a preset threshold, then the current frame image is used as a keyframe image.

[0019] Furthermore, the second detection module includes a second detection unit, used to expand the candidate defect region to obtain the region to be detected; acquire multiple consecutive historical frame images preceding the current frame image, and determine the target region corresponding to the region to be detected in each historical frame image; input the candidate defect region image of the current frame image and the target region image corresponding to the historical frame image into the classification model respectively, and count the defect results of each frame image; if the number of frames with defect results is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect; if the number of frames with defect results is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

[0020] According to another aspect of the embodiments of this application, a storage medium is also provided, the storage medium including a stored program that executes the above steps when the program is run.

[0021] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the programs stored in the memory.

[0022] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the above-described method.

[0023] This application obtains a video of the rubber product being extruded. For each frame of the video, multiple local images are extracted using a sliding window, where the size of the local image is the same as the size of the sliding window. Defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame. A classification model is then used to detect defects in the candidate defect regions, resulting in the defective product. By performing image analysis on the video of the rubber product being extruded, automatic defect detection of rubber products can be achieved. This eliminates the need for specialized technicians to observe and identify defects throughout the extrusion process, improving detection efficiency and reducing manual labor intensity. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0025] Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application;

[0026] Figure 2 This is a flowchart of a defect detection method for rubber products according to an embodiment of this application;

[0027] Figure 3 This is a schematic diagram of the process for detecting defects in the rubber of rails according to an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of the defect candidate region detection process according to an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the on-site installation of the defect detection and monitoring equipment according to an embodiment of this application;

[0030] Figure 6 This is a structural block diagram of a defect detection device for rubber products according to an embodiment of this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile phone, computer, tablet, or similar computing device. Taking running on a computer as an example, Figure 1 This is a hardware structure block diagram of a computer according to an embodiment of this application. For example... Figure 1 As shown, a computer may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. Optionally, the computer may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer described above. For example, the computer may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to a defect detection method for rubber products in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0036] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a computer's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0037] This embodiment provides a defect detection method for rubber products. The method can be based on a dual-view rubber product defect detection and monitoring device. Specifically, the rubber product in this embodiment is a rail rubber product. Rail rubber products are elastic elements of train track rubber vibration damping devices, created by combining metal and rubber through a vulcanization process. Rail rubber products not only ensure the smooth operation of trains but also guarantee their safety and comfort. Because a small number of defective products are produced during production, and these defective products pose safety hazards and can easily cause economic losses, quality inspection of the products and timely detection of defective products are of great significance for improving product quality and significantly reducing product usage risks.

[0038] Reference Figure 5 A schematic diagram of the on-site installation of the defect detection and monitoring equipment, which includes two high-resolution color industrial cameras (including, for example...). Figure 1 The system includes a left camera 1a and a right camera 1b, a ring light source 2, a fixture 4, a rubber product 5 to be inspected, and a pressurizing device 6. The distance between the rubber product 5 and the camera lens 3 can be set to 40cm, and the distance between the left camera 1a and the right camera 1b can be set to 20cm. The two cameras are placed parallel to each other, facing the rubber product extrusion station to ensure clear imaging of the extrusion process within the camera's field of view. The camera resolution is 5472*3684, and the frame rate is 5FPS. In this embodiment, because the actual area of ​​the inspected rail rubber product is large and the defects are extremely small, two cameras are used for separate inspection. Different areas are captured, with some overlap between the shooting areas, ensuring the complete presentation of the inspected rail rubber product within the camera's field of view and improving the visibility of defective areas. A light source is used to reduce the impact of lighting changes on image quality. Optionally, the defect detection monitoring equipment also includes a monitor and computing storage device. Inspectors can observe the extrusion process through the monitor without directly facing the extrusion site, improving the working environment. The storage device records the inspection process for retrospective review, allowing for repeated confirmation through a program, avoiding repeated extrusion tests due to insufficient or inattentive manual observation. The extrusion equipment is fixed to the rail rubber product using clamps, and then the extrusion tool is used to repeatedly extrude the rail rubber product. If the rail rubber product is defective, the abnormal parts will be exposed during the extrusion process.

[0039] Figure 2 This is a flowchart of a defect detection method for rubber products according to an embodiment of this application, such as... Figure 2 As shown, the process includes the following steps:

[0040] Step S10: Obtain a video of the rubber product being extruded;

[0041] Step S20: For each frame of the video, extract multiple partial images from the image using a sliding window, wherein the size of the partial images is the same as the size of the sliding window;

[0042] In this embodiment, a sliding window processing method is applied to each frame of the video. The specific sliding window processing flow is as follows: a sliding window is defined on the current frame image, and the sliding window is slid across the current frame image at preset intervals, capturing a local image of the sliding window on the current frame image at each slide. The size of the sliding window can be determined based on computational efficiency and defect detection accuracy. The preset interval is less than or equal to m, where m = min(width, height) / 2, and width and height are the width and height of the sliding window, respectively. This embodiment takes into account the application scenario of defect detection of steel rail rubber products, where the image field resolution is large and the defect feature area is a small target area. In order to further improve the feature of the defect area and reduce the influence of the normal area of ​​the image on the defect area, a sliding window processing is performed on the video frame to be tested. This can reduce the impact of the imbalance between positive and negative samples on the model prediction accuracy. For example, if the original image has 10,000 pixels and the defect area occupies 10 pixels, then the proportion of the defect area is 1 / 1000. However, if the sliding window size is 1,000 pixels, the number of pixels occupied by the defect area remains unchanged at 10, that is, the proportion of the defect area is 1 / 100. In this way, the proportion of the normal area is greatly reduced, the impact of sample imbalance is reduced, and thus the model prediction accuracy is improved. Meanwhile, when performing target detection, the input image is required to be at a fixed resolution, such as 250*250. If the resolution exceeds the fixed resolution, the image will be resized for a resolution of 1000*1000, which will cause image distortion. However, in this embodiment, the resolution of the sliding window can be set to 250*250, so resizing is not required, which can reduce the impact of image preprocessing (such as resizing) on ​​image features and ensure that the image is not distorted.

[0043] This embodiment, after capturing multiple local images from the image using a sliding window, further includes preprocessing the local images generated by the sliding window, mainly including uneven illumination processing and image denoising, to reduce the impact of uneven illumination and image noise on subsequent detection results.

[0044] Step S30: Perform defect detection on the rubber product image in each local image to obtain the candidate defect region of the current frame image;

[0045] Step S40: Use a classification model to detect defects in the candidate defect areas to obtain the defect results of the rubber product.

[0046] First, defect detection is performed on the image occupied by the rubber product in each local image to obtain the candidate defect region of the current frame image. Then, a classification model is used to detect the defect in the candidate defect region to obtain the final defect result of the rubber product as a defective product or a normal product. The defect result includes the defect classification result and the defect location. The defect classification result includes defective and normal. The defect can also include specific defect types, such as damage, cracking, bulging and bubbles.

[0047] Through the above steps, a video of the rubber product being extruded is obtained. For each frame of the video, multiple local images are extracted from the image using a sliding window, where the size of the local image is the same as the size of the sliding window. Defect detection is performed on the rubber product image within each local image to obtain candidate defect regions of the current frame image. A classification model is then used to detect defects in the candidate defect regions to obtain the defect results of the rubber product. By performing image analysis on the video of the rubber product being extruded, automatic detection of defects in rubber products can be achieved. This eliminates the need for specialized technicians to observe and identify defects that occur throughout the extrusion process, improving detection efficiency and reducing manual labor intensity.

[0048] In one embodiment of this example, defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame image, including:

[0049] S31, a first deep learning model is used to perform target detection on the current local image to obtain a first defect result, and a second deep learning model is used to perform anomaly detection on the current local image to obtain a second defect result, wherein the first deep learning model is trained using negative samples and the second deep learning model is trained using positive samples;

[0050] S32, merge the first defect result and the second defect result to obtain the defect detection result of the current local image;

[0051] S33, merge the defect detection results of all local images to obtain the candidate defect regions of the current frame image.

[0052] This embodiment primarily employs deep learning-based object detection and anomaly detection algorithms to detect defects in local images. The two detection results are then merged to obtain candidate defect regions for the current frame image. (Reference) Figure 4For a single local image, the first and second defect results are subjected to non-maximum suppression (NMS) to filter out redundancy and duplication generated by a single model. These results are then restored to the original image coordinate system to obtain multi-model detection results for a single sliding window (i.e., a local image) based on the original image coordinate system. The multi-model detection results are then merged, and the merged result is subjected to NMS to filter out redundant and duplicate results generated by multi-model fusion prediction, resulting in the final single sliding window region detection result. Finally, the detection results generated by all sliding windows are merged, and NMS is used to filter out some duplicate detection results to reduce subsequent computational load. Since the amount of rail defect data is relatively small and the defect area is a small target, this embodiment uses a model trained from both positive and negative samples to comprehensively detect the defect area. On the one hand, since there are few negative samples but a relatively abundant number of positive samples, a deep learning-based anomaly detection algorithm is used to learn from a large number of normal samples to detect defects in the image under test. However, since anomaly detection does not directly learn the features of the defect area, it may have insufficient generalization ability. Therefore, by collecting defect samples, a deep learning target detection algorithm is used to directly learn the defect area, and the trained target detection algorithm is used to detect the image under test. Subsequently, the detection results of the two are merged to complete the detection process of a single sliding window area.

[0053] The anomaly detection algorithm, designed for scenarios with a large number of positive samples and a small number of negative samples, uses positive samples as its training set. This means the anomaly detection model only learns the features of positive samples, and when encountering anomaly samples, it exposes the anomalous regions that differ from normal regions. While the anomaly detection algorithm does not directly learn defect features and has lower reliability compared to object detection algorithms, it can still obtain candidate defect regions. Furthermore, the ample positive samples in the anomaly detection algorithm allow for the acquisition of more candidate defect regions, improving the detection accuracy of subsequent classification models. Object detection algorithms, on the other hand, use labeled defect regions and directly learn their features. When there are sufficient defect samples, their reliability is higher than that of the anomaly detection algorithm, and the defect features are more obvious. Therefore, this embodiment combines the advantages of multiple models for defect detection, improving detection accuracy.

[0054] The nonmaximum suppression algorithm calculation steps include: setting a confidence threshold for the target box, typically set to 0.5, where the target box is a local region; sorting the candidate box list in descending order of confidence; selecting the box A with the highest confidence and adding it to the output list, then removing it from the candidate box list; calculating the IOU value between A and all boxes in the candidate box list, and removing candidate boxes with a confidence value greater than the threshold; repeating the above process until the candidate box list is empty, and then returning the output list.

[0055] In this embodiment, before using the first deep learning model to perform target detection on the current local image, the method further includes: acquiring a negative sample image with defects and extracting the defect region in the negative sample image; defining a sliding window on the negative sample image, and sliding the sliding window on the negative sample image in multiple directions with a preset interval, centered on the defect region; cropping a local image of the sliding window on the negative sample image at each sliding, and using the local image as a negative sample; and using the negative sample to train the target detection model to obtain the first deep learning model.

[0056] In this embodiment, a negative sample library is constructed: negative sample images with defects are acquired, and defective regions are extracted from the negative sample images. Based on the labeled defective regions, the sliding window is centered on the defective regions and the sliding window size is used as the basic size. The sliding window slides in multiple directions at certain intervals with the defective regions as the center, and the relevant regions are captured and saved as the final defective samples, i.e., negative samples. Through this process, the number and diversity of samples can be increased, thereby achieving the purpose of sample enhancement.

[0057] In this embodiment, to compensate for the lack of defect data, an open-source industrial defect dataset and a common softmax loss function are first used to train the model to obtain a pre-trained weight. Then, using the pre-trained weight as the initial weight, the defect data from the aforementioned negative sample library and the focal_loss loss function are used to further train the model. In this embodiment, a common classification loss function is first used to train the model, and then focal_loss is used to fine-tune the model, which reduces the impact of imbalanced samples and easily distinguishable samples on model training.

[0058] The specific formula for calculating focal_loss is: FL(p t )=-α t (1-p t ) γ log(p t ), where p t Let (1-p) be the probability that the result predicted by the first deep learning model is a defect. t ) γ A modulation factor, used to reduce the contribution of easily distinguishable samples, a t γ is used to adjust the ratio between the loss of positive and negative samples, and is used to control the imbalance between easily distinguishable and difficult-to-distinguish samples.

[0059] In this embodiment, before using the second deep learning model to perform anomaly detection on the current local image, the method further includes: acquiring a positive sample video of a rubber product without defects during the extrusion process; extracting multiple keyframe images from the positive sample video; for each keyframe image, cropping multiple local images from the keyframe image according to a sliding window, and using the multiple local images as positive samples; and using the positive samples to train the anomaly detection model to obtain the second deep learning model.

[0060] In this embodiment, an anomaly detection model is selected and trained using the positive samples mentioned above. The model can also be further fine-tuned using a hard sample mining strategy to optimize it.

[0061] Specifically, extracting multiple keyframe images from the positive sample video includes: retaining the first frame image of the positive sample video as a keyframe image; calculating the image similarity between the current frame image and the previous frame image for the current frame image after the first frame image; if the image similarity is lower than a preset threshold, then the current frame image is used as a keyframe image.

[0062] Data acquisition primarily involves recording videos of the extrusion inspection of rubber products on rails. Currently, keyframes are mainly obtained from these videos through frame extraction. While frame extraction is simple, it cannot quantify inter-frame differences, and setting the extraction interval is inconvenient; too small an interval can lead to redundant frames, while too large an interval can result in missed frames. To address this, this embodiment employs an image similarity-based strategy to obtain keyframes. Image similarity is calculated for adjacent consecutive frames, and the first frame is retained as the keyframe. For the current frame following the first frame, the image similarity between the current frame and the previous frame is calculated. If the image similarity is below a preset threshold, the current frame is used as the keyframe. This quantifies inter-frame differences, facilitating the acquisition of high-quality keyframes and improving the model's generalization ability. After obtaining the keyframes, a sliding window process is applied to each keyframe to obtain the final deep learning-based anomaly detection sample library.

[0063] In this embodiment, the defect detection of the candidate defect region using a classification model to obtain the defect result of the rubber product includes: expanding the candidate defect region to obtain the region to be detected; acquiring multiple consecutive historical frame images before the current frame image and determining the target region corresponding to the region to be detected in each historical frame image; inputting the candidate defect region image of the current frame image and the target region image corresponding to the historical frame images into the classification model respectively, and statistically analyzing the defect result of each frame image; if the number of frames with defect results is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect; if the number of frames with defect results is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

[0064] In this embodiment, defect detection of rail rubber products is achieved by extruding to better expose defects. During the extrusion process, the rail rubber products will undergo slight plastic deformation, resulting in small deformations in various regions. However, the actual displacement changes caused by the deformation are small. Therefore, by expanding the defect candidate area, the corresponding defect positions in the historical frames can be included.

[0065] The candidate defect regions in the current frame image and their corresponding regions in consecutive historical frames are fed into the defect classification and detection model. The defect detection results for each frame are statistically analyzed. If the number of frames detected as defects is greater than a set threshold, the candidate defect region in the current frame is considered a defect; otherwise, it is not considered a defect. Only the background (normal region) and foreground (defect region) need to be distinguished, so the classification model in this embodiment is a binary classification model. Only defect images and normal images need to be collected. For defect sample images, using the labeled defect locations, a sliding window size (the input resolution required by the classification model) is set with the defect region as the center. The sliding window slides in multiple directions at certain intervals with the defect region as the center, and relevant regions are captured and saved to compensate for insufficient defect samples and achieve data augmentation. Finally, defect samples are obtained. For normal regions, a sliding window of the same size is used to slide outside the defect region, and relevant regions are captured and saved to construct background samples.

[0066] In this embodiment, the classification model is first trained using the ordinary softmax loss function to obtain pre-trained weights. Then, using these pre-trained weights, the model is further fine-tuned using a hard sample mining strategy. The specific hard sample mining strategy is as follows: the softmax loss function values ​​are sorted from largest to smallest, and a certain proportion of the loss values ​​with the highest ranking are selected as the final loss values ​​for updating the model parameters, so as to achieve the purpose of hard sample mining and further optimize the model.

[0067] refer to Figure 3 This is a flowchart of the rail rubber defect detection process according to an embodiment of the present invention, including: acquiring video frames to be detected, performing sliding window processing on each frame image, performing preprocessing on each sliding window region, performing defect detection on the sliding window region to obtain candidate defect regions, performing defect classification detection on the candidate defect regions and several historical frame regions, detecting whether it is a defect image or a normal image, statistically analyzing the detection results, and determining whether the candidate defect region is a defect.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0069] Example 2

[0070] This embodiment also provides a defect detection device for rubber products, used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0071] Figure 6 This is a structural block diagram of a defect detection device for rubber products according to an embodiment of this application, as shown below. Figure 6 As shown, the device includes:

[0072] Acquisition module 60 is used to acquire video of the rubber product being extruded.

[0073] The cropping module 62 is used to crop multiple partial images from each frame of the video according to a sliding window, wherein the size of the partial image is the same as the size of the sliding window;

[0074] The first detection module 64 is used to perform defect detection on the rubber product image in each local image to obtain the candidate defect region of the current frame image;

[0075] The second detection module 66 is used to perform defect detection on the candidate defect area using a classification model to obtain the defect result of the rubber product.

[0076] Optionally, the first detection module includes a first detection unit, configured to perform target detection on the current local image using a first deep learning model to obtain a first defect result, and to perform anomaly detection on the current local image using a second deep learning model to obtain a second defect result, wherein the first deep learning model is trained using negative samples and the second deep learning model is trained using positive samples; merge the first defect result and the second defect result to obtain a defect detection result of the current local image; and merge the defect detection results of all local images to obtain a candidate defect region of the current frame image.

[0077] Optionally, the defect detection device for rubber products further includes a first training unit, used to acquire negative sample images containing defects and extract defect regions from the negative sample images; define a sliding window on the negative sample images, and slide the sliding window on the negative sample images in multiple directions with a preset interval, centered on the defect region; capture a local image of the sliding window on the negative sample images at each slide, and use the local image as a negative sample; use the negative samples to train the target detection model to obtain a first deep learning model.

[0078] Optionally, the first training unit is further configured to train the object detection model using the negative samples and the following loss function to obtain a first deep learning model: FL(p t )=-α t (1-p t ) γ log(p t ), where p t Let (1-p) be the probability that the result predicted by the first deep learning model is a defect. t ) γ A modulation factor, used to reduce the contribution of easily distinguishable samples, a t γ is used to adjust the ratio between the loss of positive and negative samples, and is used to control the imbalance between easily distinguishable and difficult-to-distinguish samples.

[0079] Optionally, the defect detection device for rubber products further includes a second training unit, used to acquire a positive sample video of the rubber product without defects during the extrusion process; extract multiple key frame images from the positive sample video; for each key frame image, extract multiple local images from the key frame image according to a sliding window, and use the multiple local images as positive samples; use the positive samples to train the anomaly detection model to obtain a second deep learning model.

[0080] Optionally, the second training unit is further configured to retain the first frame image of the positive sample video as a keyframe image; calculate the image similarity between the current frame image and the previous frame image of the current frame image after the first frame image; if the image similarity is lower than a preset threshold, then the current frame image is used as a keyframe image.

[0081] Optionally, the second detection module includes a second detection unit, used to expand the candidate defect region to obtain the region to be detected; acquire multiple consecutive historical frame images preceding the current frame image, and determine the target region corresponding to the region to be detected in each historical frame image; input the candidate defect region image of the current frame image and the target region image corresponding to the historical frame images into a classification model respectively, and count the defect results of each frame image; if the number of frames with defect results is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect; if the number of frames with defect results is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

[0082] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0083] Example 3

[0084] Embodiments of this application also provide a storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0085] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0086] S1, acquire video of the rubber product being extruded;

[0087] S2, for each frame of the video, extract multiple partial images from the image according to a sliding window, wherein the size of the partial image is the same as the size of the sliding window;

[0088] S3, perform defect detection on the rubber product image in each local image to obtain the candidate defect region of the current frame image;

[0089] S4. A classification model is used to detect defects in the candidate defect areas to obtain the defect results of the rubber product.

[0090] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0091] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0092] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0093] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0094] S1, acquire video of the rubber product being extruded;

[0095] S2, for each frame of the video, extract multiple partial images from the image according to a sliding window, wherein the size of the partial image is the same as the size of the sliding window;

[0096] S3, perform defect detection on the rubber product image in each local image to obtain the candidate defect region of the current frame image;

[0097] S4. A classification model is used to detect defects in the candidate defect areas to obtain the defect results of the rubber product.

[0098] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0099] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0100] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0105] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting defects in rubber products, characterized in that, The method includes: Obtain video footage of the rubber product being extruded; For each frame of the video, multiple partial images are cropped from the image using a sliding window, wherein the size of the partial images is the same as the size of the sliding window; Defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame image; A classification model is used to detect defects in the candidate defect regions to obtain the defect results of the rubber product; Specifically, defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame image, including: A first deep learning model is used to perform target detection on the current local image to obtain a first defect result, and a second deep learning model is used to perform anomaly detection on the current local image to obtain a second defect result. The first deep learning model is trained using negative samples with defects, and the second deep learning model is trained using positive samples without defects. The first defect result and the second defect result are combined to obtain the defect detection result of the current local image; The defect detection results of all local images are merged to obtain the candidate defect regions of the current frame image; The step of using a classification model to detect defects in the candidate defect regions to obtain the defect results of the rubber product includes: Expand the candidate defect region to obtain the region to be detected; Acquire consecutive historical frame images preceding the current frame image, and determine the target region corresponding to the region to be detected in each historical frame image; The candidate defect region image of the current frame image and the target region image corresponding to the historical frame image are respectively input into the classification model to count the defect results of each frame image. If the number of frames with defects is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect. If the number of frames with defects is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

2. The method according to claim 1, characterized in that, Before performing object detection on the current local image using the first deep learning model, the method further includes: Obtain a negative sample image containing defects, and extract the defective region from the negative sample image; A sliding window is defined on the negative sample image, and the sliding window is slid on the negative sample image in multiple directions with a preset interval, centered on the defect region. Extract a partial image of the sliding window on the negative sample image at each sliding action, and use the partial image as the negative sample; The target detection model is trained using the negative samples to obtain the first deep learning model.

3. The method according to claim 1, characterized in that, Before employing a second deep learning model to perform anomaly detection on the current local image, the method further includes: Obtain positive sample videos of rubber products without defects during the extrusion process; Extract multiple keyframe images from the positive sample video; For each keyframe image, multiple local images are extracted from the keyframe image using a sliding window, and these multiple local images are used as positive samples. The positive samples are used to train the anomaly detection model to obtain a second deep learning model.

4. The method according to claim 3, characterized in that, The extraction of multiple keyframe images from the positive sample video includes: The first frame of the positive sample video is retained as a keyframe image; For the current frame image following the first frame image, calculate the image similarity between the current frame image and the previous frame image; If the image similarity is lower than a preset threshold, then the current frame image is used as the keyframe image.

5. A defect detection device for rubber products, characterized in that, include: The acquisition module is used to acquire video of the rubber product being extruded. The cropping module is used to crop multiple partial images from each frame of the video using a sliding window, wherein the size of the partial images is the same as the size of the sliding window; The first detection module is used to perform defect detection on the rubber product image in each local image to obtain the candidate defect region of the current frame image; The second detection module is used to perform defect detection on the candidate defect area using a classification model to obtain the defect result of the rubber product; Specifically, defect detection is performed on the rubber product image within each local image to obtain candidate defect regions for the current frame image, including: A first deep learning model is used to perform target detection on the current local image to obtain a first defect result, and a second deep learning model is used to perform anomaly detection on the current local image to obtain a second defect result. The first deep learning model is trained using negative samples with defects, and the second deep learning model is trained using positive samples without defects. The first defect result and the second defect result are combined to obtain the defect detection result of the current local image; The defect detection results of all local images are merged to obtain the candidate defect regions of the current frame image; The step of using a classification model to detect defects in the candidate defect regions to obtain the defect results of the rubber product includes: Expand the candidate defect region to obtain the region to be detected; Acquire consecutive historical frame images preceding the current frame image, and determine the target region corresponding to the region to be detected in each historical frame image; The candidate defect region image of the current frame image and the target region image corresponding to the historical frame image are respectively input into the classification model to count the defect results of each frame image. If the number of frames with defects is greater than a preset threshold, then the candidate defect region of the current frame image is determined to be a defect. If the number of frames with defects is less than or equal to the preset threshold, then the candidate defect region of the current frame image is determined to be a non-defect.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other through the communication bus; wherein: Memory, used to store computer programs; A processor for performing the steps of the method according to any one of claims 1 to 4 by running a computer program stored in memory.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program, when executed, performs the steps of the method according to any one of claims 1 to 4.

Images