Methods, devices and storage media for detecting abnormal scrap steel

Abstract

The application provides a scrap steel abnormal object detection method and device and a storage medium, relates to the technical field of communication, and can be used for scrap steel abnormal object detection. The method comprises the following steps: obtaining a target training data set; the target training data set comprises a plurality of first to-be-detected target images; inputting the target training data set into a preset neural network model for training to generate a target neural network model; the preset neural network model comprises a deformable convolution model and a side suppression model; the deformable convolution model is used for extracting a plurality of image features in the to-be-detected target image; the side suppression model is used for statistical analysis of the plurality of image features; and the target neural network model is used for identifying abnormal objects in the to-be-detected target image. The application is used for scrap steel abnormal object detection.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method, apparatus and storage medium for detecting abnormal scrap steel. Background Technology

[0002] Currently, with the development of 5G industrial networks and deep learning networks, existing scrap steel anomaly detection systems employ a single-stage YOLOv5 algorithm combined with an attention mechanism to generate anomaly detection models. However, due to the complex working scenarios of scrap steel identification and the presence of significant anomaly detection noise, the accuracy of scrap steel anomaly detection is low. Therefore, improving the accuracy of scrap steel anomaly detection has become an urgent technical problem to be solved. Summary of the Invention

[0003] This application provides a method, apparatus, and storage medium for detecting abnormal objects in scrap steel, which can perform the detection of abnormal objects in scrap steel.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, this application provides a method for detecting abnormal objects in scrap steel. The method includes: acquiring a target training dataset; the target training dataset includes multiple first target images to be detected; inputting the target training dataset into a preset neural network model for training to generate a target neural network model; the preset neural network model includes: a deformable convolutional model and a lateral inhibition model; the deformable convolutional model is used to extract multiple image features from the target images to be detected; the lateral inhibition model is used to perform statistical analysis on the multiple image features; and the target neural network model is used to identify abnormal objects in the target images to be detected.

[0006] In conjunction with the first aspect mentioned above, in one possible implementation, the preset neural network model further includes: a first preset model; the deformable convolutional model includes: a first deformable convolutional model, a second deformable convolutional model, and a third deformable convolutional model; the first preset model is used to segment the input target image; the first deformable convolutional model is used to extract first image features from the input target image; the second deformable convolutional model is used to pool the input target image; the third deformable convolutional model is used to generate target feature images; and the target training dataset is input into the preset neural network model for training, including: step 1, inputting each first target image to be detected in the target training dataset into the first preset model to determine multiple first sub-target images to be detected for each first target image to be detected; step 2, inputting the multiple first sub-target images to be detected into the first deformable convolutional model. Step 1: Determine at least one first image feature of the first target image to be detected; Step 2: Input the at least one first image feature of the first target image to be detected into a second deformable convolutional model to determine at least one second image feature of the first target image to be detected; The number of second image features is greater than the number of first image features; Step 3: Input the second target image feature into a third deformable convolutional model to generate a target feature map; Step 4: Input the target feature map into a side-suppression model for calculation, and determine whether the calculation result of the target feature map meets a preset threshold; Step 5: If the preset threshold is met, the preset neural network model is determined to be trained; Step 6: If the preset threshold is not met, the target parameters of the preset neural network model are adjusted, and steps 1, 2, 3, 4, 5, 6, and 7 are executed until the preset neural network model is trained.

[0007] In conjunction with the first aspect mentioned above, in one possible implementation, after generating the target neural network model, the method further includes: acquiring a second target image to be detected; the second target image to be detected is a target image to be detected acquired in real time; segmenting the second target image to be detected based on a preset ratio to generate multiple second sub-target images to be detected; inputting the multiple sub-target images to be detected into a preset neural network model for detection to determine whether there are any abnormal objects in the second target image to be detected; if there are abnormal objects, then determining the first target image as the target detection result; marking the second target image to be detected with the abnormal object in the first target image; if there are no abnormal objects, then determining the target hint as the target detection result; the target hint is used to indicate that there are no abnormal objects in the second target image to be detected.

[0008] In conjunction with the first aspect above, in one possible implementation, before acquiring the target training dataset, the method further includes: acquiring multiple first initial images and multiple second initial images; the first initial images are images of scrap steel; the second initial images are images of anomalies; preprocessing each first initial image and each second initial image to generate a first initial target image and a second initial target image; and superimposing the first initial target image and the second initial target image based on a preset algorithm to generate a first target image to be detected.

[0009] Secondly, this application provides a scrap steel abnormality detection device, the device comprising: a processing unit; the processing unit being used to acquire a target training dataset; the target training dataset including multiple first target images to be detected; the processing unit being further used to input the target training dataset into a preset neural network model for training, generating a target neural network model; the preset neural network model including: a deformable convolutional model and a lateral inhibition model; the deformable convolutional model being used to extract multiple image features from the target images to be detected; the lateral inhibition model being used to perform statistical analysis on the multiple image features; and the target neural network model being used to identify abnormalities in the target images to be detected.

[0010] In conjunction with the second aspect above, in one possible implementation, the preset neural network model further includes: a first preset model; the deformable convolutional model includes: a first deformable convolutional model, a second deformable convolutional model, and a third deformable convolutional model; the first preset model is used to segment the input target image; the first deformable convolutional model is used to extract the first image features of the input target image; the second deformable convolutional model is used to pool the input target image; the third deformable convolutional model is used to generate the target feature image; the processing unit is specifically used for: step 1, inputting each first target image to be detected in the target training dataset into the first preset model to determine multiple first sub-target images to be detected for each first target image to be detected; step 2, inputting the multiple first sub-target images to be detected into the first deformable convolutional model to determine the first target image to be detected. Step 3: Input at least one first image feature of the first target image to be detected into the second deformable convolutional model to determine at least one second image feature of the first target image to be detected; the number of second image features is greater than the number of first image features; Step 4: Input the second target image features into the third deformable convolutional model to generate a target feature map; Step 5: Input the target feature map into the side-inhibition model for calculation to determine whether the calculation result of the target feature map meets the preset threshold; Step 6: If the preset threshold is met, the preset neural network model is determined to be trained; Step 7: If the preset threshold is not met, the target parameters of the preset neural network model are adjusted, and steps 1, 2, 3, 4, 5, 6, and 7 are executed until the preset neural network model is trained.

[0011] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: acquire a second target image to be detected; the second target image to be detected is a target image to be detected acquired in real time; segment the second target image to be detected based on a preset ratio to generate multiple second sub-target images to be detected; input the multiple sub-target images to be detected into a preset neural network model for detection to determine whether there is an anomalous object in the second target image to be detected; if there is an anomalous object, determine the first target image as the target detection result; mark the second target image to be detected with the anomalous object in the first target image; if there is no anomalous object, determine the target prompt as the target detection result; the target prompt is used to indicate that there is no anomalous object in the second target image to be detected.

[0012] In conjunction with the second aspect above, in one possible implementation, the processing unit is further configured to: acquire multiple first initial images and multiple second initial images; the first initial images are images of scrap steel; the second initial images are images of anomalies; preprocess each first initial image and each second initial image to generate a first initial target image and a second initial target image; and superimpose the first initial target image and the second initial target image based on a preset algorithm to generate a first target image to be detected.

[0013] Thirdly, this application provides a scrap steel anomaly detection device, which includes a processor and a memory; wherein the memory is used to store computer execution instructions, and when the scrap steel anomaly detection device is running, the processor executes the computer execution instructions stored in the memory to cause the scrap steel anomaly detection device to perform the scrap steel anomaly detection method as described in the first aspect and any possible implementation of the first aspect.

[0014] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed by a processor of a scrap metal anomaly detection device, enable the scrap metal anomaly detection device to perform the scrap metal anomaly detection method as described in the first aspect and any possible implementation thereof.

[0015] Fifthly, this application provides a computer program product containing instructions that, when run on a scrap steel anomaly detection device, cause the scrap steel anomaly detection device to perform the scrap steel anomaly detection method as described in the first aspect and any possible implementation thereof.

[0016] In a sixth aspect, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run computer programs or instructions to implement the scrap steel anomaly detection method as described in the first aspect and any possible implementation thereof.

[0017] Specifically, the chip provided in this application embodiment also includes a memory for storing computer programs or instructions.

[0018] In this application, the name of the aforementioned scrap steel abnormality detection device does not limit the equipment or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0019] These or other aspects of this application will become more readily apparent in the following description.

[0020] The technical solution provided in this application brings at least the following beneficial effects: The scrap steel anomaly detection device acquires a target training dataset containing multiple first target images to be detected. The scrap steel anomaly detection device sequentially inputs the target training dataset into a preset neural network model for training according to a preset ratio, generating a target neural network model. The target neural network model is used to identify anomalies in the target images to be detected. In this way, the scrap steel anomaly detection device collects a target training set containing multiple first target images to be detected and inputs the target training set into the preset neural network model for training according to a preset ratio. Through multiple iterations, the preset neural network model continuously adjusts its parameters, improving the scrap steel detection capability and increasing the detection accuracy of scrap steel anomalies. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of a scrap steel abnormality detection device provided in an embodiment of this application;

[0022] Figure 2 This application provides a schematic flowchart of a method for detecting abnormal materials in scrap steel.

[0023] Figure 3 This is a schematic diagram of a target neural network model structure provided in an embodiment of this application;

[0024] Figure 4 This is another flowchart illustrating a method for detecting abnormal materials in scrap steel, provided as an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of an anomaly detection result provided in an embodiment of this application;

[0026] Figure 6 This is another flowchart illustrating a method for detecting abnormal materials in scrap steel, provided as an embodiment of this application.

[0027] Figure 7 This is a schematic diagram of a first target image to be detected provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of a scrap steel abnormality detection device provided in an embodiment of this application. Detailed Implementation

[0029] The following description, in conjunction with the accompanying drawings, details the method, apparatus, and storage medium for detecting abnormal scrap steel provided in this application.

[0030] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0031] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0032] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0033] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0034] Currently, scrap steel is an important recyclable resource in solid waste recycling and a crucial raw material in the steelmaking industry. To better utilize scrap steel resources during the steelmaking process, it is necessary to identify and classify it. Traditional scrap steel identification and classification methods primarily rely on manual processing, which leads to both low efficiency and low accuracy in detecting abnormal scrap materials.

[0035] In related technologies, with the development of 5G industrial networks and deep learning networks, existing scrap steel anomaly detection systems use a combination of 5G industrial networks and convolutional neural networks (CNNs) to autonomously identify and classify scrap steel. However, due to the low accuracy and generalization rate of CNNs, as well as the complexity of the scrap steel anomaly detection environment, the accuracy of scrap steel anomaly detection is low. Therefore, how to improve the accuracy of scrap steel identification has become an urgent technical problem to be solved.

[0036] To address the aforementioned technical problems, this application provides a method for detecting abnormal objects in scrap steel. In this method, a scrap steel abnormality detection device acquires a target training dataset containing multiple first target images to be detected. The scrap steel abnormality detection device sequentially inputs the target training dataset into a preset neural network model according to a preset ratio for training, generating a target neural network model. This target neural network model is used to identify abnormal objects in the target images to be detected. Thus, the scrap steel abnormality detection device collects a target training set containing multiple first target images to be detected and inputs the target training set into the preset neural network model for training according to a preset ratio. Through multiple iterations, the preset neural network model continuously adjusts its parameters, improving its scrap steel detection capability and increasing the detection accuracy of abnormal objects in scrap steel.

[0037] Figure 1 This is a schematic diagram of the structure of a scrap steel abnormality detection device provided in an embodiment of this application. The scrap steel abnormality detection device 100 includes at least one processor 101, a communication line 102, and at least one communication interface 104, and may also include a memory 103. The processor 101, memory 103, and communication interface 104 can be connected to each other through the communication line 102.

[0038] The processor 101 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0039] Communication line 102 may include a path for transmitting information between the aforementioned components.

[0040] The communication interface 104 is used to communicate with other devices or communication networks. It can use any transceiver-like device, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc.

[0041] The memory 103 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of including or storing desired program code having the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0042] In one possible design, the memory 103 can exist independently of the processor 101, meaning the memory 103 can be an external memory of the processor 101. In this case, the memory 103 can be connected to the processor 101 via the communication line 102 to store execution instructions or application code, and its execution is controlled by the processor 101 to implement the space measurement determination method provided in the following embodiments of this application. In another possible design, the memory 103 can also be integrated with the processor 101, meaning the memory 103 can be an internal memory of the processor 101. For example, the memory 103 can be a cache, which can be used to temporarily store some data and instruction information.

[0043] As one possible implementation, processor 101 may include one or more CPUs, for example Figure 1 CPU0 and CPU1 in the example. Alternatively, the scrap metal anomaly detection device 100 may include multiple processors, such as... Figure 1 The processors 101 and 107 are included. Alternatively, the scrap metal anomaly detection device 100 may also include an output device 105 and an input device 106.

[0044] Through the above description of the implementation methods, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the network node can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, modules, and network nodes described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0045] Figure 2 The scrap steel abnormality detection method provided in the embodiments of this application, such as Figure 2 As shown, the scrap steel abnormality detection method provided in this application embodiment can be implemented through the following steps 201 to 202.

[0046] Step 201: The scrap steel anomaly detection device acquires the target training dataset.

[0047] The target training dataset includes multiple first target images to be detected.

[0048] In one possible implementation, the scrap steel anomaly detection device acquires a target training dataset comprising multiple first target images to be detected.

[0049] In one example, each first image to be detected in the target training dataset has a consistent aspect ratio. For instance, the aspect ratio of each first image to be detected is 1.5. Each first image to be detected contains annotation information for the anomaly to be detected.

[0050] It should be noted that the target training dataset includes both large-scale and small-scale datasets. The number of the first target images to be detected in the large-scale dataset and the number of the first target images to be detected in the small-scale dataset can be set according to the actual detection scenario. For example, the number of the first target images to be detected in the small-scale dataset is 690, while the number of the first target images to be detected in the large-scale dataset is 4140.

[0051] Step 202: The scrap steel abnormality detection device inputs the target training dataset into the preset neural network model for training, and generates the target neural network model.

[0052] The preset neural network models include: a deformable convolutional model and a lateral inhibition model; the deformable convolutional model is used to extract multiple image features from the target image to be detected; the lateral inhibition model is used to perform statistical analysis on multiple image features; and the target neural network model is used to identify abnormal objects in the target image to be detected.

[0053] In one possible implementation, the scrap steel anomaly detection device inputs multiple first target images from the target training dataset into a preset neural network model for training according to a preset size.

[0054] In one example, the scrap steel anomaly detection device inputs multiple first target object images from the target training set into a preset neural network model at a size of (640, 640, 3) according to a preset ratio. The algorithm for identifying anomalies in the target images within the preset neural network model can be trained using the YOLOv7 network architecture.

[0055] Specifically, such as Figure 3 The diagram shows the target neural network model structure, which includes the YOLOv7 backbone network. The scrap steel anomaly detection device replaces all 3×3 convolutions (Max Pooling-Convolution (MP-Conv), Spatial Pyramid Pooling-Convolutions-Spatial Pyramid Convolution (SPPCSPC), and Enhanced Large Activation Network Blocks (ELANB)) in the fifth structural block of the YOLOv7 backbone network with deformable convolutional networks (DCN). Furthermore, it adds selective information maximization attention modules (SimAM) before the three scales of the feature fusion layer.

[0056] It should be noted that the performance metrics of the target neural network model include, but are not limited to, precision, recall, and mean average precision (mAP). Precision, recall, and mAP satisfy the following formulas 1, 2, and 3, respectively: Precision Recall rate

[0057] Where TP is the number of positive samples predicted as positive by the model; FP is the number of negative samples predicted as positive by the model; FN is the number of negative samples predicted as negative by the model; mAP is the average precision of different classes under a fixed intersection-union ratio; AP represents the area under the PR curve plotted with recall on the x-axis and precision on the y-axis. AP also satisfies the following formula 4:

[0058] Furthermore, Table 1 shows a comparison of the training data for the target neural network model.

[0059] Table 1. Comparison of Training Data for Target Neural Network Models

[0060]

[0061]

[0062] Wherein, mean precision mAP50 represents the mean of the mean precision rate (IoU = 0.5); mean precision mAP50:95 represents the mean of the mean precision rate (IoU = 0.5:0.95).

[0063] The above scheme brings at least the following beneficial effects: The scrap steel anomaly detection device acquires a target training dataset containing multiple images of first targets to be detected. The scrap steel anomaly detection device sequentially inputs the target training dataset into a preset neural network model according to a preset ratio for training, generating a target neural network model. This target neural network model is used to identify anomalies in the target images to be detected. In this way, the scrap steel anomaly detection device collects a target training set containing multiple images of first targets to be detected and inputs the target training set into the preset neural network model for training according to a preset ratio. Through multiple iterations, the preset neural network model continuously adjusts its parameters, improving its scrap steel detection capability and increasing the detection accuracy of scrap steel anomalies.

[0064] In one possible implementation, step 202 above, where the scrap steel anomaly detection device inputs the target training dataset into a preset neural network model for training, can be achieved through the following steps 1-7:

[0065] The preset neural network model further includes: a first preset model; the deformable convolution model includes: a first deformable convolution model, a second deformable convolution model, and a third deformable convolution model; the first preset model is used to segment the input target image; the first deformable convolution model is used to extract the first image features of the input target image; the second deformable convolution model is used to pool the input target image; and the third deformable convolution model is used to generate the target feature image.

[0066] Step 1: Input each first target image to be detected in the target training dataset into the first preset model to determine multiple first sub-target images to be detected for each first target image;

[0067] Step 2: Input multiple first target images to be detected into the first deformable convolutional model to determine at least one first image feature of the first target image to be detected;

[0068] Step 3: Input at least one first image feature of the first target image to be detected into the second deformable convolution model to determine at least one second image feature of the first target image to be detected; the number of second image features is greater than the number of first image features;

[0069] Step 4: Input the second target image features into the third deformable convolutional model to generate the target feature map;

[0070] Step 5: Input the target feature map into the side suppression model for calculation, and determine whether the calculation result of the target feature map meets the preset threshold;

[0071] Step 6: If the preset threshold is met, the preset neural network model training is considered complete.

[0072] Step 7: If the preset threshold is not met, adjust the target parameters of the preset neural network model and execute steps 1, 2, 3, 4, 5, 6, and 7 until the preset neural network model is trained.

[0073] Combination Figure 2 ,like Figure 4 As shown, after generating the target neural network model, the following steps 401-405 are also included:

[0074] Step 401: The scrap steel abnormality detection device acquires the image of the second target to be detected.

[0075] The second target image to be detected is a target image acquired in real time.

[0076] The first initial image is an image of scrap steel; the second initial image is an image of an anomaly.

[0077] In one possible implementation, after the preset neural network model is trained, the scrap steel anomaly detection device inputs the second target image to be detected, which is collected in real time, into the trained preset neural network to identify scrap steel anomalies.

[0078] In one example, a scrap steel anomaly detection device acquires images of scrap steel in real time during loading and unloading. The device performs image segmentation and image augmentation on the acquired images to generate a second target image. This second target image is then input into a pre-trained neural network model for anomaly identification. Based on the identification results, the device determines the anomaly detection result for the scrap steel.

[0079] Specifically, if no anomalies are detected in the second target image, the scrap steel anomaly detection device generates a target detection result of "no anomalies detected" and terminates the scrap steel anomaly detection program. Optionally, if anomalies are detected in the second target image, the scrap steel anomaly detection device marks the anomalies in the second target image based on the anomaly detection box and generates a result such as... Figure 5 The diagram shows the results of the anomaly detection.

[0080] Step 402: The scrap steel abnormality detection device segments the second target image to be detected based on a preset ratio, generating multiple second sub-target images to be detected.

[0081] In one possible implementation, the scrap steel anomaly detection device segments the second target image to be detected according to a preset ratio, generating multiple second sub-target images to be detected.

[0082] In one example, the scrap metal anomaly detection device divides a second target image into multiple second target sub-images of size (640, 640, 3). Here, the size (640, 640, 3) indicates that the second target image is a color image with a width of 640 pixels, a height of 640 pixels, and three RGB channels.

[0083] Step 403: The scrap steel abnormality detection device inputs multiple second target images to be detected into a preset neural network model for detection to determine whether there are abnormalities in the second target images to be detected.

[0084] In one possible implementation, the scrap steel anomaly detection device inputs multiple second sub-target images of the same preset size into a preset neural network model for detection to determine whether there are anomalies in the second target images.

[0085] In one example, the scrap steel anomaly detection device inputs multiple second target images of size (640, 640, 3) into a preset neural network module. For example, the preset neural network model is a YOLOv7 network model. The scrap steel anomaly detection device uses the YOLOv7 network model to detect whether there are anomalies in the second target images.

[0086] Step 404: If there are abnormal objects, the scrap steel abnormal object detection device determines the first target image as the target detection result.

[0087] Among them, the first target image marks the second image to be detected of the anomaly.

[0088] In one possible implementation, if an anomaly is detected in the second target image, the scrap steel anomaly detection device marks the anomaly in the second target image based on the anomaly detection box, and determines the marked second target image to be detected as the target detection result.

[0089] Step 405: If no abnormal object is found, the scrap steel abnormal object detection device will determine the target prompt as the target detection result.

[0090] The target cue is used to indicate that there are no abnormal objects in the second image to be detected.

[0091] In one possible implementation, if there are no abnormal objects in the second image to be detected, the scrap steel abnormal object detection device will generate target prompt information as the target detection result.

[0092] In one example, if no abnormality is found in the second image to be detected, the scrap steel abnormality detection device generates a target prompt message of "no abnormality detected" and determines the target prompt message as the target detection result.

[0093] The above scheme brings at least the following beneficial effects: The scrap steel anomaly detection device acquires multiple second target images in real time. These second target images are images of scrap steel being loaded and unloaded from scrap steel vehicles, captured by the device. The device segments these second target images according to a preset ratio, generating multiple sub-target images. Further, the device inputs these sub-target images into a preset neural network model for detection, determining whether anomalies exist in the second target images. If anomalies are found, the device marks them, generating a first target image. If no anomalies are found, the device generates a target prompt. This prompt indicates that no target is detected in the second target image. Thus, during the detection and analysis of scrap steel images, the scrap steel anomaly detection device improves the accuracy and efficiency of scrap steel anomaly identification by real-time image acquisition and segmentation, detection using a preset neural network model, marking anomalies, and generating target prompts.

[0094] Combination Figure 2 ,like Figure 6As shown, before step 201 above, that is, before the scrap steel anomaly detection device acquires the target training dataset, the following steps 601-603 are also included:

[0095] Step 601: The scrap steel abnormality detection device acquires multiple first initial images and multiple second initial images.

[0096] The first initial image is an image of scrap steel, and the second initial image is an image of an anomaly.

[0097] In one possible implementation, the scrap steel anomaly detection device acquires multiple images of scrap steel as a first initial image, and acquires multiple images of anomalies as a second initial image.

[0098] In one example, the scrap steel anomaly detection device uses images of scrap steel being unloaded from scrap steel trucks captured by cameras at the steel plant as the first initial image. The device then uses images of multiple anomalies obtained through public datasets, web crawlers, or other methods as the second initial image. These anomalies include, but are not limited to, hazardous materials such as plastics, rubber, and gas cylinders, as well as impurities.

[0099] Step 602: The scrap steel abnormality detection device preprocesses each first initial image and each second initial image to generate a first initial target image and a second initial target image.

[0100] In one possible implementation, the scrap steel abnormality detection device performs image preprocessing on each acquired first initial image and each second initial image, and determines the preprocessed first initial image as the first initial image, and determines the preprocessed second initial image as the second initial image.

[0101] In one example, the scrap steel anomaly detection device crops the acquired scrap steel image according to a preset ratio, such as an aspect ratio of 1.5. Further, the device performs image augmentation on the cropped scrap steel image by random rotation, affine transformation, and color conversion, and determines the augmented scrap steel image as the first initial target image.

[0102] Optionally, the scrap steel anomaly detection device can label hazardous materials and impurities in the acquired anomaly images using Labelme software. Further, the device can crop the labeled hazardous materials and impurities using programs such as OpenCV. The scrap steel impurity detection device then performs image augmentation on the cropped hazardous materials and impurities, and determines the augmented image as the second initial target image.

[0103] Step 603: The scrap steel abnormality detection device superimposes the first initial target image and the second initial target image based on a preset algorithm to generate the first target image to be detected.

[0104] In one possible implementation, the scrap steel anomaly detection device will merge and superimpose the first initial target image and the second initial target image according to a preset algorithm to generate the first target image to be detected.

[0105] In one example, the scrap steel anomaly detection device uses a first initial target image, i.e., an image of scrap steel, as a background image. The device then randomly copies a second initial target image, i.e., an image of impurities or hazardous materials, into the first initial target image. Further, the device uses the copied position of the second initial target image as a relative position center and superimposes it onto the first initial target image through matrix operations to generate a first target image to be detected. For example, as... Figure 7 The image shown is a schematic diagram of the first target image generated.

[0106] The above scheme brings at least the following beneficial effects: The scrap steel anomaly detection device acquires multiple first initial images and multiple second initial images. The first initial image is an image of the scrap steel, and the second initial images are images of the anomaly. The scrap steel anomaly detection device preprocesses each first initial image and each second initial image to generate a first initial target image and a second initial target image. Further, the scrap steel anomaly detection device superimposes the first initial target image and the second initial target image according to a preset algorithm to generate a first target image to be detected. In this way, during the acquisition and processing of scrap steel images and anomaly images, the scrap steel anomaly detection device generates the first target image to be detected through preprocessing and superposition operations. This reduces background interference, highlights the anomaly, and thus improves the accuracy and effectiveness of anomaly detection.

[0107] The above provides a detailed description of the scrap steel abnormality detection device involved in the embodiments of this application, as well as the functions of each device in the scrap steel abnormality detection device and the interaction between the devices.

[0108] As can be seen, the above mainly describes the technical solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the modules and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] This application embodiment can divide the scrap steel abnormality detection device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0110] This application provides a scrap steel anomaly detection device for performing the method required by any device in the above-described scrap steel anomaly detection system. This scrap steel anomaly detection device can be the scrap steel anomaly detection device involved in this application, or a module within a scrap steel anomaly detection device; it can also be a chip within a scrap steel anomaly detection device, or other devices used to perform the scrap steel anomaly detection method; this application does not limit its scope.

[0111] Figure 8 The diagram shown is a structural schematic of a scrap steel abnormality detection device provided in an embodiment of this application. The device includes a processing unit 801 and a communication unit 802.

[0112] The processing unit 801 is used to acquire a target training dataset; the target training dataset includes multiple first target images to be detected; the processing unit 801 is also used to input the target training dataset into a preset neural network model for training to generate a target neural network model; the preset neural network model includes: a deformable convolutional model and a lateral inhibition model; the deformable convolutional model is used to extract multiple image features from the target images to be detected; the lateral inhibition model is used to perform statistical analysis on the multiple image features; the target neural network model is used to identify abnormal objects in the target images to be detected.

[0113] Optionally, the processing unit 801 is specifically used for: Step 1, inputting each first target image to be detected in the target training dataset into a first preset model to determine multiple first sub-target images to be detected for each first target image to be detected; Step 2, inputting the multiple first sub-target images to be detected into a first deformable convolutional model to determine at least one first image feature of the first target image to be detected; Step 3, inputting at least one first image feature of the first target image to be detected into a second deformable convolutional model to determine at least one second image feature of the first target image to be detected; the number of second image features is greater than the number of first image features; Step 4, inputting the second target image features into a third deformable convolutional model to generate a target feature map; Step 5, inputting the target feature map into a side-inhibition model for calculation to determine whether the calculation result of the target feature map meets a preset threshold; Step 6, if the preset threshold is met, the preset neural network model is determined to be trained; Step 7, if the preset threshold is not met, the target parameters of the preset neural network model are adjusted, and steps 1, 2, 3, 4, 5, 6, and 7 are executed until the preset neural network model is trained.

[0114] Optionally, the processing unit 801 is further configured to: acquire a second target image to be detected; the second target image to be detected is a target image to be detected acquired in real time; segment the second target image to be detected based on a preset ratio to generate multiple second sub-target images to be detected; input the multiple sub-target images to be detected into a preset neural network model for detection to determine whether there is an anomalous object in the second target image to be detected; if there is an anomalous object, determine the first target image as the target detection result; mark the second target image to be detected with the anomalous object in the first target image; if there is no anomalous object, determine the target prompt as the target detection result; the target prompt is used to indicate that there is no anomalous object in the second target image to be detected.

[0115] Optionally, the processing unit 801 is further configured to: acquire multiple first initial images and multiple second initial images; the first initial image is an image of scrap steel; the second initial image is an image of an anomaly; preprocess each first initial image and each second initial image to generate a first initial target image and a second initial target image; and superimpose the first initial target image and the second initial target image based on a preset algorithm to generate a first target image to be detected.

[0116] This application provides a scrap steel anomaly detection device for performing the method required by any device in the above-described scrap steel anomaly detection system. This scrap steel anomaly detection device can be the scrap steel anomaly detection device involved in this application, or a module within a scrap steel anomaly detection device; it can also be a chip within a scrap steel anomaly detection device, or other devices used to perform the scrap steel anomaly detection method; this application does not limit its scope.

[0117] This application also provides a computer-readable storage medium storing instructions. When a computer executes these instructions, the computer performs each step of the method flow shown in the above-described method embodiments.

[0118] Embodiments of this application provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the scrap steel anomaly detection method described in the above method embodiments.

[0119] Embodiments of this application provide a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to implement the scrap steel abnormality detection method as described in the above method embodiments.

[0120] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), registers, hard disks, optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing, or any other form of computer-readable storage medium in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Since the apparatus, device, computer-readable storage medium, and computer program product in the embodiments of this application can be applied to the above methods, the technical effects that can be obtained can also be referred to the above method embodiments. The embodiments of this application will not be repeated here.

[0122] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A scrap steel anomaly detection method characterized by, The method comprises: acquiring a target training data set; the target training data set comprises a plurality of first to-be-detected target images; inputting the target training data set into a preset neural network model for training to generate a target neural network model; the preset neural network model comprises a deformable convolution model and a lateral inhibition model; the deformable convolution model is used to extract a plurality of image features in a to-be-detected target image; the lateral inhibition model is used to statistically analyze the plurality of image features; the target neural network model is used to identify an abnormal object in the to-be-detected target image; the preset neural network model further comprises a first preset model; the deformable convolution model comprises a first deformable convolution model, a second deformable convolution model, and a third deformable convolution model; the first preset model is used to segment an input target image; the first deformable convolution model is used to extract first image features of the input target image; the second deformable convolution model is used to pool the input target image; and the third deformable convolution model is used to generate a target feature image; the inputting the target training data set into the preset neural network model for training comprises: step 1, inputting each of the first to-be-detected target images in the target training data set into the first preset model to determine a plurality of first to-be-detected sub-target images of each of the first to-be-detected target images; step 2, inputting the plurality of first to-be-detected sub-target images into the first deformable convolution model to determine at least one first image feature of the first to-be-detected target image; step 3, inputting the at least one first image feature of the first to-be-detected target image into the second deformable convolution model to determine at least one second image feature of the first to-be-detected target image; the number of the second image features is greater than the number of the first image features; step 4, inputting the second image features into the third deformable convolution model to generate a target feature map; step 5, inputting the target feature map into the lateral inhibition model for calculation to determine whether a calculation result of the target feature map meets a preset threshold; step 6, if the preset threshold is met, it is determined that the preset neural network model is trained; step 7, if the preset threshold is not met, adjusting a target parameter of the preset neural network model, and performing the steps 1, 2, 3, 4, 5, 6, and 7 until the preset neural network model is trained.

2. The method of claim 1, wherein, after the target neural network model is generated, the method further comprises: acquiring a second to-be-detected target image; the second to-be-detected target image is a real-time acquired to-be-detected target image; segmenting the second to-be-detected target image based on a preset ratio to generate a plurality of second to-be-detected sub-target images; inputting the plurality of second to-be-detected sub-target images into the preset neural network model for detection to determine whether the abnormal object exists in the second to-be-detected target image; If the abnormal object exists, determine the first target image as the target detection result; the first target image marks the second to-be-detected image of the abnormal object; If the abnormal object does not exist, determine the target prompt as the target detection result; the target prompt is used to represent that the second to-be-detected image does not exist.

3. The method of claim 1, wherein, Before the target training data set is acquired, further comprising: Acquire a plurality of first initial images and a plurality of second initial images; the first initial image is an image of scrap steel; the second initial image is an image of an abnormal object; Each of the first initial image and each of the second initial image is preprocessed to generate a first initial target image and a second initial target image; Based on a preset algorithm, the first initial target image and the second initial target image are superimposed to generate the first to-be-detected target image.

4. A scrap steel abnormality detection device characterized by comprising: The device comprises a processing unit; The processing unit is used to acquire a target training data set; the target training data set comprises a plurality of first to-be-detected target images; The processing unit is further used to input the target training data set into a preset neural network model for training to generate a target neural network model; the preset neural network model comprises a deformable convolution model and a side suppression model; the deformable convolution model is used to extract a plurality of image features in a to-be-detected target image; the side suppression model is used to statistically analyze the plurality of image features; the target neural network model is used to identify an abnormal object in a to-be-detected target image; The preset neural network model further comprises a first preset model; the deformable convolution model comprises a first deformable convolution model, a second deformable convolution model, and a third deformable convolution model; the first preset model is used to segment an input target image; the first deformable convolution model is used to extract a first image feature of the input target image; the second deformable convolution model is used to pool the input target image; and the third deformable convolution model is used to generate a target feature image; The processing unit is specifically used for: Step 1, input each of the first to-be-detected target images in the target training data set into the first preset model to determine a plurality of first to-be-detected sub-target images of each of the first to-be-detected target images; Step 2, input the plurality of first to-be-detected sub-target images into the first deformable convolution model to determine at least one first image feature of the first to-be-detected target image; Step 3, input the at least one first image feature of the first to-be-detected target image into the second deformable convolution model to determine at least one second image feature of the first to-be-detected target image; the number of the second image features is greater than the number of the first image features; Step 4, input the second image features into the third deformable convolution model to generate a target feature map; Step 5, input the target feature map into the side suppression model for calculation to determine whether a calculation result of the target feature map meets a preset threshold; Step 6, if the preset threshold is met, it is determined that the preset neural network model is trained. Step 7, if the preset threshold is not met, adjust the target parameter of the preset neural network model, and perform the step 1, the step 2, the step 3, the step 4, the step 5, the step 6, the step 7, until the preset neural network model training is completed.

5. The apparatus of claim 4, wherein, The processing unit is further configured to: obtain a second to-be-detected target image; the second to-be-detected target image is a real-time obtained to-be-detected target image; segment the second to-be-detected target image based on a preset ratio to generate a plurality of second to-be-detected sub-target images; input the plurality of second to-be-detected sub-target images into the preset neural network model for detection to determine whether the second to-be-detected target image contains the abnormal object; if the abnormal object exists, determine that the first target image is the target detection result; the first target image marks the second to-be-detected target image of the abnormal object; if the abnormal object does not exist, determine that the target prompt is the target detection result; the target prompt is used to represent that the second to-be-detected image does not contain the abnormal object.

6. The apparatus of claim 4, wherein, The processing unit is further configured to: obtain a plurality of first initial images and a plurality of second initial images; the first initial image is an image of scrap steel; the second initial image is an image of an abnormal object; preprocess each first initial image and each second initial image to generate a first initial target image and a second initial target image; superimpose the first initial target image and the second initial target image based on a preset algorithm to generate the first to-be-detected target image.

7. A scrap steel anomaly detection device characterized by comprising: comprising: a processor and a communication interface; the communication interface and the processor are coupled, and the processor is used to run computer programs or instructions to realize the scrap steel abnormal object detection method as claimed in any one of claims 1-3.

8. A computer-readable storage medium having stored therein instructions, the computer-readable storage medium comprising: When the computer executes the instruction, the computer executes the scrap steel abnormal object detection method as claimed in any one of claims 1-3.

Images