Method, device and equipment for supervising scrap steel transport vehicle and storage medium

Abstract

This application discloses a method, device, equipment, and storage medium for monitoring scrap steel transport vehicles, relating to the field of image recognition technology. The method includes: acquiring license plate images and cargo box images of scrap steel transport vehicles captured by an image acquisition device; extracting license plate character information from the license plate images using an OCRNet network model and establishing a license plate-timestamp-location binding relationship; performing multi-attribute recognition on the cargo box images using a cargo box classification model to obtain classification attributes including color category and cargo type; detecting the entry and exit status of scrap steel transport vehicles based on the license plate-timestamp-location binding relationship; when the same scrap steel transport vehicle has been detected to have completed both entry and exit, comparing the cargo box classification attributes at entry and exit; and triggering a cargo box replacement alarm based on the comparison result. This application can accurately identify cargo box replacement behavior of vehicles with counterfeit license plates during transportation, effectively preventing fraud risks and ensuring the safety of materials in steel mills.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a scrap steel transport vehicle supervision method, device, equipment and storage medium. BACKGROUND

[0002] Steel plants need to supervise hundreds of scrap steel transport vehicles every day in the whole chain of "entering the factory-weighing-leaving the factory" to prevent the replacement of the cargo box during transportation, the theft of scrap steel or the mixing of inferior materials, causing material loss and quality risk. The core requirement of supervision is to confirm in real time that "the cargo box of the vehicle with the same license plate number remains consistent when leaving the factory and when entering the factory".

[0003] Currently, the industry generally adopts manual or semi-automatic methods: taking pictures at the entrance, the weighbridge and the exit, manually comparing the license plate number, vehicle color and logo, and then visually checking the appearance of the cargo box. Some enterprises have introduced license plate recognition cameras, but they only focus on comparing license plate characters and lack automatic detection means for the type, shape and appearance changes of the cargo box inside.

[0004] Problems existing in the current approach: (1) Manual comparison is low in efficiency and easily affected by subjective factors, with a high rate of missed detection. (2) License plate recognition cannot identify the behavior of replacing or changing license plates. If the vehicle replaces the cargo box while the license plate remains unchanged, the traditional method cannot detect it. (3) In the case of similar cargo box appearance, light changes and partial occlusion, the misjudgment rate of manual or simple image threshold segmentation is more than 15%. (4) The data is scattered throughout the whole process, and the information at the entrance, the weighbridge and the exit does not form a closed loop, so it is impossible to associate and verify in real time "the same license plate-cargo box-time". Therefore, how to identify the replacement behavior of the cargo box of the license plate vehicle during transportation has become a problem to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The present application aims to provide a scrap steel transport vehicle supervision method, device, equipment and storage medium, which aims to solve the technical problem of how to identify the replacement behavior of the cargo box of the license plate vehicle during transportation.

[0007] To achieve the above-mentioned purpose, the present application provides a scrap steel transport vehicle supervision method, which comprises:

[0008] Obtaining the license plate image and the cargo box image of the scrap steel transport vehicle collected by an image collection device, wherein the image collection device is deployed at the entrance node and the exit node of the factory;

[0009] Extracting the license plate character information of the license plate image through an OCRNet network model, and establishing a license plate-time stamp-location binding relationship according to the license plate character information;

[0010] The multi-attribute recognition of the container image is performed through a container classification model to obtain classification attributes including color categories and cargo types, and the container classification model is obtained by improving a YOLOv3 classification network;

[0011] The entry and exit plant situations of the scrap steel transport vehicle are detected according to the license plate-time stamp-location binding relationship, and when it is detected that the same scrap steel transport vehicle has completed the entry and exit plant situations, the classification attributes of the first container image when the scrap steel transport vehicle enters the plant are compared with the classification attributes of the second container image when the scrap steel transport vehicle exits the plant to obtain a comparison result;

[0012] The container replacement alarm is triggered according to the comparison result.

[0013] In an embodiment, the container replacement alarm includes a first-level alarm and a second-level alarm.

[0014] The step of triggering the container replacement alarm according to the comparison result includes:

[0015] When the comparison result is that the color categories or the cargo types are inconsistent, a first-level alarm is triggered.

[0016] When the comparison result is that the color categories and the cargo types are consistent, the similarity of the first container image and the second container image is calculated through multiple image feature algorithms.

[0017] When the similarity calculated by all the image feature algorithms is lower than a preset similarity threshold, a second-level alarm is triggered.

[0018] In an embodiment, the similarity includes a first similarity, a second similarity, and a third similarity.

[0019] The step of calculating the similarity of the first container image and the second container image through multiple image feature algorithms when the comparison result is that the color categories and the cargo types are consistent includes:

[0020] When the comparison result is that the color categories and the cargo types are consistent, the overall pixel distribution similarity of the first container image and the second container image is calculated through a histogram algorithm to obtain a first similarity.

[0021] The gray pixel distribution similarity of the first container image and the second container image is calculated through a gray histogram algorithm to obtain a second similarity.

[0022] The RGB channel weighted fusion similarity of the first container image and the second container image is calculated through a three-channel histogram algorithm to obtain a third similarity.

[0023] In one embodiment, the similarity includes a fourth similarity, a fifth similarity, and a sixth similarity;

[0024] When the comparison result shows that both the color category and the cargo type are consistent, the step of calculating the similarity between the first cargo box image and the second cargo box image using multiple image feature algorithms includes:

[0025] The fourth similarity is obtained by calculating the overall pixel mean encoding similarity between the first cargo box image and the second cargo box image using the mean hash algorithm.

[0026] The fifth similarity is obtained by calculating the difference pattern similarity between adjacent pixels of the first cargo box image and the second cargo box image using the differential hashing algorithm.

[0027] The sixth similarity is obtained by calculating the frequency domain feature similarity between the first cargo box image and the second cargo box image using a perceptual hash algorithm.

[0028] In one embodiment, the steps for constructing and training the cargo container classification model include:

[0029] The improved network is obtained by replacing the backbone network of the YOLOv3 classification network with the Darknet-53 structure.

[0030] Collect a dataset of scrap steel images containing large silver blocks, large blocks of other colors, small silver blocks, small cylinders, steel bars, large scrap pieces, and small scrap pieces;

[0031] Based on the scrap steel image dataset, the improved network is trained using the backpropagation algorithm to obtain a cargo container classification model.

[0032] In one embodiment, the cargo box classification model includes an image size normalization unit, a convolutional neural network module, a feature pyramid network module, a color classification fully connected layer, and a cargo type classification fully connected layer;

[0033] The step of performing multi-attribute recognition on the cargo box image using a cargo box classification model to obtain classification attributes including color category and cargo type includes:

[0034] The input cargo box image is normalized in resolution using an image size normalization unit to obtain a uniformly sized RGB image;

[0035] The RGB image is used to extract features through a convolutional neural network module to obtain a multi-level feature map.

[0036] The multi-level feature maps are fused using the feature pyramid network module to generate a fused feature vector;

[0037] The fused feature vector is processed by a color classification fully connected layer to obtain a color category probability distribution, and the color category label corresponding to the maximum probability value in the color category probability distribution is selected.

[0038] The fused feature vector is processed by a fully connected layer for cargo type classification to obtain a cargo type probability distribution, and the cargo type label corresponding to the maximum probability value in the cargo type probability distribution is selected.

[0039] The color category label and the cargo type label are combined to form the classification attribute of the cargo box image.

[0040] In one embodiment, the OCRNet network includes a region detection branch, a geometric correction branch, and a character recognition branch;

[0041] The steps of extracting license plate character information from the license plate image using the OCRNet network model and establishing a license plate-timestamp-location binding relationship based on the license plate character information include:

[0042] The license plate region in the license plate image is located using a region detection branch.

[0043] A standard rectangular license plate image is obtained by performing perspective transformation on the license plate area through a geometric correction branch.

[0044] The license plate character information in the standard rectangular license plate image is identified through the character recognition branch;

[0045] Obtain the image acquisition timestamp and deployment location identifier of the image acquisition device;

[0046] A license plate-timestamp-location binding relationship is established based on the image acquisition timestamp, the deployment location identifier, and the license plate character information.

[0047] Furthermore, to achieve the above objectives, this application also proposes a monitoring device for scrap steel transport vehicles, the device comprising:

[0048] The data acquisition module is used to acquire license plate images and cargo box images of scrap steel transport vehicles collected by the image acquisition equipment, which is deployed at the inbound and outbound nodes of the plant area.

[0049] The license plate recognition module is used to extract license plate character information from the license plate image through the OCRNet network model, and to establish a license plate-timestamp-location binding relationship based on the license plate character information;

[0050] The cargo box recognition module is used to perform multi-attribute recognition on the cargo box image through a cargo box classification model to obtain classification attributes including color category and cargo type. The cargo box classification model is an improvement on the YOLOv3 classification network.

[0051] The attribute comparison module is used to detect the entry and exit status of the scrap steel transport vehicle based on the license plate-timestamp-location binding relationship. When the same scrap steel transport vehicle has been detected to have completed both entry and exit, the classification attribute of the first cargo box image when the scrap steel transport vehicle enters the factory is compared with the classification attribute of the second cargo box image when it leaves the factory to obtain the comparison result.

[0052] An alarm module is used to trigger a cargo box replacement alarm based on the comparison results.

[0053] In addition, to achieve the above objectives, this application also proposes a scrap steel transport vehicle monitoring device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the scrap steel transport vehicle monitoring method described above.

[0054] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the scrap steel transport vehicle supervision method described above.

[0055] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the scrap steel transport vehicle monitoring method described above.

[0056] One or more technical solutions proposed in this application have at least the following technical effects:

[0057] First, the scrap steel transportation monitoring system captures real-time images of license plates and cargo boxes using image acquisition devices deployed at the entry and exit points of the steel plant, providing basic data for subsequent identification and ensuring that vehicle status is traceable. Second, the system extracts license plate character information through an OCRNet network and establishes a license plate-timestamp-location binding relationship to achieve precise association between vehicle identity and spatiotemporal information, providing a unique identifier for comparison. Next, the system uses an improved YOLOv3 classification network to perform multi-attribute recognition on cargo box images, outputting color category and cargo type labels to accurately characterize the cargo box status. Subsequently, the system detects vehicle entry and exit based on the license plate binding relationship, compares the cargo box classification attributes at entry and exit, and quickly determines whether the cargo box has been replaced. Finally, based on the comparison results, a cargo box replacement alarm is triggered. This application can accurately identify cargo box replacement behavior by vehicles with counterfeit license plates during transportation, effectively preventing fraud risks and ensuring the safety of materials in steel plants. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating an embodiment of the method for supervising scrap steel transport vehicles in this application.

[0061] Figure 2 A schematic diagram of the data processing flow of the OCRNet network model provided in Embodiment 1 of the method for supervising scrap steel transport vehicles in this application;

[0062] Figure 3 A schematic diagram of the data processing flow of the cargo box classification model provided in Embodiment 1 of the method for supervising scrap steel transport vehicles in this application;

[0063] Figure 4 This is a flowchart illustrating Embodiment 2 of the method for supervising scrap steel transport vehicles in this application.

[0064] Figure 5 A simplified flowchart illustrating the scrap steel transport vehicle supervision method provided in Embodiment 2 of this application;

[0065] Figure 6 This is a schematic diagram of the module structure of the scrap steel transport vehicle monitoring device according to an embodiment of this application;

[0066] Figure 7This is a schematic diagram of the equipment structure of the hardware operating environment involved in the scrap steel transport vehicle supervision method in this application embodiment.

[0067] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0069] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0070] It should be noted that the executing entity of this application embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of realizing the above functions, such as a scrap steel transportation monitoring system. The following uses a scrap steel transportation monitoring system as an example to describe this embodiment and the following embodiments.

[0071] Based on this, the embodiments of this application provide a method for supervising scrap steel transport vehicles, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the scrap steel transport vehicle supervision method of this application.

[0072] In this embodiment, the method for monitoring scrap steel transport vehicles includes steps S10 to S50:

[0073] Step S10: Obtain the license plate image and cargo box image of the scrap steel transport vehicle captured by the image acquisition device, which is deployed at the inbound and outbound nodes of the factory area.

[0074] It should be noted that license plate images refer to digital images captured by industrial cameras when vehicles enter or exit, focusing on and completely containing the front or rear license plate area of ​​the vehicle, used to extract the license plate number and character information. Cargo box images refer to digital panoramic images taken from the same perspective, covering the entire interior of the cargo box containing scrap steel and its side walls, used for subsequent analysis of scrap steel categories and appearance characteristics. The entry node refers to the first access control point of the factory area and its associated weighbridge area, the fixed location where vehicles first enter the factory area and complete identification registration and weighing. The exit node refers to the last access control point of the factory area and its associated weighbridge area, the fixed location where vehicles, after loading scrap steel, complete verification and weighing before leaving the factory area.

[0075] It can be understood that the scrap steel transportation supervision system detects the vehicle arrival signal in real time through a trigger arranged in front of the inductive loop. Once triggered, it immediately sends synchronous flash and shutter commands to two industrial cameras. A narrow-angle camera continuously takes 3 frames at a shutter speed of 1 / 1000s aiming at the front license plate area. The system automatically selects the one with the highest clarity as the license plate image to reduce motion blur. Then, the system drives another wide-angle camera installed on the top of the gantry to take a downward shot of the cargo box at the same timestamp, and uses HDR synthesis technology to continuously take 5 frames at different exposure times to generate a 16-bit cargo box image that not only retains the details of the high-brightness scrap steel but does not overexpose the sky, ensuring the integrity of the scrap steel texture and color information.

[0076] Step S20: Extract the license plate character information of the license plate image through the OCRNet network model, and establish a license plate - timestamp - location binding relationship based on the license plate character information.

[0077] It should be noted that the OCRNet network model refers to a deep convolutional - attention hybrid architecture designed specifically for scene text recognition. It extracts local character features through multiple layers of convolution and uses the attention mechanism to capture the sequence context relationship, so as to output the license plate character sequence with high precision under natural lighting, soiled or tilted conditions. The license plate character information refers to the complete number string containing provincial Chinese characters, English letters, numbers and color attributes recognized from the license plate image, such as "Xiang A1xxx5 blue", which is used to uniquely identify a scrap steel transportation vehicle. The license plate - timestamp - location binding relationship refers to the triple mapping of the recognized license plate characters, the timestamp of the image acquisition moment, and the fixed geographical coordinate coding of the acquisition node in the factory area, so that the system can reverse-locate the corresponding vehicle, shooting moment and shooting location through any element at any time.

[0078] As an example, the OCRNet network includes a region detection branch, a geometric correction branch, and a character recognition branch; the steps of extracting the license plate character information of the license plate image through the OCRNet network model and establishing a license plate - timestamp - location binding relationship based on the license plate character information include: locating the license plate region in the license plate image through the region detection branch; performing perspective transformation on the license plate region through the geometric correction branch to obtain a standard rectangular license plate image; recognizing the license plate character information in the standard rectangular license plate image through the character recognition branch; obtaining the image acquisition timestamp and deployment location identifier of the image acquisition device; establishing a license plate - timestamp - location binding relationship based on the image acquisition timestamp, the deployment location identifier, and the license plate character information.

[0079] The region detection branch in the OCRNet network is a convolutional subnetwork responsible for predicting the bounding rectangle of the license plate within the entire license plate image. Its output is the coordinates of the top-left and bottom-right corners of the license plate in the pixel coordinate system. The geometric correction branch in the OCRNet network is a thin-plate spline transformation subnetwork that maps tilted or distorted license plate regions to a frontal view by predicting perspective transformation parameters. The character recognition branch in the OCRNet network is a sequence-to-sequence subnetwork that uses a bidirectional LSTM and attention decoder to output the license plate character sequence character by character. A license plate region refers to a rectangular pixel region in a license plate image that contains only the license plate itself and excludes the vehicle body background. A standard rectangular license plate image refers to a rectangular license plate image with a fixed aspect ratio, horizontally arranged characters, and no perspective distortion after geometric correction. An image acquisition timestamp is a time value in milliseconds generated by the system clock at the moment the camera shutter is triggered, used to mark the absolute capture time of that frame. A deployment location identifier is a unique spatial code for the image acquisition equipment within the factory area, consisting of a node number and geographical coordinates, used to distinguish different acquisition points such as those entering the factory, leaving the factory, or weighbridges.

[0080] First, the monitoring system sends the entire license plate image to the region detection branch. It first reduces computational load through a 1×1 convolution, then uses a two-layer feature pyramid network to slide anchor boxes at scales of 1 / 4, 1 / 8, and 1 / 16. A threshold of IoU > 0.7 is used to filter candidate boxes that best fit the license plate edge. Finally, the coordinates of the single box with the highest confidence are retained and immediately cropped to integer pixels, ensuring that subsequent networks only process the license plate itself and ignore the vehicle background. Second, the cropped license plate region is sent to the geometric correction branch. The network first predicts the offsets of the four corner points, then uses thin-plate spline interpolation to generate a 3×3 perspective transformation matrix in one step, mapping the original image to a 128×48 pixel horizontal rectangle. Bicubic interpolation anti-aliasing is performed during the mapping process, ensuring that a standard frontal view image is output regardless of the license plate's tilt, distortion, or distance, improving character recognition accuracy. Finally, the standard rectangular image is fed into the character recognition branch. First, two layers of bidirectional LSTM are used to extract the features of 8 frames of sequence. Then, an attention decoder with a coverage mechanism is used to output the 7-digit license plate string character by character. After each character is recognized, it is immediately compared with the dictionary. If the confidence is lower than 0.9, a second inference is triggered. At the same time, the system captures the millisecond-level Unix timestamp from the camera clock register and reads the node number and latitude and longitude from the local JSON configuration. The three are concatenated into a fixed format record and written to the Redis queue to ensure that the vehicle trajectory can be located in seconds using the "license plate + time + location" triplet during subsequent comparisons.

[0081] Please refer to Figure 2 , Figure 2This diagram illustrates the data processing flow of the OCRNet network model provided in Embodiment 1 of the scrap steel transport vehicle supervision method of this application. First, the system receives the original license plate image as input. Then, the region detection branch identifies and outputs the coordinates of the license plate region. Next, the geometric correction branch performs perspective transformation on the detected license plate region, outputting the corrected license plate image. Subsequently, the character recognition branch processes the corrected image to identify the license plate character sequence. Simultaneously, the system collects the current timestamp and deployment location identifier. Finally, the information binding module binds the identified character sequence with the collected timestamp and location identifier, forming a license plate-timestamp-location binding relationship, which serves as the final output. This process automatically realizes the detection, correction, recognition, and information binding of the license plate, significantly improving the efficiency and accuracy of scrap steel transport vehicle supervision.

[0082] Step S30: Perform multi-attribute recognition on the cargo box image using a cargo box classification model to obtain classification attributes including color category and cargo type. The cargo box classification model is an improvement on the YOLOv3 classification network.

[0083] It should be noted that the cargo container classification model refers to a convolutional neural network based on the YOLOv3 backbone, with added multi-task branches and fine-tuned using scrap steel scene data from steel mills. It is used to simultaneously output cargo container regions and their corresponding attribute labels. Color category refers to the discrete set of labels given by the model for the overall visual color of the scrap steel within the cargo container; typical values ​​include silver, brown, black, and mixed colors. Cargo type refers to the discrete set of labels for classifying the shape and size of the scrap steel; typical values ​​include large scrap, small scrap, rebar, and cylinder. Classification attribute refers to a binary label composed of color category and cargo type, used to uniquely describe the overall state of a cargo container image frame. The YOLOv3 classification network refers to a YOLOv3 derivative architecture using Darknet-53 as the backbone, removing the detector head and retaining only global average pooling and fully connected layers for image-level classification.

[0084] As an example, the cargo box classification model includes an image size normalization unit, a convolutional neural network module, a feature pyramid network module, a color classification fully connected layer, and a cargo type classification fully connected layer. The step of performing multi-attribute recognition on the cargo box image using the cargo box classification model to obtain classification attributes including color category and cargo type includes: normalizing the resolution of the input cargo box image using the image size normalization unit to obtain a uniformly sized RGB image; extracting features from the RGB image using the convolutional neural network module to obtain multi-level feature maps; fusing the multi-level feature maps using the feature pyramid network module to generate a fused feature vector; processing the fused feature vector using the color classification fully connected layer to obtain a color category probability distribution, and selecting the color category label corresponding to the maximum probability value in the color category probability distribution; processing the fused feature vector using the cargo type classification fully connected layer to obtain a cargo type probability distribution, and selecting the cargo type label corresponding to the maximum probability value in the cargo type probability distribution; and combining the color category label and the cargo type label to form the classification attribute of the cargo box image.

[0085] The image size normalization unit is a preprocessing sub-network that scales and fills any input cargo box image to a fixed resolution (e.g., 416×416 pixels), outputting a uniformly sized RGB image for subsequent network processing. The convolutional neural network module is a multi-layer convolutional structure with Darknet-53 as its backbone, responsible for extracting multi-level feature maps from the normalized image, ranging from shallow edges to deep semantics. The feature pyramid network module is a cross-layer connection and upsampling fusion structure used to compress and integrate the multi-scale feature maps output by the convolutional neural network into a single fused feature vector, while preserving detailed information about both large and small objects. The color classification fully connected layer is a single-layer perceptron with the fused feature vector as input and an output dimension equal to the number of color categories; its activation values, after Softmax transformation, represent the probability distribution of each color. The cargo type classification fully connected layer is a single-layer perceptron parallel to the color classification fully connected layer, with an output dimension equal to the number of cargo types; its activation values, after Softmax transformation, represent the probability distribution of each cargo type. Multi-level feature maps refer to a set of feature maps generated at different depths of a convolutional neural network, with spatial resolution decreasing progressively and the number of channels increasing progressively, corresponding to low-level local texture and high-level global semantics, respectively. A fused feature vector is a fixed-length one-dimensional vector obtained by weighted fusion of multi-level feature maps by a feature pyramid network, integrating global color and morphological information of the cargo container image. The color category probability distribution is the Softmax vector output by the fully connected layer for color classification, with each element representing the confidence score of the corresponding color label. The color category label is the discrete color name identifier corresponding to the dimension with the highest score in the color category probability distribution. The cargo type probability distribution is the Softmax vector output by the fully connected layer for cargo type classification, with each element representing the confidence score of the corresponding cargo type label.

[0086] First, the scrap steel transportation monitoring system sends the original cargo box image into a size normalization unit. It first scales the image to 416 pixels proportionally along the shorter side and fills the surrounding area with zeros to form a square. Then, it performs pixel value normalization to [-1,1] to ensure that all input tensors have consistent sizes and stable numerical distributions, preventing feature drift caused by differences in camera resolution. Second, the system sends the normalized RGB image into a convolutional neural network module. It uses 5 levels of stride convolution to downsample step-by-step, preserving residual connections in each level's output feature map. Subsequently, the feature pyramid network module upsamples the deepest 13×13 feature map by a factor of 2 and adds it element-wise to the corresponding 26×26 and 52×52 feature maps. This is then smoothed by 3×3 convolution and global average pooling to obtain a 2048-dimensional fused feature vector. This cross-scale fusion preserves both the global texture of large scrap steel pieces and the detailed edges of small steel bars, improving the robustness of subsequent classification. Finally, the system feeds the fused vector into the color classification fully connected layer and the goods type fully connected layer in parallel. After each layer outputs the Softmax probability, the argmax is taken to obtain the color label and type label. The fused vector is then concatenated into the final classification attribute in the "color-type" format and written into the cache queue for subsequent nodes to complete the comparison, thus avoiding redundant calculations.

[0087] Please refer to Figure 3 , Figure 3 This diagram illustrates the data processing flow of the cargo box classification model provided in Embodiment 1 of the scrap steel transport vehicle supervision method of this application. First, the original cargo box image is received as input. Then, the image size is normalized by an image size normalization unit. Next, a convolutional neural network module extracts multi-level feature maps. Subsequently, a feature pyramid network module fuses these feature maps to generate a fused feature vector. Then, a color classification fully connected layer and a cargo type classification fully connected layer process the fused feature vector, outputting color category probability distributions and cargo type probability distributions. The decision unit combines these two probability distributions to output the final classification attribute, which serves as the model's final output. This process automatically realizes the classification attribute recognition of cargo box images, significantly improving the efficiency and accuracy of scrap steel transport vehicle supervision.

[0088] As an example, the construction and training steps of the cargo box classification model include: replacing the backbone network of the YOLOv3 classification network with a Darknet-53 structure to obtain an improved network; collecting a dataset of scrap steel images containing large silver blocks, large blocks of other colors, small silver blocks, small cylinders, steel bars, large scrap pieces, and small scrap pieces; and training the improved network using the backpropagation algorithm based on the scrap steel image dataset to obtain the cargo box classification model.

[0089] The Darknet-53 architecture refers to a deep backbone network consisting of 53 convolutional layers and residual units, used to extract multi-level image features. The improved network refers to the overall model obtained by replacing the original backbone of the YOLOv3 classification network with Darknet-53. A large silver block refers to a single piece of scrap steel with a longest side ≥ 400mm and a projected area ≥ 0.16m². 2 Silver metallic blocks. Other colors and large blocks refer to single pieces of scrap steel with a longest side ≥ 400mm and a projected area ≥ 0.16m². 2 Non-silver metallic blocks. Small silver blocks refer to single pieces of scrap steel with the longest side < 400mm and a projected area < 0.16m². 2 The images depict silvery metallic blocks. Small cylinders refer to cylindrical scrap steel with a diameter of 30–80 mm and a length of 100–300 mm. Large scrap refers to irregularly shaped metal scrap with a longest side ≥ 400 mm. Small scrap refers to irregularly shaped metal scrap with a longest side < 400 mm. The scrap steel image dataset refers to a collection of RGB images collected on-site at steel mills and manually labeled according to the above categories.

[0090] First, the scrap steel transportation monitoring system modified the backbone field in the configuration file of the open-source YOLOv3 classification framework from Darknet-19 to Darknet-53, saved the weight paths, and recompiled to generate a new network skeleton with 53 layers of residual convolutions. Second, the system used three 12MP industrial cameras to continuously capture images at the access control, weighbridge, and factory exit for 28 days, randomly sampling by time slice each day. After manually selecting frames for each image, the images were automatically renamed according to seven categories and written to JSON, accumulating a total of 42,000 416×416 JPG samples to form the training set. Finally, the system performed backpropagation with an initial learning rate of 0.001, SGD momentum of 0.9, batch size of 32, and cosine annealing for 50 epochs. Every 5 epochs, mAP was calculated on the validation set. If the improvement was less than 0.5%, the system stopped early. After training, the weights were fixed and exported to ONNX, thus obtaining a cargo box classification model that can be directly deployed on edge GPUs.

[0091] Step S40: Detect the entry and exit status of the scrap steel transport vehicle based on the license plate-timestamp-location binding relationship. When the same scrap steel transport vehicle has been detected to have completed both entry and exit, compare the classification attribute of the first cargo box image when the scrap steel transport vehicle enters the factory with the classification attribute of the second cargo box image when it leaves the factory to obtain the comparison result.

[0092] It should be noted that "entry and exit status" refers to the complete closed-loop state of a scrap steel transport vehicle, where an "arrival" event is recorded at the factory entry node and a "departure" event is recorded at the factory exit node. The first cargo box image refers to the panoramic image of the cargo box captured at the factory entry node, used to extract initial classification attributes. The second cargo box image refers to the panoramic image of the cargo box captured at the factory exit node for the same vehicle with the same license plate, used for attribute comparison with the first cargo box image. The comparison result is a Boolean judgment output indicating whether the first and second cargo box images are consistent in the two-dimensional labels of color category and cargo type.

[0093] Understandably, the monitoring system uses the license plate-timestamp-location binding relationship as the key to maintain a status dictionary for each vehicle in Redis: when a license plate first appears in the "entering" node record, it is written as "status:entered"; when the same license plate subsequently appears in the "exiting" node record, it is updated to "status:exited". Once the status changes from entered to exited, the system immediately extracts the first cargo box image and its classification attributes from the corresponding entry's entry timestamp index, and simultaneously extracts the second cargo box image and its classification attributes from the exit timestamp index. The two sets of color categories are compared dimension by dimension with the cargo type labels. If they are equal, "consistent" is returned; otherwise, "inconsistent" is returned, and this boolean result is written to the comparison result field for subsequent alarm logic calls.

[0094] Step S50: Trigger a cargo box replacement alarm based on the comparison result.

[0095] It should be noted that the cargo box replacement alarm refers to an electronic alarm signal that includes the license plate number, timestamp, node location, and anomaly type.

[0096] Understandably, after receiving the comparison results, if the system finds an "inconsistency," it immediately writes a record to the memory alarm table, with fields including the license plate number, entry timestamp, exit timestamp, node coordinates, and exception code. Simultaneously, it pushes the record to the edge gateway via MQTT. The gateway then drives the on-site red audible and visual alarm to flash for 5 seconds and forwards the same JSON packet to the monitoring center and the duty mobile app, achieving a second-level cargo box replacement alarm.

[0097] This embodiment provides a method for monitoring scrap steel transport vehicles. First, the scrap steel transport monitoring system captures real-time images of license plates and cargo boxes using image acquisition devices deployed at the entry and exit points of the steel plant, providing basic data for subsequent identification and ensuring that vehicle status is traceable. Second, the system extracts license plate character information through an OCRNet network and establishes a license plate-timestamp-location binding relationship, achieving precise association between vehicle identity and spatiotemporal information, providing a unique identifier for comparison. Next, the system uses an improved YOLOv3 classification network to perform multi-attribute recognition on cargo box images, outputting color category and cargo type labels to accurately characterize the cargo box status. Subsequently, the system detects vehicle entry and exit based on the license plate binding relationship, comparing the cargo box classification attributes at entry and exit to quickly determine whether the cargo box has been replaced. Finally, a cargo box replacement alarm is triggered based on the comparison results. This embodiment can accurately identify cargo box replacement behavior by vehicles using counterfeit license plates during transportation, effectively preventing cheating risks and ensuring the safety of materials in steel mills.

[0098] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the scrap steel transport vehicle monitoring method of this application. The cargo box replacement alarm includes a first-level alarm and a second-level alarm. Step S50 of the scrap steel transport vehicle monitoring method includes steps S51 to S53:

[0099] Step S51: If the comparison result is inconsistent with the color category or the type of goods, a level one alarm is triggered.

[0100] It should be noted that a Level 1 alarm means that when the color category or cargo type does not match, the system immediately displays a red warning box on the monitoring screen and activates the on-site audible and visual alarm, indicating that there is a suspicion of cargo box substitution. A Level 2 alarm means that after the Level 1 alarm is triggered, the system further calls multiple algorithm similarity verifications. If the difference is confirmed to be significant, an emergency notification is sent to the on-duty personnel via SMS, telephone, and App, requiring immediate on-site verification.

[0101] Understandably, when the monitoring system detects a discrepancy in color category or cargo type label, it immediately writes the abnormal record to the Kafka queue within 200ms, triggers a red audible and visual alarm to flash for 5 seconds, displays the license plate number, the discrepancy category, and a snapshot thumbnail on the monitoring center's large screen, and pushes a desktop alarm to the duty computer, thus completing a level one alarm.

[0102] Step S52: When the comparison result shows that the color category and the cargo type are consistent, the similarity between the first cargo box image and the second cargo box image is calculated using multiple image feature algorithms.

[0103] It should be noted that image feature algorithms refer to a set of computational rules for extracting and quantifying color, texture, or shape features from cargo box images for comparison. Similarity refers to the numerical metric output by the above algorithms, representing the degree of visual similarity between the first and second cargo box images; a higher value indicates greater similarity.

[0104] As an example, the similarity includes a first similarity, a second similarity, and a third similarity; the step of calculating the similarity between the first cargo box image and the second cargo box image using multiple image feature algorithms when the comparison result shows that the color category and the cargo type are both consistent includes: when the comparison result shows that the color category and the cargo type are both consistent, calculating the overall pixel distribution similarity between the first cargo box image and the second cargo box image using a histogram algorithm to obtain a first similarity; calculating the grayscale pixel distribution similarity between the first cargo box image and the second cargo box image using a grayscale histogram algorithm to obtain a second similarity; and calculating the RGB channel weighted fusion similarity between the first cargo box image and the second cargo box image using a three-channel histogram algorithm to obtain a third similarity.

[0105] The first similarity refers to the overall pixel distribution similarity value obtained through the histogram algorithm. The histogram algorithm is a method that statistically analyzes the distribution of all pixel grayscale or color values ​​in an entire image and calculates the Bach distance or correlation coefficient between two distribution vectors. The overall pixel distribution similarity is a quantified value of the closeness between the first and second cargo box images in terms of the distribution of all pixel grayscale values. The second similarity refers to the grayscale pixel distribution similarity value obtained through the grayscale histogram algorithm. The grayscale histogram algorithm is a method that converts the image to single-channel grayscale, statistically analyzes the grayscale distribution, and calculates the similarity between two grayscale histograms. The grayscale pixel distribution similarity is a quantified value of the closeness between the first and second cargo box images in terms of the grayscale distribution. The third similarity refers to the RGB channel weighted fusion similarity value obtained through the three-channel histogram algorithm. The three-channel histogram algorithm is a method that statistically analyzes the R, G, and B channel histograms separately, fuses them according to a 1:1:1 ratio or empirical weights, and then calculates the similarity between the two fused histograms. RGB channel weighted fusion similarity refers to the quantified value of the closeness between the first cargo box image and the second cargo box image on the RGB three-channel fusion histogram.

[0106] First, the monitoring system uses a histogram algorithm to statistically analyze the global pixel distribution of the first and second cargo container images using 256 gray levels. The Barthel Index (BCI) of the two distribution vectors is then used as the first similarity score, quickly capturing overall brightness differences. Second, the two images are converted to single-channel grayscale using a ratio of 0.299R + 0.587G + 0.114B. A 256-bin histogram is then calculated again, and the Pearson correlation coefficient is determined to obtain the second similarity score, eliminating color interference and focusing on texture structure. Finally, the 256-bin histograms of the R, G, and B channels are extracted separately, linearly fused at a 1:1:1 ratio into a 768-dimensional vector, and the BCI is calculated to generate the third similarity score. This approach considers both color and brightness variations, improving the robustness of similarity assessment.

[0107] As an example, the similarity includes a fourth similarity, a fifth similarity, and a sixth similarity; the step of calculating the similarity between the first cargo box image and the second cargo box image using multiple image feature algorithms when the comparison result is that the color category and the cargo type are both consistent includes: calculating the overall pixel mean encoding similarity between the first cargo box image and the second cargo box image using a mean hash algorithm to obtain a fourth similarity; calculating the adjacent pixel difference pattern similarity between the first cargo box image and the second cargo box image using a difference hash algorithm to obtain a fifth similarity; and calculating the frequency domain feature similarity between the first cargo box image and the second cargo box image using a perceptual hash algorithm to obtain a sixth similarity.

[0108] The fourth similarity refers to the Hamming distance quantization value between the overall pixel mean codes obtained through the mean hash algorithm. The mean hash algorithm calculates the overall pixel mean after scaling the image to 8×8, converting it to grayscale, and then comparing each pixel with the mean to generate a 64-bit 0-1 code. The overall pixel mean code similarity refers to the degree of similarity reflected by the Hamming distance between the 64-bit hash codes of the first and second cargo box images. The fifth similarity refers to the Hamming distance quantization value between adjacent pixel difference patterns obtained through the difference hash algorithm. The difference hash algorithm compares the sizes of adjacent pixels horizontally after scaling the image to 9×8, converting it to grayscale, and generating a 64-bit 0-1 code. The adjacent pixel difference pattern similarity refers to the degree of similarity reflected by the Hamming distance between the 64-bit difference hash codes of the first and second cargo box images. The sixth similarity refers to the Hamming distance quantization value between frequency domain features obtained through the perceptual hash algorithm. Perceptual hashing is a hashing method that scales an image to 32×32, converts it to grayscale, performs a discrete cosine transform, extracts the low-frequency coefficients in the upper left 8×8 area, and generates a 64-bit 0-1 code. Frequency domain feature similarity refers to the degree of similarity reflected by the Hamming distance between the 64-bit perceptual hash codes of the first and second cargo box images.

[0109] First, the monitoring system uniformly scales the images of the first and second cargo containers to 8×8 grayscale images. After subtracting the global mean pixel by pixel, a 64-bit 0 / 1 code is obtained. The Hamming distance between the two codes is calculated as the fourth similarity to quickly capture the overall brightness difference. Second, the two images are scaled to 9×8 grayscale images. The size of each pair of adjacent pixels is compared in row-first, column-second order to generate a 64-bit difference hash code. The Hamming distance is then calculated to obtain the fifth similarity, which can enhance the detection of local texture changes. Finally, the two images are scaled to 32×32 grayscale images. After performing 8×8 DCT, the low-frequency coefficients of the upper left 8×8 are retained. After median binarization, a 64-bit perceptual hash code is obtained. The Hamming distance is then calculated to obtain the sixth similarity, thereby suppressing Gaussian noise and slight rotation, and improving robustness.

[0110] Step S53: When the similarity calculated by all the image feature algorithms is lower than the preset similarity threshold, a secondary alarm is triggered.

[0111] It should be noted that the preset similarity threshold refers to the minimum acceptable similarity value that the system pre-sets for each image feature algorithm. When the similarity obtained by any algorithm is lower than this value, the two images of the cargo box are judged to be significantly different.

[0112] Understandably, the monitoring system sequentially compares the six similarities obtained from the histogram, grayscale histogram, three-channel histogram, mean hash, difference hash, and perceptual hash with their respective preset thresholds of 0.85, 0.82, 0.80, 0.90, 0.88, and 0.87. If all are lower than the corresponding threshold, a secondary alarm packet is generated within 100ms and pushed to the duty mobile phone, monitoring screen, and audible and visual alarm via MQTT, indicating that the cargo box may have been replaced.

[0113] This embodiment first compares the color category and cargo type of the images of the inbound and outbound cargo boxes. If any dimension is inconsistent, a first-level alarm is immediately triggered: a red light flashes on site, a pop-up window appears on the monitoring screen, and management personnel can stop the vehicle immediately. If the color and type are consistent, the system then starts multiple algorithms such as histogram, mean hash, difference hash, and perceptual hash to calculate the similarity between the two images in parallel. When all results are lower than their respective preset thresholds, a second-level alarm is triggered: SMS, App, and stronger sound and light signals are pushed simultaneously, prompting manual review. The two-level alarm linkage can accurately identify cargo box replacement behavior of vehicles using counterfeit license plates during transportation.

[0114] For example, to help understand the implementation process of the scrap steel transport vehicle supervision method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 5 , Figure 5 A simplified flowchart illustrating a method for supervising scrap steel transport vehicles is provided, specifically:

[0115] First, the system acquires license plate and cargo box images collected at the inbound / outbound nodes. Then, the system recognizes the license plate images and establishes a binding relationship between the license plate, timestamp, and location. Simultaneously, the cargo box images are classified, generating classification attributes. Next, the system performs inbound / outbound checks, comparing the bound license plate information with the cargo box classification attributes. If the attributes do not match, a level one alarm is triggered. If the attributes match, the system uses six algorithms to calculate the similarity between the first and second cargo box images. If the similarity calculated by all algorithms is below a preset threshold, a level two alarm is triggered. If the similarity is not below the threshold, the process ends, and no anomalies are detected. This automated process achieves full-process monitoring of scrap steel transport vehicles, effectively identifying fraudulent activities such as cargo box swapping and cargo theft, significantly reducing the risk of material loss for steel mills.

[0116] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for supervising scrap steel transport vehicles in this application. Any simple modifications based on this technical concept are within the scope of protection of this application.

[0117] This application also provides a monitoring device for scrap steel transport vehicles; please refer to... Figure 6 The scrap steel transport vehicle monitoring device includes:

[0118] Data acquisition module 10 is used to acquire license plate images and cargo box images of scrap steel transport vehicles collected by image acquisition equipment, which is deployed at the inbound and outbound nodes of the plant area.

[0119] The license plate recognition module 20 is used to extract license plate character information from the license plate image through the OCRNet network model, and to establish a license plate-timestamp-location binding relationship based on the license plate character information;

[0120] The cargo box recognition module 30 is used to perform multi-attribute recognition on the cargo box image through a cargo box classification model to obtain classification attributes including color category and cargo type. The cargo box classification model is an improvement on the YOLOv3 classification network.

[0121] The attribute comparison module 40 is used to detect the entry and exit status of the scrap steel transport vehicle according to the license plate-timestamp-location binding relationship. When the same scrap steel transport vehicle has been detected to have completed both entry and exit, the classification attribute of the first cargo box image when the scrap steel transport vehicle enters the factory is compared with the classification attribute of the second cargo box image when it leaves the factory to obtain the comparison result.

[0122] Alarm module 50 is used to trigger a cargo box replacement alarm based on the comparison result.

[0123] The scrap steel transport vehicle monitoring device provided in this application, employing the scrap steel transport vehicle monitoring method described in the above embodiments, can solve the technical problem of how to identify cargo box replacement behavior by vehicles with counterfeit license plates during transportation. Compared with the prior art, the beneficial effects of the scrap steel transport vehicle monitoring device provided in this application are the same as those of the scrap steel transport vehicle monitoring method provided in the above embodiments, and other technical features in the scrap steel transport vehicle monitoring device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0124] This application provides a scrap steel transport vehicle monitoring device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the scrap steel transport vehicle monitoring method in the above embodiment 1.

[0125] The following is for reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing the scrap steel transport vehicle monitoring equipment in the embodiments of this application. The scrap steel transport vehicle monitoring equipment in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), vehicle-mounted terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The illustrated scrap steel transport vehicle monitoring equipment is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0126] like Figure 7As shown, the scrap metal transport vehicle monitoring equipment may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in ROM (Read Only Memory) 1002 or programs loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the scrap metal transport vehicle monitoring equipment. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the scrap metal transport vehicle monitoring equipment to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows scrap metal transport vehicle monitoring equipment with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0127] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0128] The scrap steel transport vehicle monitoring equipment provided in this application, employing the scrap steel transport vehicle monitoring method described in the above embodiments, can solve the technical problem of identifying cargo box replacement behavior by vehicles with counterfeit license plates during transportation. Compared with the prior art, the beneficial effects of the scrap steel transport vehicle monitoring equipment provided in this application are the same as those of the scrap steel transport vehicle monitoring method provided in the above embodiments, and other technical features in this scrap steel transport vehicle monitoring equipment are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0131] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the scrap steel transport vehicle supervision method in the above embodiments.

[0132] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0133] The aforementioned computer-readable storage medium may be included in the scrap steel transport vehicle monitoring equipment; or it may exist independently and not be installed in the scrap steel transport vehicle monitoring equipment.

[0134] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the scrap steel transport vehicle monitoring equipment, the equipment performs the following actions: acquires license plate and cargo box images of the scrap steel transport vehicle from image acquisition devices deployed at the factory's entry and exit nodes; extracts license plate character information from the license plate image using an OCRNet network model and establishes a license plate-timestamp-location binding relationship based on the license plate character information; performs multi-attribute recognition on the cargo box image using a cargo box classification model to obtain classification attributes including color category and cargo type, wherein the cargo box classification model is an improvement on the YOLOv3 classification network; detects the entry and exit status of the transport vehicle based on the license plate-timestamp-location binding relationship; when the same transport vehicle has completed both entry and exit, compares the classification attributes of the first cargo box image when the vehicle enters the factory with the classification attributes of the second cargo box image when the vehicle exits the factory to obtain a comparison result; and triggers a cargo box replacement alarm based on the comparison result.

[0135] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0138] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described method for monitoring scrap steel transport vehicles. This method can solve the technical problem of identifying cargo box replacement behavior by vehicles using counterfeit license plates during transportation. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the scrap steel transport vehicle monitoring method provided in the above embodiments, and will not be elaborated upon here.

[0139] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the scrap steel transport vehicle monitoring method described above.

[0140] The computer program product provided in this application can solve the technical problem of how to identify cargo box replacement behavior of vehicles with counterfeit license plates during transportation. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the scrap steel transportation vehicle supervision method provided in the above embodiments, and will not be repeated here.

[0141] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for supervising scrap steel transport vehicles, characterized in that, The method includes: The image acquisition device acquires images of the license plate and cargo box of the scrap steel transport vehicle, which are deployed at the inbound and outbound nodes of the plant area. The license plate character information of the license plate image is extracted using the OCRNet network model, and a license plate-timestamp-location binding relationship is established based on the license plate character information; The cargo box image is subjected to multi-attribute recognition by a cargo box classification model to obtain classification attributes including color category and cargo type. The cargo box classification model is an improvement of the YOLOv3 classification network. The entry and exit status of the scrap steel transport vehicle is detected based on the license plate-timestamp-location binding relationship. When the same scrap steel transport vehicle has been detected to have completed both entry and exit, the classification attribute of the first cargo box image when the scrap steel transport vehicle enters the factory is compared with the classification attribute of the second cargo box image when it leaves the factory to obtain the comparison result. Based on the comparison results, a cargo box replacement alarm is triggered. The cargo box classification model includes an image size normalization unit, a convolutional neural network module, a feature pyramid network module, a color classification fully connected layer, and a cargo type classification fully connected layer. The step of performing multi-attribute recognition on the cargo box image using a cargo box classification model to obtain classification attributes including color category and cargo type includes: The input cargo box image is normalized in resolution using an image size normalization unit to obtain a uniformly sized RGB image; The RGB image is used to extract features through a convolutional neural network module to obtain a multi-level feature map. The multi-level feature maps are fused using the feature pyramid network module to generate a fused feature vector; The fused feature vector is processed by a color classification fully connected layer to obtain a color category probability distribution, and the color category label corresponding to the maximum probability value in the color category probability distribution is selected. The fused feature vector is processed by a fully connected layer for cargo type classification to obtain a cargo type probability distribution, and the cargo type label corresponding to the maximum probability value in the cargo type probability distribution is selected. The color category label and the cargo type label are combined to form the classification attribute of the cargo box image.

2. The method as described in claim 1, characterized in that, The cargo box replacement alarm includes a level one alarm and a level two alarm; The step of triggering the cargo box replacement alarm based on the comparison result includes: If the comparison result is inconsistent with the color category or the type of goods, a level one alarm is triggered. When the comparison result shows that the color category and the cargo type are both consistent, the similarity between the first cargo box image and the second cargo box image is calculated using multiple image feature algorithms; A level 2 alarm is triggered when the similarity calculated by all the image feature algorithms is lower than a preset similarity threshold.

3. The method as described in claim 2, characterized in that, The similarity includes a first similarity, a second similarity, and a third similarity; When the comparison result shows that both the color category and the cargo type are consistent, the step of calculating the similarity between the first cargo box image and the second cargo box image using multiple image feature algorithms includes: When the comparison result is that the color category and the cargo type are both consistent, the overall pixel distribution similarity between the first cargo box image and the second cargo box image is calculated by the histogram algorithm to obtain the first similarity. The grayscale pixel distribution similarity between the first cargo box image and the second cargo box image is calculated using a grayscale histogram algorithm to obtain a second similarity score. The third similarity is obtained by calculating the RGB channel weighted fusion similarity between the first cargo box image and the second cargo box image using a three-channel histogram algorithm.

4. The method as described in claim 2, characterized in that, The similarity includes fourth similarity, fifth similarity, and sixth similarity; When the comparison result shows that both the color category and the cargo type are consistent, the step of calculating the similarity between the first cargo box image and the second cargo box image using multiple image feature algorithms includes: The fourth similarity is obtained by calculating the overall pixel mean encoding similarity between the first cargo box image and the second cargo box image using the mean hash algorithm. The fifth similarity is obtained by calculating the difference pattern similarity between adjacent pixels of the first cargo box image and the second cargo box image using the differential hashing algorithm. The sixth similarity is obtained by calculating the frequency domain feature similarity between the first cargo box image and the second cargo box image using a perceptual hash algorithm.

5. The method as described in claim 1, characterized in that, The steps for constructing and training the cargo container classification model include: The improved network is obtained by replacing the backbone network of the YOLOv3 classification network with the Darknet-53 structure. Collect a dataset of scrap steel images containing large silver blocks, large blocks of other colors, small silver blocks, small cylinders, steel bars, large scrap pieces, and small scrap pieces; Based on the scrap steel image dataset, the improved network is trained using the backpropagation algorithm to obtain a cargo container classification model.

6. The method according to any one of claims 1 to 5, characterized in that, The OCRNet network includes a region detection branch, a geometric correction branch, and a character recognition branch; The steps of extracting license plate character information from the license plate image using the OCRNet network model and establishing a license plate-timestamp-location binding relationship based on the license plate character information include: The license plate region in the license plate image is located using a region detection branch. A standard rectangular license plate image is obtained by performing perspective transformation on the license plate area through a geometric correction branch. The license plate character information in the standard rectangular license plate image is identified through the character recognition branch; Obtain the image acquisition timestamp and deployment location identifier of the image acquisition device; A license plate-timestamp-location binding relationship is established based on the image acquisition timestamp, the deployment location identifier, and the license plate character information.

7. A monitoring device for scrap steel transport vehicles, characterized in that, The device includes: The data acquisition module is used to acquire license plate images and cargo box images of scrap steel transport vehicles collected by the image acquisition equipment, which is deployed at the inbound and outbound nodes of the plant area. The license plate recognition module is used to extract license plate character information from the license plate image through the OCRNet network model, and to establish a license plate-timestamp-location binding relationship based on the license plate character information; The cargo box recognition module is used to perform multi-attribute recognition on the cargo box image using a cargo box classification model to obtain classification attributes including color category and cargo type. The cargo box classification model is an improvement on the YOLOv3 classification network. The cargo box classification model includes an image size normalization unit, a convolutional neural network module, a feature pyramid network module, a fully connected layer for color classification, and a fully connected layer for cargo type classification. The step of performing multi-attribute recognition on the cargo box image using the cargo box classification model to obtain classification attributes including color category and cargo type includes: normalizing the resolution of the input cargo box image using the image size normalization unit to obtain uniformly sized RGB images. The image is processed as follows: Features are extracted from the RGB image using a convolutional neural network module to obtain multi-level feature maps; the multi-level feature maps are fused using a feature pyramid network module to generate a fused feature vector; the fused feature vector is processed through a color classification fully connected layer to obtain a color category probability distribution, and the color category label corresponding to the maximum probability value in the color category probability distribution is selected; the fused feature vector is processed through a cargo type classification fully connected layer to obtain a cargo type probability distribution, and the cargo type label corresponding to the maximum probability value in the cargo type probability distribution is selected; the color category label and the cargo type label are combined to form the classification attribute of the cargo box image. The attribute comparison module is used to detect the entry and exit status of the scrap steel transport vehicle based on the license plate-timestamp-location binding relationship. When the same scrap steel transport vehicle has been detected to have completed both entry and exit, the classification attribute of the first cargo box image when the scrap steel transport vehicle enters the factory is compared with the classification attribute of the second cargo box image when it leaves the factory to obtain the comparison result. An alarm module is used to trigger a cargo box replacement alarm based on the comparison results.

8. A monitoring device for scrap steel transport vehicles, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the scrap steel transport vehicle monitoring method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the scrap steel transport vehicle supervision method as described in any one of claims 1 to 6.

Images