Container sealing supervision device

Abstract

The invention discloses a container sealing supervision device which is characterized in that a video of an environment behind a vehicle is shot in real time through an intelligent lock with a camera and uploaded to a server side, image processing and similarity detection are carried out in combination with a deep learning algorithm of the server side, and whether the intelligent lock is mounted at a correct position of the tail of the vehicle or not can be accurately judged; the server side adopts a bilateral filtering algorithm to carry out denoising processing on the image and carries out similarity detection on the image through a twin neural network, so that misjudgment caused by environmental noise or background change can be effectively avoided, and the accuracy of a container opening process is improved. When it is detected that the mounting position of the intelligent lock is abnormal or the similarity between the shot image and the reference image is too low, the server side can send abnormal prompt information to the customer service side in time, so that a worker can quickly respond and deal with the abnormal situation, and real-time monitoring and effective management of the container transportation process are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of container sealing, and in particular to a container sealing monitoring device. Background Art

[0002] With the rapid development of the logistics industry, higher requirements are placed on the safety and accuracy of container sealing and supervision. The existing container supervision methods mainly rely on technical means such as RFID tags and electronic seals, but these technologies have some limitations in practical applications. For example, RFID tags are easy to be copied or replaced, and although electronic seals can record the time of opening and closing, they cannot monitor the status and environment of the container in real time. In order to solve the above problems, some video-based container supervision systems have appeared on the market. These systems usually install cameras on the containers and transmit video data to the monitoring center through wireless networks. However, most of these systems rely on manual monitoring and cannot automatically determine the correctness of container sealing and unsealing, and are easily affected by network delays and image quality. In addition, although some advanced video surveillance systems have image recognition functions, they often require high hardware costs and complex algorithm support, making it difficult to achieve large-scale deployment and application. Summary of the invention

[0003] In view of this, the present invention proposes a container sealing monitoring device, which can solve the defects of low monitoring efficiency, low precision and high cost in the prior art.

[0004] The technical solution of the present invention is achieved in this way:

[0005] A container sealing monitoring device, specifically comprising:

[0006] The client is used to receive and process abnormal prompt information;

[0007] On the server side, a deep learning algorithm is built in to receive and process video stream data from smart locks, perform image processing and similarity detection;

[0008] The smart lock with a camera is fixed on the rear of the vehicle to capture video of the environment behind the vehicle and upload it to the server in real time.

[0009] As a further optional solution of the container sealing monitoring device, the server receives and processes the video stream data from the smart lock, performs image processing and similarity detection, specifically including:

[0010] Perform bilateral filtering denoising on the received image frames to filter out noise and retain image edge information;

[0011] The denoised image and the standard reference image are sent to the twin neural network for similarity detection. When the similarity is lower than the preset threshold, further detection is triggered.

[0012] If the similarity detection result is lower than the preset threshold, target detection is performed on the fixed objects in the image. If the preset fixed objects are not detected, an abnormal prompt is sent to the customer service end.

[0013] As a further optional solution of the container sealing monitoring device, the bilateral filtering denoising process is performed on the received image frame to filter out noise and retain image edge information, specifically including:

[0014] Get the neighborhood range of the filter;

[0015] For each pixel in the neighborhood, calculate the spatial distance and pixel value difference between it and the central pixel;

[0016] Calculate the corresponding first weight coefficient using the formula of the spatial domain kernel and the spatial distance;

[0017] Calculate the corresponding second weight coefficient using the formula of the range kernel and the pixel value difference;

[0018] Calculate a final weight coefficient based on the first weight coefficient and the second weight coefficient;

[0019] Using the final weight coefficient, the pixel values ​​in the neighborhood are weighted averaged to obtain the filtered pixel value;

[0020] Repeat the above steps, traverse every pixel in the entire image, perform bilateral filtering on each pixel, and finally obtain an image with noise filtered out and edge information retained.

[0021] As a further optional solution of the container sealing monitoring device, the twin neural network includes:

[0022] A dual-input network with shared weights to process two input images separately;

[0023] A feature extraction layer, used to extract features of an input image, wherein the feature extraction layer adopts a VGG16 convolutional neural network;

[0024] The feature comparison layer is used to calculate the difference between the two input images by subtracting the features of the two input images and taking the absolute value;

[0025] The fully connected layer is used to flatten the previous feature map into a one-dimensional vector and perform a weighted sum operation;

[0026] The Sigma ID activation function is used to calculate the similarity probability value between two input images.

[0027] As a further optional solution of the container sealing monitoring device, the denoised image and the standard reference image are sent to the twin neural network for similarity detection, specifically including:

[0028] The denoised image and the standard reference image are divided into multiple blocks and sent to the twin neural network for similarity detection.

[0029] The block with the lowest similarity value is removed, and the similarity values ​​of the remaining blocks are added and averaged to obtain the final similarity value.

[0030] As a further optional solution of the container sealing monitoring device, the YOLOv5s network structure is used to detect fixed objects in the image, and the YOLOv5s network structure includes:

[0031] Convolutional layers, used to extract features from the input image;

[0032] Batch normalization layer, used to normalize the output of the convolutional layer;

[0033] Non-linear activation function, used to increase the non-linear expression ability of the model;

[0034] BackBone feature extraction layer, used to extract effective feature representation from the input image to form a feature map;

[0035] Neck feature fusion layer, used to fuse feature maps of different scales to form a fused feature map;

[0036] The output layer is used to output the final detection result based on the fused feature map;

[0037] The loss function is used to evaluate the difference between the model's prediction results and the true label, and optimize the model's parameters through the back propagation algorithm, wherein the loss function includes the classification loss function, the confidence loss function and the prediction box loss function.

[0038] As a further optional solution of the container sealing supervision device, the Resun it residual network in the BackBone feature extraction layer is replaced by a MobileNetv3block network, and the weight ratio of the classification loss function, the confidence loss function and the position loss function in the loss function is adjusted to 3:4:3.

[0039] A container sealing supervision method, the method using any one of the container sealing supervision devices described above, specifically comprising:

[0040] Step S1, the client receives and processes abnormal prompt information from the server, wherein the abnormal prompt information is used to indicate possible abnormalities in the mounting position of the smart lock during the sealing or unsealing process of the container;

[0041] Step S2: The server has a built-in deep learning algorithm and performs the following steps:

[0042] Step S21, receiving video stream data of the vehicle rear environment uploaded in real time from a smart lock with a camera;

[0043] Step S22, performing image processing on the received video stream data;

[0044] Step S23, performing similarity detection between the processed image and the preset reference image, and using the twin neural network to determine whether the smart lock is mounted in the correct position at the rear of the vehicle;

[0045] Step S24: If the similarity detection result is lower than the preset threshold, the YOLOv5s target detection network is further used to detect the fixed objects in the image to confirm the correctness of the mounting position of the smart lock;

[0046] Step S25, if the smart lock mounting position is detected to be abnormal, an abnormal prompt message is sent to the customer service end;

[0047] Step S3: The smart lock with camera is fixedly mounted on the rear of the vehicle, and the following steps are performed:

[0048] Step S31, shooting a video of the environment behind the vehicle;

[0049] Step S32, uploading the captured video to the server in real time for subsequent processing by the server.

[0050] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the container sealing supervision method when executing the computer program.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the container sealing supervision method described above.

[0052] The beneficial effects of the present invention are as follows: by using a smart lock with a camera to capture a video of the environment behind the vehicle in real time and uploading it to a server, and combining the deep learning algorithm on the server for image processing and similarity detection, it is possible to accurately determine whether the smart lock is mounted in the correct position at the rear of the vehicle, thereby effectively preventing illegal opening or tampering with the container, and improving the safety of container sealing and unsealing operations. The server uses a bilateral filtering algorithm to denoise the image, and uses a twin neural network to perform similarity detection on the image, which can effectively avoid misjudgment caused by environmental noise or background changes. When it is detected that the mounting position of the smart lock is abnormal or the similarity between the captured image and the reference image is too low, the server will promptly send an abnormal prompt message to the client, so that the staff can quickly respond and handle the abnormal situation, realizing real-time monitoring and effective management of the container transportation process. The client can receive and process the abnormal prompt message from the server, providing a convenient information acquisition channel for the staff. At the same time, the staff can also obtain the on-site video from the server through the client to send instructions for viewing, further enhancing the user experience and supervision efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A schematic diagram of the composition of a container sealing monitoring device provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the processing steps of a deep learning algorithm in a container sealing monitoring device provided by an embodiment of the present invention

[0056] Figure 3 A schematic flow chart of the processing steps of a twin neural network in a container sealing monitoring device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] refer to Figures 1 to 3 , a container sealing monitoring device, specifically comprising:

[0059] The client is used to receive and process abnormal prompt information;

[0060] On the server side, a deep learning algorithm is built in to receive and process video stream data from smart locks, perform image processing and similarity detection;

[0061] The smart lock with a camera is fixed on the rear of the vehicle to capture video of the environment behind the vehicle and upload it to the server in real time.

[0062] In this embodiment, a smart lock with a camera takes a real-time video of the environment behind the vehicle and uploads it to the server. The image processing and similarity detection are performed in combination with the deep learning algorithm on the server. It can accurately determine whether the smart lock is mounted in the correct position at the rear of the vehicle, thereby effectively preventing illegal opening or tampering with the container and improving the safety of the container sealing and unsealing operations. The server uses a bilateral filtering algorithm to denoise the image and performs similarity detection on the image through a twin neural network, which can effectively avoid misjudgment caused by environmental noise or background changes. When it is detected that the mounting position of the smart lock is abnormal or the similarity between the captured image and the reference image is too low, the server will promptly send an abnormal prompt message to the client, so that the staff can respond quickly and handle the abnormal situation, realizing real-time monitoring and effective management of the container transportation process. The client can receive and process the abnormal prompt message from the server, providing a convenient information acquisition channel for the staff. At the same time, the staff can also send instructions from the client to obtain the on-site video from the server for viewing, further enhancing the user experience and supervision efficiency.

[0063] Preferably, the server receives and processes the video stream data from the smart lock, performs image processing and similarity detection, specifically including:

[0064] Perform bilateral filtering denoising on the received image frames to filter out noise and retain image edge information;

[0065] The denoised image and the standard reference image are sent to the twin neural network for similarity detection. When the similarity is lower than the preset threshold, further detection is triggered.

[0066] If the similarity detection result is lower than the preset threshold, target detection is performed on the fixed objects in the image. If the preset fixed objects are not detected, an abnormal prompt is sent to the customer service end.

[0067] In this embodiment, the noise in the image is effectively filtered out through bilateral filtering denoising processing, while the edge information of the image is retained. This step is crucial to the accuracy of subsequent similarity detection because it reduces the misjudgment caused by noise interference; the denoised image and the standard reference image are sent to the twin neural network for similarity detection, which can accurately determine whether the smart lock is mounted in the correct position at the rear of the vehicle. When the similarity is lower than the preset threshold, it indicates that there may be an abnormality, thereby triggering further detection procedures; when the similarity detection result is lower than the preset threshold, target detection is performed on fixed objects in the image. This step makes up for the possible missed detection problem that only relies on similarity detection. The position of the smart lock is further confirmed by detecting whether the image contains preset fixed objects. If no fixed object is detected, an abnormal prompt is sent to the customer service end, which improves the reliability of abnormal detection. By combining image processing, similarity detection and target detection, a complete detection process is formed. This process not only improves the accuracy of detection, but also effectively reduces the false alarm rate caused by environmental factors (such as flowing objects behind the rear of the vehicle). The entire supervision process is automated and real-time. From the smart lock shooting video to the server processing data, and then to the customer service end receiving abnormal prompts, the entire process is efficient and fast, which greatly improves the efficiency and accuracy of container sealing supervision and reduces the cost and difficulty of manual supervision.

[0068] Preferably, the bilateral filtering denoising process is performed on the received image frame to filter out noise and retain image edge information, specifically including:

[0069] Get the neighborhood range of the filter;

[0070] For each pixel in the neighborhood, calculate the spatial distance and pixel value difference between it and the central pixel;

[0071] Calculate the corresponding first weight coefficient using the formula of the spatial domain kernel and the spatial distance;

[0072] Calculate the corresponding second weight coefficient using the formula of the range kernel and the pixel value difference;

[0073] Calculate a final weight coefficient based on the first weight coefficient and the second weight coefficient;

[0074] Using the final weight coefficient, the pixel values ​​in the neighborhood are weighted averaged to obtain the filtered pixel value;

[0075] Repeat the above steps, traverse every pixel in the entire image, perform bilateral filtering on each pixel, and finally obtain an image with noise filtered out and edge information retained.

[0076] In this embodiment, the bilateral filtering algorithm calculates the weight coefficient by combining the spatial distance and the pixel value difference, which can effectively retain the edge information of the image while removing the image noise. This feature can reduce the influence of noise on the detection results and improve the accuracy of detection when performing similarity detection and target detection in the future; the core of the bilateral filtering algorithm is that it can intelligently adjust the weight coefficient so that at the edge of the image, the points with large pixel value differences are given smaller weights, thereby avoiding the blurring of the edge, which is particularly important for subsequent detection tasks, because edge information is often an important basis for judging the position and shape of objects; the image processed by bilateral filtering has significantly improved quality, reduced noise, and clear edges, which enables the subsequent twin neural network similarity detection and YOLOv5s target detection to more accurately identify key information in the image, such as the position of the smart lock and the preset fixed object, thereby improving the detection accuracy of the entire system; although the bilateral filtering algorithm is relatively complex, it avoids unnecessary calculations by intelligently adjusting the weight coefficient, so that the overall calculation efficiency can be optimized. At the same time, since the noise is removed and the edge information is retained, the misjudgment and missed judgment in the subsequent detection process are reduced, and the overall performance of the system is further improved.

[0077] It should be noted that the spatial domain kernel formula is:

[0078]

[0079] d(i,j,k,l) ​​represents the Euclidean distance between a point (k,l) and the center point (i,j) in the domain, i represents the row coordinate of the pixel (or y coordinate, depending on the definition of the coordinate system), j represents the column coordinate of the pixel (or x coordinate), k and l represent the position coordinates of a pixel in the domain of the currently processed pixel (i,j), which is usually a small window or neighborhood centered at (i,j);

[0080] Range kernel formula:

[0081]

[0082] r(i,j,k,l) ​​represents the absolute value of the difference between the grayscale value f(i,j) of a point (k,l) in the domain and the grayscale value f(i,j) of the center point (i,j);

[0083] According to the first weight coefficient and the second weight coefficient, the final weight coefficient is calculated, which is specifically:

[0084] ω(i,j,k,l)=d(i,j,k,l)r(i,j,k,l);

[0085] ω(i, j, k, l) is the final weight coefficient, d(i, j, k, l) is the first weight coefficient, and r(i, j, k, l) is the second weight coefficient.

[0086] In addition, calculate the spatial distance weight: for each pixel point (k, l) in the neighborhood, calculate the spatial distance d(i, j, k, l) between it and the central pixel point (i, j). This distance is measured using the Euclidean distance. Then, based on the spatial distance d, use the formula of the spatial domain kernel to calculate the corresponding weight coefficient. This weight coefficient decreases as the spatial distance increases, so that pixels that are closer have a greater impact on the central pixel point. Calculate the pixel value difference weight: At the same time, it is also necessary to calculate the pixel value difference r(i, j, k, l) between the neighborhood pixel point (k, l) and the central pixel point (i, j). This difference is measured using the absolute value of the difference in the grayscale values ​​of the two points. Then, based on the pixel value difference r, use The formula of the range kernel calculates the corresponding weight coefficient, which will decrease as the difference in pixel values ​​increases, so that pixels with similar pixel values ​​have a greater impact on the central pixel; multiply the above two weight coefficients to obtain the final weight coefficient ω(i, j, k, l), which takes into account both spatial distance and pixel value difference, and can better balance the needs of denoising and retaining edge information; use the above weight coefficients to perform weighted averaging on the pixel values ​​in the neighborhood to obtain the filtered pixel value. Specifically, the grayscale value of each pixel in the neighborhood is multiplied by its corresponding weight coefficient, and then the sum is divided by the sum of all weight coefficients to obtain the filtered grayscale value of the central pixel (i, j).

[0087] Preferably, the twin neural network comprises:

[0088] A dual-input network with shared weights to process two input images separately;

[0089] A feature extraction layer, used to extract features of an input image, wherein the feature extraction layer adopts a VGG16 convolutional neural network;

[0090] The feature comparison layer is used to calculate the difference between the two input images by subtracting the features of the two input images and taking the absolute value;

[0091] The fully connected layer is used to flatten the previous feature map into a one-dimensional vector and perform a weighted sum operation;

[0092] The Sigmoid activation function is used to calculate the similarity probability value between two input images.

[0093] In this embodiment, by using the VGG16 convolutional neural network as the feature extraction layer, key features can be efficiently extracted from the input image; the twin neural network adopts a dual-path input network structure with shared weights. This design enables the two input images to be processed under the same network conditions, thereby ensuring the fairness and consistency of the comparison. The feature comparison layer calculates the difference between the two input images by subtracting and then taking the absolute value. This method is direct and effective and can reflect the subtle differences between the images; the fully connected layer flattens the feature map into a one-dimensional vector and performs a weighted sum operation, which provides comprehensive and effective information for subsequent similarity calculations. Finally, S The igmoid activation function compresses the output result to between 0 and 1, and represents the similarity between the two input images in the form of a probability value. This method is intuitive and easy to understand, and provides a strong basis for judging whether the mounting position of the smart lock is correct. By adjusting the VGG16 network, such as increasing the input scale, adding convolutional layers and pooling layers, and adjusting the output layer dimension, the present invention retains the advantages of VGG16 while also optimizing specific applications to improve the accuracy of similarity detection. Since smart locks may face various complex environments in actual applications, such as lighting changes, object occlusion, etc., the design of the present invention has high robustness and adaptability. Through the precise detection of the twin neural network and the auxiliary verification of the YOLOv5s target detection network, these unfavorable factors can be overcome to a certain extent to ensure the accurate judgment of the mounting position of the smart lock.

[0094] It should be noted that the present invention uses a VGG16 convolutional neural network in the backbone feature extraction network to extract features from the image. The two input networks share weights, and then the two outputs g(x1) and g(x2) are subtracted and the absolute value is taken. The output f(x3) after the fully connected convolution layer is then activated by the Sigma ID activation function. The Sigma ID activation function formula is:

[0095]

[0096] The sigmoid activation function compresses the output value to between 0 and 1, which is used to represent the similarity probability value between two pictures. The similarity of the two pictures can be intuitively understood through the output value. The present invention also modifies the VGG16 network. The image input scale of the original network is 224. When the picture is resolved to this scale, more information of the original image will be lost. The present invention adjusts the input scale to 448, and adds a pooling layer and a convolution layer in the middle to ensure that the scale of the final output is consistent. At the same time, the final output is also adjusted from 1x1x100 to 1x1x1024, which can provide more information for the subsequent subtraction of the two information, and can improve the detection accuracy to a certain extent.

[0097] Preferably, the step of sending the denoised image and the standard reference image to a twin neural network for similarity detection specifically includes:

[0098] The denoised image and the standard reference image are divided into multiple blocks and sent to the twin neural network for similarity detection.

[0099] The block with the lowest similarity value is removed, and the similarity values ​​of the remaining blocks are added and averaged to obtain the final similarity value.

[0100] In this embodiment, through image segmentation and block detection, each part of the image can be analyzed more carefully, the overall similarity misjudgment caused by local occlusion or change can be reduced, the blocks with the lowest similarity value are removed and the average value of the remaining blocks is taken, thereby further improving the accuracy and robustness of the similarity detection; this method can adapt to image changes in different scenarios, such as changes in lighting, weather changes, etc., because block detection can capture local features in the image, not just overall features; by reducing false alarms caused by misjudgment, the overall performance and user experience of the system are improved. When the similarity detection is low, the YOLOv5s target detection network is further introduced for fixed object detection, which can more comprehensively determine whether the mounting position of the smart lock is correct, further improving the supervision accuracy and security of the system.

[0101] Preferably, the YOLOv5s network structure is used to detect fixed objects in the image, and the YOLOv5s network structure includes:

[0102] Convolutional layers, used to extract features from the input image;

[0103] Batch normalization layer, used to normalize the output of the convolutional layer;

[0104] Non-linear activation function, used to increase the non-linear expression ability of the model;

[0105] BackBone feature extraction layer, used to extract effective feature representation from the input image to form a feature map;

[0106] Neck feature fusion layer, used to fuse feature maps of different scales to form a fused feature map;

[0107] The output layer is used to output the final detection result based on the fused feature map;

[0108] The loss function is used to evaluate the difference between the model's prediction results and the true label, and optimize the model's parameters through the back propagation algorithm, wherein the loss function includes the classification loss function, the confidence loss function and the prediction box loss function.

[0109] In this embodiment, the YOLOv5s network structure can effectively extract rich feature representations from the input image by combining convolutional layers, batch normalization layers and nonlinear activation functions. These feature representations play a key role in the subsequent detection process and help to achieve high-precision detection of fixed objects in the image; the BackBone feature extraction layer and the Neck feature fusion layer in YOLOv5s enable the model to capture different levels of detail information in the image by fusing feature maps of different scales, thereby improving the detection capability of fixed objects in the image, especially for the detection of small objects or occluded objects; the design of the loss function includes classification, classification and classification. Class loss, confidence loss and prediction box loss, these three parts are combined with different weights, so that the model can pay more attention to the classification accuracy, confidence and positioning accuracy of the target bounding box during training. In particular, by adjusting the weights of classification loss and confidence loss, the model is more adaptable in specific application scenarios and improves the detection effect. When the twin neural network detects a low similarity value, the YOLOv5s target detection model is started for further verification. This mechanism can effectively make up for the possible missed detection problem of the twin neural network, ensure the correctness of the smart lock mounting position through double detection, and enhance the robustness and reliability of the entire system.

[0110] It should be noted that the loss function of the YOLOv5s network consists of three parts: classification loss, confidence loss, and prediction box loss. The three parts constitute the total loss function through different weights. The loss function formula is as follows:

[0111] L all =λ 1 L cls +λ 2 L obj +λ 3 L loc

[0112] Among them, L cls , L obj , L loc They are classification, confidence, and position losses, respectively. λ represents the corresponding weight. Both the classification loss function and the confidence loss function use the binary cross entropy loss function, and the formula is as follows:

[0113]

[0114] Among them, y i is the predicted value, is the true value, N is the total number of categories;

[0115] The bounding box prediction loss function uses the E IOU loss function, and its formula is as follows:

[0116]

[0117] Where IOU is the ratio of the overlapping area of ​​the predicted box and the real box to the area of ​​the minimum circumscribed rectangle of the predicted box and the real box; ρ is the Euclidean distance function; c is the length of the diagonal of the minimum circumscribed rectangle of the predicted box and the real box; b represents the predicted box, b gt represents the real box; w and h represent the width and height of the predicted box, w gt and h gt Indicates the width and height of the real box; c w and c h are the width and height of the minimum bounding rectangle of the predicted box and the true box, respectively.

[0118] Preferably, the Resun it residual network in the BackBone feature extraction layer is replaced by a MobileNetv3block network, and the weight ratio of the classification loss function, the confidence loss function and the position loss function in the loss function is adjusted to 3:4:3.

[0119] In this embodiment, the MobileNetv3block network has lower parameters and computational complexity than the Resun it residual network. This replacement can significantly reduce the complexity and computational requirements of the model, thereby reducing the operating cost of the server and speeding up the inference speed, making the model more suitable for real-time application scenarios; by adjusting the weight ratio of each item in the loss function (λ1:λ2:λ3=3:4:3), the weights of the classification loss and the confidence loss are increased, so that the model pays more attention to the existence of objects and the accuracy of categories during the training process. Since the detection background in the application scenario of the present invention is relatively fixed and the positioning accuracy of the target is not required to be high, such an adjustment can improve the accuracy of the model in identifying fixed objects; the focus of the target detection task is placed on classification and confidence, rather than over-reliance on position accuracy, so that the model can more stably identify fixed objects in the image in practical applications, and can maintain a high detection accuracy even when the environment changes slightly or is blocked.

[0120] It should be noted that the MobileNetv3block network includes deep separable convolution, linear bottleneck layer, and residual connection:

[0121] Depthwise separable convolution is one of the core optimization techniques of the MobileNet series. It splits the standard convolution operation into depthwise convolution and pointwise convolution (also known as 1x1 convolution). Depthwise convolution applies the convolution kernel to each input channel independently, while pointwise convolution is used to combine the output of depthwise convolution. This method significantly reduces the number of parameters and computation of the model, because depthwise convolution only operates on a single channel, while pointwise convolution combines the output of depthwise convolution instead of performing a complete convolution operation directly on all channels of the input.

[0122] The design of the linear bottleneck layer is based on the observation that the activation values ​​in the high-dimensional space are prone to lose information when mapped to the low-dimensional space, but after linear transformation in the low-dimensional space, when mapped back to the high-dimensional space, useful information can be retained. The use of the linear bottleneck layer can reduce the loss of information in the process of dimensionality reduction and dimensionality increase, while maintaining the nonlinear characteristics of the model (through nonlinear activation functions in other layers), thereby reducing the complexity and computational complexity of the model without sacrificing performance;

[0123] Residual connection is the core idea in the residual network (ResNet). It helps the model learn the identity mapping (i.e., the mapping where the input is equal to the output) more easily by adding the input of a layer directly to the output of the layer. Residual connection helps solve the gradient vanishing or gradient exploding problems that may occur in the training process of deep neural networks, allowing the network to be trained deeper without performance degradation. In addition, it can also improve the generalization ability and convergence speed of the model. In MobileNetV3, residual connection is combined with other optimization techniques to jointly improve the performance and efficiency of the model.

[0124] A container sealing supervision method, the method using any one of the container sealing supervision devices described above, specifically comprising:

[0125] Step S1, the client receives and processes abnormal prompt information from the server, wherein the abnormal prompt information is used to indicate possible abnormalities in the mounting position of the smart lock during the sealing or unsealing process of the container;

[0126] Step S2: The server has a built-in deep learning algorithm and performs the following steps:

[0127] Step S21, receiving video stream data of the vehicle rear environment uploaded in real time from a smart lock with a camera;

[0128] Step S22, performing image processing on the received video stream data;

[0129] Step S23, performing similarity detection between the processed image and the preset reference image, and using the twin neural network to determine whether the smart lock is mounted in the correct position at the rear of the vehicle;

[0130] Step S24: If the similarity detection result is lower than the preset threshold, the YOLOv5s target detection network is further used to detect the fixed objects in the image to confirm the correctness of the mounting position of the smart lock;

[0131] Step S25, if the smart lock mounting position is detected to be abnormal, an abnormal prompt message is sent to the customer service end;

[0132] Step S3: The smart lock with camera is fixedly mounted on the rear of the vehicle, and the following steps are performed:

[0133] Step S31, shooting a video of the environment behind the vehicle;

[0134] Step S32, uploading the captured video to the server in real time for subsequent processing by the server.

[0135] A computing device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the container sealing supervision method when executing the computer program.

[0136] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the container sealing supervision method described above.

[0137] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0138] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A container sealing monitoring device, characterized in that: Specifically include: The client is used to receive and process abnormal prompt information; On the server side, a deep learning algorithm is built in to receive and process video stream data from smart locks, perform image processing and similarity detection; The smart lock with a camera is fixed on the rear of the vehicle to capture video of the environment behind the vehicle and upload it to the server in real time.

2. A container sealing monitoring device according to claim 1, characterized in that: The server receives and processes the video stream data from the smart lock, performs image processing and similarity detection, specifically including: Perform bilateral filtering denoising on the received image frames to filter out noise and retain image edge information; The denoised image and the standard reference image are sent to the twin neural network for similarity detection. When the similarity is lower than the preset threshold, further detection is triggered. If the similarity detection result is lower than the preset threshold, target detection is performed on the fixed objects in the image. If the preset fixed objects are not detected, an abnormal prompt is sent to the customer service end.

3. A container sealing monitoring device according to claim 2, characterized in that: The step of performing bilateral filtering denoising on the received image frame to filter out noise and retain image edge information specifically includes: Get the neighborhood range of the filter; For each pixel in the neighborhood, calculate the spatial distance and pixel value difference between it and the central pixel; Calculate the corresponding first weight coefficient using the formula of the spatial domain kernel and the spatial distance; Calculate the corresponding second weight coefficient using the formula of the range kernel and the pixel value difference; Calculate a final weight coefficient based on the first weight coefficient and the second weight coefficient; Using the final weight coefficient, the pixel values ​​in the neighborhood are weighted averaged to obtain the filtered pixel value; Repeat the above steps, traverse every pixel in the entire image, perform bilateral filtering on each pixel, and finally obtain an image with noise filtered out and edge information retained.

4. A container sealing monitoring device according to claim 3, characterized in that: The twin neural network comprises: A dual-input network with shared weights to process two input images separately; A feature extraction layer, used to extract features of an input image, wherein the feature extraction layer adopts a VGG16 convolutional neural network; The feature comparison layer is used to calculate the difference between the two input images by subtracting the features of the two input images and taking the absolute value; The fully connected layer is used to flatten the previous feature map into a one-dimensional vector and perform a weighted sum operation; Sigmoid activation function is used to calculate the similarity probability value between two input images.

5. A container sealing monitoring device according to claim 4, characterized in that: The denoised image and the standard reference image are sent to the twin neural network for similarity detection, specifically including: The denoised image and the standard reference image are divided into multiple blocks and sent to the twin neural network for similarity detection. The block with the lowest similarity value is removed, and the similarity values ​​of the remaining blocks are added and averaged to obtain the final similarity value.

6. A container sealing monitoring device according to claim 5, characterized in that: The YOLOv5s network structure is used to detect fixed objects in the image, and the YOLOv5s network structure includes: Convolutional layers, used to extract features from the input image; Batch normalization layer, used to normalize the output of the convolutional layer; Non-linear activation function, used to increase the non-linear expression ability of the model; BackBone feature extraction layer, used to extract effective feature representation from the input image to form a feature map; Neck feature fusion layer, used to fuse feature maps of different scales to form a fused feature map; The output layer is used to output the final detection result based on the fused feature map; The loss function is used to evaluate the difference between the model's prediction results and the true label, and optimize the model's parameters through the back propagation algorithm, wherein the loss function includes the classification loss function, the confidence loss function and the prediction box loss function.

7. A container sealing monitoring device according to claim 6, characterized in that: The Resunit residual network in the BackBone feature extraction layer is replaced with a MobileNetv3 block network, and the weight ratio of the classification loss function, the confidence loss function, and the position loss function in the loss function is adjusted to 3:4:

3.

8. A container sealing supervision method, characterized in that: The method uses the container sealing monitoring device described in any one of claims 1 to 7, and specifically comprises: Step S1, the client receives and processes abnormal prompt information from the server, wherein the abnormal prompt information is used to indicate possible abnormalities in the mounting position of the smart lock during the sealing or unsealing process of the container; Step S2: The server has a built-in deep learning algorithm and performs the following steps: Step S21, receiving video stream data of the vehicle rear environment uploaded in real time from a smart lock with a camera; Step S22, performing image processing on the received video stream data; Step S23, performing similarity detection between the processed image and the preset reference image, and using the twin neural network to determine whether the smart lock is mounted in the correct position at the rear of the vehicle; Step S24: If the similarity detection result is lower than the preset threshold, the YOLOv5s target detection network is further used to detect the fixed objects in the image to confirm the correctness of the mounting position of the smart lock; Step S25, if the smart lock mounting position is detected to be abnormal, an abnormal prompt message is sent to the customer service end; Step S3: The smart lock with camera is fixedly mounted on the rear of the vehicle, and the following steps are performed: Step S31, shooting a video of the environment behind the vehicle; Step S32, uploading the captured video to the server in real time for subsequent processing by the server.

9. A computing device, characterized in that The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the container sealing supervision method as claimed in claim 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the container sealing supervision method described in claim 8.

Images