Logistics Violation Behavior Identification Method, Device, Equipment and Storage Medium

Abstract

The present invention discloses a method, device, equipment and storage medium for identifying logistics violation behaviors. The method includes: S1: Collecting and obtaining a number of pictures of logistics violation behaviors; S2: Performing data annotation on the logistics violation behaviors appearing in the number of pictures, and converting and generating a picture data set in TFRecord format; S3: Based on the TensorFlow object detection API, training the YOLOv3 model that has been pre-converted into a TensorFlow frozen model through the picture data set, and verifying the trained YOLOv3 model to obtain a violation identification model; S4: Identifying the logistics scenario through the violation identification model to determine whether there are logistics violation behaviors, and obtaining an identification result. By combining YOLOv3 with the TensorFlow object detection API and applying it to the logistics field for violation behaviors, the present invention not only greatly improves the training speed of the model, the accuracy and speed during the training and supervision processes, but also guarantees the interests of users, improves the service quality of users, reduces the losses of the logistics industry, improves the transfer efficiency and quality, and makes the logistics closer to the path of intelligence.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent logistics, and in particular relates to a method, device, equipment and storage medium for identifying logistics violations. Background Art

[0002] Object detection is a hot topic in computer vision and digital image processing. It is widely used in many fields such as robot navigation, intelligent video surveillance, industrial inspection, aerospace, etc. It has important practical significance to reduce the consumption of human capital through computer vision. Therefore, object detection has become a hot topic in theoretical and applied research in recent years. It is an important branch of image processing and computer vision, and a core part of intelligent monitoring systems. At the same time, object detection is also a basic algorithm in the field of pan-identity recognition, which plays a vital role in subsequent tasks such as face recognition, gait recognition, crowd counting, and instance segmentation. Due to the widespread use of deep learning, object detection algorithms have developed rapidly.

[0003] In the field of logistics, target detection also plays a role that cannot be underestimated. Logistics has a huge data flow, and a large amount of data flows every moment, circulating in logistics networks. And the express deliveries that go through various outlets are countless. However, in actual operations, there are always some unsatisfactory situations, such as the occurrence of logistics violations such as package damage caused by operators trampling and throwing. Although target detection can play a certain regulatory role in the actual process, due to the uncontrollability of logistics personnel, once new logistics violations are exposed, they often need to go through the process of collecting resources, building models, model training, etc. This process has problems such as long preparation time, slow model training, and poor supervision timeliness, which is very unfavorable to the improvement of logistics supervision and the improvement of logistics quality. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method, device, equipment and storage medium for identifying logistics violations, which has the technical characteristics of high logistics violation identification training efficiency and intelligent logistics.

[0005] The technical solution of the present invention is:

[0006] A method for identifying logistics violations comprises the following steps:

[0007] S1: Collect and obtain several pictures of logistics violations;

[0008] S2: Label the logistics violations in several pictures and convert them into TFRecord format image datasets;

[0009] S3: Based on the TensorFlow Object Detection API, train the YOLOv3 model that has been pre-converted into a TensorFlow frozen model using an image dataset, and verify the trained YOLOv3 model to obtain a violation recognition model;

[0010] S4: Use the violation recognition model to identify the logistics scenario, determine whether there are any logistics violations, and obtain the recognition result.

[0011] According to an embodiment of the present invention, step S2 further includes:

[0012] Perform data annotation on the logistics violations that appear in a number of pictures through rectangular box annotation.

[0013] According to an embodiment of the present invention, the logistics violations at least include package trampling. In step S2, the content of the rectangular box annotation includes a human foot and the trampled package.

[0014] According to an embodiment of the present invention, in step S2, further including in the process of converting and generating the picture dataset in TFRecord format:

[0015] Convert a number of pictures after data annotation into Example objects;

[0016] Serialize the Example objects into strings and write them into a TFRecord file through a preset format conversion script to obtain a picture dataset in TFRecord format. Among them, the information written into the TFRecord file at least includes picture itself information, data annotation information, and annotation name information.

[0017] According to an embodiment of the present invention, step S4 further includes:

[0018] Monitor and obtain on-site pictures or on-site videos of the logistics scenario;

[0019] Use the violation recognition model to identify each static image of the on-site pictures or on-site videos to determine whether there are any logistics violations and obtain the recognition result.

[0020] According to an embodiment of the present invention, before step S1 is executed, it further includes step S0:

[0021] Configure the TensorFlow Object Detection API and convert the pre-trained YOLOv3 model into a TensorFlow frozen model, where the TensorFlow frozen model is in PB format.

[0022] A device for identifying logistics violation behaviors includes:

[0023] A data acquisition module, configured to acquire several pictures of logistics violation behaviors;

[0024] A data annotation conversion module, configured to perform data annotation on the logistics violation behaviors appearing in several pictures, and convert and generate a picture data set in the TFRecord format;

[0025] A model training and verification module, configured to train a YOLOv3 model pre-converted into a TensorFlow frozen model based on the TensorFlow Object Detection API through the picture data set, and verify the trained YOLOv3 model to obtain a violation recognition model;

[0026] A behavior recognition module, configured to recognize a logistics scenario through the violation recognition model, determine whether there is a logistics violation behavior, and obtain a recognition result.

[0027] According to an embodiment of the present invention, the data annotation conversion module is specifically configured to perform data annotation on the logistics violation behaviors appearing in several pictures through rectangular box annotation, convert the several pictures after data annotation into Example objects, serialize the Example objects into strings, and write them into a TFRecord file through a preset format conversion script to obtain a picture data set in the TFRecord format.

[0028] According to an embodiment of the present invention, the behavior recognition module is specifically configured to monitor and acquire on-site pictures or on-site videos of a logistics scenario, recognize each static image of the on-site pictures or on-site videos through the violation recognition model, determine whether there is a logistics violation behavior, and obtain a recognition result.

[0029] A computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes the logistics violation behavior recognition method as described above.

[0030] A storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the logistics violation behavior recognition method as described above.

[0031] The present invention has the following advantages and positive effects compared with the prior art:

[0032] The present invention combines YOLOv3 with the TensorFlow object detection API and applies it to the logistics field for detecting violations. Among them, the YOLOv3 model is converted into a PB solidified model of TensorFlow, image data for model training is collected and converted into the TFRecord format. Based on the configured API, training and verification are performed using the image data set in the TFRecord format. In this way, the training speed of the YOLOv3 model for identifying logistics violations is greatly improved. At the same time, based on the YOLOv3 model, the accuracy and speed in the training and supervision processes are improved. In addition, if it is necessary to supervise new logistics violations, through the above technical solution, only by re-collecting images of logistics violations and modifying the configuration file of the TensorFlow object detection API, the corresponding violation identification model can be obtained very conveniently, so as to realize the intelligent supervision of logistics violations, standardize the behaviors of logistics personnel, ensure the interests of users, provide higher service quality, reduce the losses of the logistics industry, and improve the transfer efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0034] Figure 1 It is the overall flowchart of a method for identifying logistics violations of the present invention;

[0035] Figure 2 It is the flowchart of step S2 of a method for identifying logistics violations of the present invention;

[0036] Figure 3 It is the schematic diagram of the network structure of the YOLOv3 model of a method for identifying logistics violations of the present invention;

[0037] Figure 4 It is the structural block diagram of a device for identifying logistics violations of the present invention;

[0038] Description of the reference numerals:

[0039] 1 - Data acquisition module; 2 - Data annotation conversion module; 3 - Model training and verification module; 4 - Behavior recognition module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the specific embodiments of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, and other embodiments can also be obtained.

[0041] To make the drawings concise, only the parts related to the present invention are schematically shown in each drawing, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, in some drawings, components with the same structure or function are only schematically shown for one of them, or only one of them is marked. In this article, "one" not only means "only this one", but also means "more than one" situation.

[0042] See Figure 1 , this application provides a method for identifying logistics violation behaviors, including the following steps:

[0043] S1: Collect and obtain several pictures of logistics violation behaviors;

[0044] S2: Perform data annotation on the logistics violation behaviors appearing in several pictures, and convert and generate a picture data set in TFRecord format;

[0045] S3: Based on the TensorFlow object detection API, train the YOLOv3 model that has been pre-converted into a TensorFlow frozen model through the picture data set, and verify the trained YOLOv3 model to obtain a violation recognition model;

[0046] S4: Identify the logistics scenario through the violation recognition model, judge whether there are logistics violation behaviors, and obtain the recognition result.

[0047] The above embodiments will now be described in detail, but not limited thereto.

[0048] This embodiment is applicable to the field of logistics intelligent technology for identifying logistics violation behaviors. Among them, the technical solution of this embodiment focuses on the implementation method of logistics violation behavior identification, aiming to propose an efficient, fast, and easily repeatable implementation method for logistics violation behavior identification. Secondly, it lies in the application of the logistics violation behavior identification based on the implementation, especially for the identification of package trampling. Obviously, this application can also be applied to logistics violation behaviors such as package throwing and package kicking in the logistics field.

[0049] In step S1 of this embodiment, based on the application scenario and the logistics violation behaviors to be recognized, several pictures of the logistics violation behaviors can be collected in the form of photography. There is no limit to the number of pictures obtained in this embodiment. In principle, the more pictures are collected, the better, so as to improve the recognition effect of logistics violation behaviors in actual applications. Preferably, under the condition of limited pictures, this embodiment can also expand the number of pictures through image enhancement methods. For example, translate the image within a certain scale range, rotate the image within a certain angle range, horizontally or vertically flip the image, crop a piece from the original image, magnify or reduce the image within a certain scale, perform some changes on the RGB color space of the image, add some artificially generated noise (such as Gaussian noise) to the image, etc., so as to significantly increase the data volume of the pictures. Specifically, if there are pictures with too large sizes and unclear feature regions among the several pictures obtained in step S1, the several pictures can be pre-processed by cropping to form pictures of the same size, so as to facilitate subsequent model training operations and improve the training effect and efficiency.

[0050] In step S2 of this embodiment, refer to Figure 2 , first, through step S21, for each picture collected, this embodiment annotates the logistics violation behaviors through data annotation-related software, and assigns information about the logistics violation behaviors. Among them, the annotation form adopted in this embodiment is rectangular box annotation. Now, the above data annotation will be described by examples: For example, for the logistics violation behavior of package trampling in this embodiment, the annotation includes two parts. One part is the operator's foot, and the other part is the trampled package. These two parts together constitute a package trampling behavior; Another example is the logistics violation behavior of package throwing. The annotation includes two parts. One part is the operator's hand, and the other part is the trampled package. These two parts together constitute a package throwing behavior. Thus, for a logistics violation behavior, corresponding feature annotations can be made according to the actual violation phenomenon. For the data annotation of specific other logistics violation behaviors, this embodiment will not elaborate here.

[0051] In step S2 of this embodiment, for the several images obtained after data annotation, this embodiment converts them into a picture dataset in the TFRecord format. Among them, the data in the TFRecord file is stored in the format of tf.train.Example Protocol Buffer. The TFRecord format is a binary file, which can make better use of memory, is more convenient for copying and moving, and does not require a separate label file. Specifically, refer to Figure 2, in this embodiment, after step S22, several images after data annotation are converted into Example objects (tf.train.Example objects), filled into the Example protocol buffer, and then after step S23, the Example objects are serialized into strings and written into the TFRecord file through a preset format conversion script (tf.python_io.TFRecordWriter) to obtain the image dataset in the TFRecord format required by this embodiment. Among them, the information written into the TFRecord file at least includes the information of the image itself, the data annotation information, and the annotation name information. The data standard information is the position information of the annotation rectangle box, and the annotation name information is the name of the content annotated in the rectangle box.

[0052] In step S3 of this embodiment, based on the TensorFlow Object Detection API, the YOLOv3 model pre-converted into a TensorFlow frozen model is trained through an image dataset, and the trained YOLOv3 model is verified to obtain a violation recognition model. Specifically, TensorFlow is a symbolic mathematics system based on dataflow programming and is widely used in the programming implementation of various machine learning algorithms, while the TensorFlow Object Detection API is an object detection application programming interface in TensorFlow. Before step S1 is executed, in this embodiment, through step S0, the configuration file of the TensorFlow Object Detection API is configured, and the pre-trained YOLOv3 model is converted into a pb frozen model of TensorFlow. The pb frozen model is the YOLOv3 model in pb format. The API of this embodiment is configured with, for example, a pipeline file for configuring parameters related to image preprocessing, a labe_map.pbtxt file, etc. Among them, various violations in logistics transportation are recorded in the labe_map.pbtxt file, and each behavior is specifically marked or named. In actual operation, when identifying the collected images, it is necessary to call the labe_map.pbtxt file to determine whether there are recorded violations in the images. The labe_map.pbtxt file can be regarded as a violation behavior library in logistics transportation and is the control object for the identification of logistics violation behaviors in the present invention. By adopting the above API configuration in this embodiment, relevant preprocessing, training and other related parameters in the above files can be directly changed, which greatly simplifies the subsequent implementation process of identifying new logistics violation behaviors. Based on this API, the YOLOv3 model that has been converted into a TensorFlow frozen model is trained through an image dataset in TFRecord format, and the trained YOLOv3 model is verified to obtain a violation recognition model. Preferably, the training process and the verification process in this embodiment are preferably carried out synchronously. In this way, a complete mAP (mean Average Precision) curve can be observed.

[0053] In step S4 of this embodiment, the on-site images or on-site videos of the logistics scenario can be monitored and obtained; each static image of the on-site images or on-site videos is identified through the violation recognition model to determine whether there are logistics violations, and the recognition result is obtained. Specifically, the violation recognition model based on the YOLOv3 model uses the entire image as the input of the network and directly regresses the position of the bounding box and the category to which the bounding box belongs at the output layer. For the specific network structure, see Figure 3, where for the image to be detected, the feature map is obtained through feature extraction, and then according to the size of the feature map, the image is divided into an N x N grid of equal proportion. Then the corresponding grid is used for corresponding object detection. Each grid predicts 3 bounding boxes, and the one with the largest intersection over union (IoU) between the predicted bounding box and the actually labeled bounding box is the predicted bounding box. For each bounding box, five values need to be predicted, namely (x, y, w, h) and confidence. The first four values represent the coordinate information of the predicted bounding box, including the x and y coordinates of a point of a rectangle and the width w and height h of the rectangle. Confidence represents two pieces of information, namely the confidence that the predicted box contains an object and how accurate the prediction of this box is, and is calculated as:

[0054]

[0055] Among them, if an object falls into a grid cell, the first item takes 1, otherwise it takes 0. The second item is the IoU value between the predicted bounding box and the actual ground truth. Each grid also needs to predict a class information, that is, the number of classes to be predicted. For example, for N x N grids, the total number of predicted bounding boxes per grid is denoted as B, and the total number of predicted categories is denoted as C, then the output is a tensor of size N x N x (5 x B + C).

[0056] In this embodiment, YOLOv3 and the TensorFlow Object Detection API are combined and applied to the logistics field for detecting violations. Among them, the YOLOv3 model is converted into a TensorFlow PB frozen model, and the image data for model training is collected and converted into the TFRecord format. Based on the configured API, training and validation are performed through the image dataset in the TFRecord format. In this way, the training speed of the YOLOv3 model for logistics violation recognition is greatly improved. At the same time, based on the YOLOv3 model, the accuracy and speed in the training and supervision processes are improved. In addition, if it is necessary to supervise new logistics violations, through the above technical solution, only by re-collecting the images of logistics violations and modifying the configuration file of the TensorFlow Object Detection API, the corresponding violation recognition model can be obtained very conveniently, so as to realize the intelligent supervision of logistics violations, standardize the behavior of logistics personnel, ensure the interests of users, provide higher service quality, reduce the losses of the logistics industry, and improve the transfer efficiency and quality.

[0057] See Figure 4, this application also provides a logistics violation behavior recognition device based on the above embodiments, including: a data acquisition module 1, a data annotation conversion module 2, a model training and verification module 3, and a behavior recognition module 4.

[0058] The data acquisition module 1 is used to collect a number of pictures of logistics violation behaviors. Among them, based on the application scenario and the logistics violation behavior to be recognized, a number of pictures of logistics violation behaviors can be collected in the form of photography. In this embodiment, there is no limit to the number of pictures obtained. In principle, the more pictures are collected, the better, so as to improve the recognition effect of logistics violation behaviors in actual applications. Preferably, under the condition of limited pictures, this embodiment can also expand the number of pictures through image enhancement methods. For example, translating the image within a certain scale range, rotating the image within a certain angle range, horizontally flipping or vertically flipping the image, cropping a piece from the original image, magnifying or reducing the image within a certain scale, making some changes to the RGB color space of the image, adding some artificially generated noise (such as Gaussian noise) to the image, etc., so as to significantly increase the data volume of the pictures. Specifically, if there are pictures with too large sizes and unclear feature regions among the obtained pictures, the pictures can be pre-processed by cropping to form pictures of the same size, so as to facilitate subsequent model training operations and improve the training effect and efficiency.

[0059] The data annotation conversion module 2 is used to perform data annotation on the logistics violation behaviors that appear in a number of pictures and convert them into a picture data set in TFRecord format. Among them, first, for each picture collected, this embodiment annotates the logistics violation behaviors through data marking-related software, endowing the information of the logistics violation behaviors. Among them, the annotation form adopted in this embodiment is rectangular box annotation. Now, the above data annotation is illustrated by examples: For example, for the logistics violation behavior of package trampling in this embodiment, the annotation includes two parts. One part is the operator's foot, and the other part is the trampled package. These two parts together constitute a package trampling behavior; Another example is the logistics violation behavior of package throwing. Then the annotation includes two parts. One part is the operator's hand, and the other part is the trampled package. These two parts together constitute a package throwing behavior. Thus, for a logistics violation behavior, corresponding feature annotations can be made according to the actual violation phenomenon. For the data annotation of other specific logistics violation behaviors, this embodiment will not elaborate here.

[0060] After data annotation by the data annotation conversion module 2, a number of images are obtained. In this embodiment, they are converted into a picture dataset in TFRecord format. Among them, the data in the TFRecord file are all stored in the format of tf.train.Example Protocol Buffer. The TFRecord format is a binary file, which can make better use of memory, is more convenient for copying and moving, and does not require a separate label file. Specifically, the data annotation conversion module 2 in this embodiment converts a number of pictures after data annotation into Example objects (tf.train.Example objects), fills them into the Example protocol buffer, then serializes the Example objects into strings, and writes them into the TFRecord file through a preset format conversion script (tf.python_io.TFRecordWriter) to obtain the picture dataset in TFRecord format required by this embodiment. Among them, the information written into the TFRecord file at least includes the picture itself information, data annotation information, and annotation name information. The data standard information is the position information of the annotation rectangle, and the annotation name information is the name of the content marked in the rectangle.

[0061] The model training and verification module 3 is used to train the YOLOv3 model pre-converted into a TensorFlow frozen model based on the TensorFlow Object Detection API with a picture dataset, and verify the trained YOLOv3 model to obtain a violation recognition model. Specifically, TensorFlow is a symbolic mathematics system based on dataflow programming and is widely used in the programming implementation of various machine learning algorithms, while the TensorFlow Object Detection API is an object detection application programming interface in TensorFlow. Before model training, in this embodiment, the configuration file of the TensorFlow Object Detection API is configured, and the pre-trained YOLOv3 model is converted into a pb frozen model of TensorFlow, and the pb frozen model is a YOLOv3 model in pb format. The API of this embodiment is configured with, for example, a pipeline file for configuring parameters related to preprocessing of pictures, a labe_map.pbtxt file, etc. Among them, various violations in logistics transportation are recorded in the labe_map.pbtxt file, and each behavior is specifically marked or named. In actual operation, when identifying the collected pictures, it is necessary to call the labe_map.pbtxt file to determine whether there are recorded violations in the pictures. The labe_map.pbtxt file can be regarded as a violation behavior library in logistics transportation and is the reference object for the present invention to identify logistics violation behaviors. By adopting the above API configuration in this embodiment, relevant preprocessing, training and other related parameters in the above file can be directly changed, which greatly facilitates and simplifies the subsequent implementation process of identifying new logistics violation behaviors. Based on this API, the YOLOv3 model converted into a TensorFlow frozen model is trained with a picture dataset in TFRecord format, and the trained YOLOv3 model is verified to obtain a violation recognition model. Preferably, the training process and the verification process in this embodiment are preferably carried out synchronously. In this way, a complete mAP (mean Average Precision) curve can be observed.

[0062] The behavior recognition module 4 is used to identify the logistics scenario through a violation recognition model, determine whether there is a logistics violation behavior, and obtain the recognition result. Among them, it is possible to monitor and obtain on-site pictures or on-site videos of the logistics scenario; identify each frame of static image of the on-site picture or on-site video through the violation recognition model, determine whether there is a logistics violation behavior, and obtain the recognition result. Specifically, the violation recognition model based on the YOLOv3 model uses the entire image as the input of the network and directly regresses the position of the bounding box and the category to which the bounding box belongs at the output layer. For the picture to be detected, a feature map is obtained through feature extraction, and then according to the size of the feature map, the image is divided into N x N grid cells of equal proportion. Then the corresponding grid is used for the corresponding object detection. Each grid will predict 3 bounding boxes, and the one with the largest intersection over union (IoU) between the predicted bounding box and the actually labeled bounding box is the predicted bounding box. For each bounding box, five values need to be predicted, namely (x, y, w, h) and confidence. The first four values represent the coordinate information of the predicted bounding box, including the x and y coordinates of a point and the width w and height h of a rectangle. Confidence represents two pieces of information: the confidence that the predicted box contains an object and how accurate the prediction of this box is. It is calculated as:

[0063]

[0064] Among them, if an object falls into a grid cell, the first item takes 1, otherwise it takes 0. The second item is the IoU value between the predicted bounding box and the actual ground truth. Each grid also needs to predict a category information, that is, the number of categories to be predicted. For example, for N x N grids, each grid, the total number of predicted bounding boxes is denoted as B, and the total number of predicted categories is denoted as C, then the output is a tensor of size N x N x (5 x B + C).

[0065] In this embodiment, YOLOv3 is combined with the TensorFlow object detection API and applied to the logistics field for detecting violations. Specifically, the YOLOv3 model is converted into a TensorFlow PB frozen model, and the image data for model training is collected and converted into the TFRecord format. Based on the configured API, training and validation are performed using this TFRecord-format image dataset. In this way, the training speed of the YOLOv3 model for logistics violation recognition is greatly improved. At the same time, based on the YOLOv3 model, the accuracy and speed in the training and supervision processes are enhanced. Additionally, if it is necessary to supervise new logistics violations, through the above technical solution, only by re-collecting the images of logistics violations and modifying the configuration file of the TensorFlow object detection API, the corresponding violation recognition model can be obtained very conveniently, thereby realizing the intelligent supervision of logistics violations, standardizing the behavior of logistics personnel, protecting the interests of users, providing higher service quality, reducing the losses of the logistics industry, and improving the transfer efficiency and quality.

[0066] This application also proposes a computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the logistics violation recognition method mentioned in the above embodiment.

[0067] This application also proposes a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the logistics violation recognition method mentioned in the above embodiment.

[0068] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0069] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, provided that these changes fall within the scope of the claims of the present invention and their equivalent technologies, they still fall within the protection scope of the present invention.

Claims

1. A method for identifying logistics violation behaviors, characterized in that, it includes the following steps: S1: Collect and obtain several pictures of logistics violation behaviors S2: Perform data annotation on the logistics violation behaviors appearing in several pictures, and convert and generate a picture dataset in TFRecord format; S3: Based on the TensorFlow Object Detection API, train the YOLOv3 model that has been pre-converted into a TensorFlow frozen model through the picture dataset, and verify the trained YOLOv3 model to obtain a violation identification model; S4: Identify the logistics scenario through the violation identification model, judge whether there is the logistics violation behavior, and obtain the identification result Among them, the step S4 further includes: The violation identification module extracts features from the picture to be detected to obtain a feature map, and then divides the image into N × N grids of equal proportion according to the size of the feature map; then target detection is performed by the corresponding grids; When performing target detection, each grid predicts three bounding boxes, and the one with the largest intersection over union of the predicted bounding box and the actual annotated bounding box is the output bounding box; For each output bounding box, five values are predicted, namely x, y, w, h, and confidence. Among them, x and y are the coordinates of the midpoint of the bounding box, w is the width of the bounding box, h is the height of the bounding box, and Confidence represents the confidence and accuracy that the predicted bounding box contains the target.

2. The method for identifying logistics violation behaviors according to claim 1, characterized in that, in the step S2, the conversion and generation of the picture dataset in TFRecord format further includes: Convert several pictures after data annotation into Example objects; Serialize the Example objects into strings and write them into a TFRecord file through a preset format conversion script to obtain the picture dataset in TFRecord format. Among them, the information written into the TFRecord file at least includes picture itself information, data annotation information, and annotation name information.

3. The method for identifying logistics violation behaviors according to claim 1, characterized in that, before the execution of the step S1, it further includes a step S0: Configure the TensorFlow Object Detection API and convert the pre-trained YOLOv3 model into the TensorFlow frozen model, where the TensorFlow frozen model is in PB format.

4. The method for identifying logistics violation behaviors according to claim 1, characterized in that, the step S4 further includes: Monitor and obtain on-site pictures or on-site videos of the logistics scenario; Identify each static image of the on-site pictures or the on-site videos through the violation identification model, judge whether there is the logistics violation behavior, and obtain the identification result.

5. A device for identifying logistics violation behaviors, characterized in that, it includes: A data acquisition module for collecting and obtaining several pictures of logistics violation behaviors; A data annotation conversion module, which is used to perform data annotation on the logistics violation behaviors appearing in a number of pictures, and convert and generate a picture dataset in TFRecord format; A model training and verification module, which is used to train a YOLOv3 model pre-converted into a TensorFlow frozen model based on the TensorFlow Object Detection API through the picture dataset, and verify the trained YOLOv3 model to obtain a violation recognition model; A behavior recognition module, which is used to recognize a logistics scenario through the violation recognition model, judge whether there is the logistics violation behavior, and obtain a recognition result Among them, the behavior recognition module is further configured as: For a picture to be detected, a feature map is obtained through feature extraction, and then according to the size of the feature map, the image is divided into N × N grids of equal proportion; then target detection is performed by the corresponding grids; When performing target detection, each grid predicts three bounding boxes, and the one with the largest intersection over union of the predicted bounding box and the actual annotated bounding box is the output bounding box; For each output bounding box, five values are predicted, namely x, y, w, h, and confidence. Among them, x and y are the coordinates of the midpoint of the bounding box, w is the width of the bounding box, h is the height of the bounding box, and Confidence represents the confidence and accuracy that the predicted bounding box contains a target.

6. The logistics violation behavior recognition device according to claim 5, wherein, The data annotation conversion module is specifically used to perform data annotation on the logistics violation behaviors appearing in a number of pictures through rectangular box annotation, convert a number of pictures after data annotation into Example objects, serialize the Example objects into strings, and write them into a TFRecord file through a preset format conversion script to obtain the picture dataset in TFRecord format, wherein the information written into the TFRecord file at least includes picture itself information, data annotation information, and annotation name information.

7. A computer device, wherein, It includes a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes the logistics violation behavior recognition method according to any one of claims 1 to 4.

8. A storage medium storing computer-readable instructions, wherein, When the computer-readable instructions are executed by one or more processors, the one or more processors execute the logistics violation behavior recognition method according to any one of claims 1 to 4.

Images