Method, device, equipment and medium for real-time on-the-job identification of loading and unloading personnel

Abstract

The present invention provides a method, device, equipment and medium for real-time on-the-job identification of loading and unloading personnel, including: obtaining on-site pictures of a logistics transfer center, and preprocessing the obtained on-site pictures of the logistics transfer center to obtain preprocessed on-site pictures; constructing a data set based on the preprocessed on-site pictures; constructing a target detector; using the data set to train the target detector, and using the trained target detector to perform on-the-job identification of the loading and unloading personnel of the logistics transfer center; the target detector is an end-to-end target detector based on Transformer, which is used to perform real-time on-the-job identification and detection of the loading and unloading personnel of the logistics transfer center. The present invention not only improves the intelligence and automation of the management of loading and unloading personnel, but also realizes the efficient use and safety of resources, and provides significant support and optimization for the overall operation of the logistics transfer center.

Description

Technical Field

[0001] The present invention relates to the field of logistics management technology, and specifically, to a method, device, equipment and medium for real-time on-the-job identification of loading and unloading personnel, and more specifically, to a method, device, equipment and medium for real-time on-the-job identification of loading and unloading personnel in a logistics transfer center. Background Art

[0002] In modern logistics transfer centers, ensuring the on-the-job status of loading and unloading personnel is crucial to maintaining operational efficiency and safety. However, traditional manual monitoring methods have problems such as low efficiency and susceptibility to human factors. With the development of artificial intelligence and computer vision technology, especially the advancement of deep learning technology, it has become possible to realize the automated and intelligent monitoring of the on-the-job status of loading and unloading personnel. Summary of the invention

[0003] The main purpose of the present invention is to solve the technical problems in the prior art that manual monitoring is inefficient and easily affected by human factors.

[0004] A first aspect of the present invention provides a method for real-time on-the-job identification of loading and unloading personnel, comprising: Acquire on-site pictures of the logistics transfer center, and pre-process the acquired on-site pictures of the logistics transfer center to obtain pre-processed on-site pictures; Construct a dataset based on preprocessed on-site images; Build an object detector; Use the dataset to train the target detector, and use the trained target detector to identify the loading and unloading personnel in the logistics transfer center on the job; The target detector is an end-to-end target detector based on Transformer, which is used for real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the acquiring of a site picture of the logistics transfer center and preprocessing the acquired site picture of the logistics transfer center to obtain a preprocessed site picture include: Obtain on-site pictures based on the cameras in the logistics transfer center; De-noising the acquired on-site pictures of the logistics transfer center to obtain denoised on-site pictures; The denoised scene picture is cropped and scaled according to preset requirements to obtain a cropped and scaled scene picture; The pixel values ​​of the cropped and scaled scene picture are normalized to obtain a normalized scene picture.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the denoising process of the acquired on-site pictures of the logistics transfer center includes: The obtained on-site pictures of the logistics transfer center are denoised by using the Gaussian denoising method to obtain denoised on-site pictures; The Gaussian denoising method includes at least one of mean filtering, median filtering and Gaussian filtering.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the target detector includes: a backbone network, a hybrid encoder, and a Transformer decoder with an auxiliary prediction head; The backbone network is used to extract features from on-site pictures, and then the output features of the last three stages of the backbone network are extracted and marked as S3, S4 and S5 respectively; the hybrid encoder decouples the intra-scale interaction of the output feature S5, marks its output feature as F5, and performs cross-scale feature fusion of feature F5 with features S3 and S4 to convert them into image feature sequences; the Transformer decoder with an auxiliary prediction head filters a fixed number of image features from the image feature sequence through the IoU-aware query selection technology, regards the filtered image features as the initial objects of the Transformer decoder, and iteratively optimizes the object query in the Transformer decoder to generate bounding boxes and corresponding scores, thereby realizing the prediction of loading and unloading personnel in the logistics transfer center based on on-site pictures.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, in the process of training the target detector using the data set, the data set is enhanced using Mosaic data enhancement technology to obtain a data set after data enhancement processing; The target detector is trained using the data set processed with data enhancement to obtain a trained target detector.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the method further includes: Performing quantization and pruning processing on the trained target detector; The quantization process includes: integer quantization and floating point quantization; The pruning process includes: structured pruning and unstructured pruning; During the process of quantizing and pruning the trained target detector, experiments and verification are performed on the current target detector to ensure that the quantized and pruned target detector meets preset requirements.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, the using of the trained target detector to perform on-the-job identification of the loading and unloading personnel of the logistics transfer center includes: using the trained target detector deployed on the edge device to perform on-the-job identification of the loading and unloading personnel of the logistics transfer center; The on-site image is acquired based on the camera of the logistics transfer center, and the acquired on-site image is subjected to denoising, cropping, scaling and normalization processing to obtain a processed on-site image; The processed on-site images are input into the target detector deployed on the edge device, and the target detector deployed on the edge device is used to detect and identify the loading and unloading personnel to determine whether the loading and unloading personnel are on duty; At the same time, monitoring tools are used to monitor the performance indicators of target detectors deployed on edge devices and to diagnose and repair faults; wherein the performance indicator monitoring includes: inference speed, accuracy and recall rate.

[0011] A second aspect of the present invention provides a real-time on-the-job identification device for loading and unloading personnel, comprising: The picture acquisition module is used to acquire the on-site pictures of the logistics transfer center and pre-process the acquired on-site pictures of the logistics transfer center to obtain the pre-processed on-site pictures; A dataset construction module, used to construct a dataset based on preprocessed on-site images; Model building module for building object detectors; The target recognition module is used to train the target detector using the data set, and use the trained target detector to perform on-the-job recognition of the loading and unloading personnel in the logistics transfer center; The target detector is an end-to-end target detector based on Transformer, which is used for real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center.

[0012] Optionally, in a first implementation of the second aspect of the present invention, the picture acquisition module includes: The on-site picture acquisition submodule is used to acquire on-site pictures based on the camera of the logistics transfer center; The denoising submodule is used to denoise the acquired on-site pictures of the logistics transfer center to obtain denoised on-site pictures; The cropping and scaling submodule is used to crop and scale the denoised scene picture according to preset requirements to obtain a cropped and scaled scene picture; The normalization processing submodule is used to normalize the pixel values ​​of the cropped and scaled scene picture to obtain the normalized scene picture.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the denoising submodule includes: The obtained on-site pictures of the logistics transfer center are denoised by using the Gaussian denoising method to obtain denoised on-site pictures; The Gaussian denoising method includes at least one of mean filtering, median filtering and Gaussian filtering.

[0014] Optionally, in a third implementation of the second aspect of the present invention, the target detector includes: a backbone network, a hybrid encoder, and a Transformer decoder with an auxiliary prediction head; The backbone network is used to extract features from on-site pictures, and then the output features of the last three stages of the backbone network are extracted and marked as S3, S4 and S5 respectively; the hybrid encoder decouples the intra-scale interaction of the output feature S5, marks its output feature as F5, and performs cross-scale feature fusion of feature F5 with features S3 and S4 to convert them into image feature sequences; the Transformer decoder with an auxiliary prediction head filters a fixed number of image features from the image feature sequence through the IoU-aware query selection technology, regards the filtered image features as the initial objects of the Transformer decoder, and iteratively optimizes the object query in the Transformer decoder to generate bounding boxes and corresponding scores, thereby realizing the prediction of loading and unloading personnel in the logistics transfer center based on on-site pictures.

[0015] Optionally, in a fourth implementation of the second aspect of the present invention, in the process of training the target detector using the data set, the data set is enhanced using Mosaic data enhancement technology to obtain a data set after data enhancement processing; The target detector is trained using the data set processed with data enhancement to obtain a trained target detector.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the device further includes: a quantization and pruning processing submodule, configured to perform quantization processing and pruning processing on the trained target detector; The quantization process includes: integer quantization and floating point quantization; The pruning process includes: structured pruning and unstructured pruning; During the process of quantizing and pruning the trained target detector, experiments and verification are performed on the current target detector to ensure that the quantized and pruned target detector meets preset requirements.

[0017] Optionally, in a sixth implementation of the second aspect of the present invention, the using of the trained target detector to perform on-the-job identification of the loading and unloading personnel of the logistics transfer center includes: using the trained target detector deployed on the edge device to perform on-the-job identification of the loading and unloading personnel of the logistics transfer center; The on-site image is acquired based on the camera of the logistics transfer center, and the acquired on-site image is subjected to denoising, cropping, scaling and normalization processing to obtain a processed on-site image; The processed on-site images are input into the target detector deployed on the edge device, and the target detector deployed on the edge device is used to detect and identify the loading and unloading personnel to determine whether the loading and unloading personnel are on duty; At the same time, monitoring tools are used to monitor the performance indicators of target detectors deployed on edge devices and to diagnose and repair faults; wherein the performance indicator monitoring includes: inference speed, accuracy and recall rate.

[0018] A third aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for real-time on-the-job identification of loading and unloading personnel.

[0019] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the above-mentioned method for real-time on-the-job identification of loading and unloading personnel.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention adopts a Transformer-based target detector, which can accurately identify loading and unloading personnel in different scenes and lighting conditions, improving the accuracy and response speed of the system; 2. Real-time acquisition and processing of on-site images to ensure that management personnel monitor the on-the-job status of loading and unloading personnel in real time, and promptly discover and handle potential safety hazards or personnel abnormalities; 3. Through multi-level feature extraction and fusion, the device of the present invention can effectively process image features of different scales and improve the robustness and accuracy of detection.

[0021] 4. The present invention is integrated with the management system of the logistics transfer center to realize the automation of personnel management. Managers can obtain job identification results in real time, which is helpful to optimize human resource scheduling and arrangement.

[0022] 5. The present invention ensures that the target detector can run efficiently on the edge device through the application of quantization and pruning technology, meeting the real-time recognition requirements in a resource-limited environment.

[0023] 6. The present invention adopts Mosaic data enhancement technology to enrich the training data, improve the generalization ability and robustness of the model, and make it still perform well in various complex scenarios; 7. The monitoring tools deployed on edge devices can monitor the model’s reasoning speed, accuracy, and recall rate in real time, ensuring the stable and efficient operation of the system and timely diagnosing and repairing faults; 8. The present invention can reduce the burden of manual management through intelligent management, while improving work efficiency and creating a safer and more effective environment for loading and unloading operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 A first flow chart of a method for real-time on-the-job identification of loading and unloading personnel provided in an embodiment of the present invention.

[0025] Figure 2 A second flow chart of the method for real-time on-the-job identification of loading and unloading personnel provided in an embodiment of the present invention.

[0026] Figure 3 A schematic structural diagram of a device for real-time on-the-job identification of loading and unloading personnel provided by an embodiment of the present invention.

[0027] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The embodiment of the present invention provides a method, device, equipment and medium for real-time on-the-job identification of loading and unloading personnel, obtains on-site pictures of the logistics transfer center, and preprocesses the obtained on-site pictures of the logistics transfer center to obtain preprocessed on-site pictures; constructs a data set based on the preprocessed on-site pictures; constructs a target detector; uses the data set to train the target detector, and uses the trained target detector to perform on-the-job identification of loading and unloading personnel in the logistics transfer center; the target detector is an end-to-end target detector based on Transformer, which is used to perform real-time on-the-job identification and detection of loading and unloading personnel in the logistics transfer center. The present invention solves the technical problems in the prior art that manual monitoring is inefficient and easily affected by human factors.

[0029] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 The first embodiment of the method for real-time on-the-job identification of loading and unloading personnel in the embodiment of the present invention includes: 101. Acquire on-site pictures of the logistics transfer center, and pre-process the acquired on-site pictures of the logistics transfer center to obtain pre-processed on-site pictures; In this embodiment, on-site pictures are obtained based on cameras in the logistics transfer center; wherein the cameras are installed at key locations in the logistics transfer center to ensure that different angles and scenes can be covered; The acquired scene pictures are preprocessed, including: denoising, cropping, scaling and normalization. Specifically, the acquired scene pictures are denoised by Gaussian denoising algorithm to obtain denoised scene pictures. By removing or reducing Gaussian noise in the image, the image quality can be improved and the image details can be further preserved. Then, according to the input requirements of the target detector, the image is cropped and scaled to ensure the consistency of the image size. The pixel values ​​of the image are normalized to be distributed within a specific range.

[0031] The Gaussian denoising algorithm includes mean filtering, median filtering and Gaussian filtering, etc., and the cv2.GaussianBlue() function is used to perform Gaussian filtering on the image.

[0032] This embodiment preprocesses the scene image to improve the performance and robustness of the target detector.

[0033] 102. Construct a data set based on the preprocessed on-site pictures; In this embodiment, the preprocessed on-site pictures are annotated to construct a data set.

[0034] 103. Build an object detector; In this embodiment, the object detector includes: a backbone network, a hybrid encoder, and a Transformer decoder with an auxiliary prediction head; The backbone network is used to extract features from on-site pictures, and then the output features of the last three stages of the backbone network are extracted and marked as S3, S4 and S5 respectively; the hybrid encoder decouples the intra-scale interaction of the output feature S5, marks its output feature as F5, and performs cross-scale feature fusion of feature F5 with features S3 and S4 to convert them into image feature sequences; the Transformer decoder with an auxiliary prediction head filters a fixed number of image features from the image feature sequence through the IoU-aware query selection technology, regards the filtered image features as the initial objects of the Transformer decoder, and iteratively optimizes the object query in the Transformer decoder to generate bounding boxes and corresponding scores, thereby realizing the prediction of loading and unloading personnel in the logistics transfer center based on on-site pictures.

[0035] The target detector in this embodiment combines the structure of the backbone network, hybrid encoder and Transformer decoder, which can effectively extract and fuse features of different scales and improve the recognition accuracy of loading and unloading personnel. This multi-level feature processing makes the model more adaptable to various scene changes.

[0036] 104. Use the dataset to train the target detector, and use the trained target detector to perform on-the-job identification of loading and unloading personnel in the logistics transfer center.

[0037] In this embodiment, in the process of training the target detector, the Mosaic data enhancement technology is used to enhance the data set; wherein the Mosaic data enhancement technology is a technology widely used in the field of target detection, which increases the diversity and richness of the data set by splicing multiple pictures into a new picture. Mosaic data enhancement technology is a technology that enhances image data sets and improves the generalization ability and robustness of target detectors. Two or four different images are spliced ​​together to form a new image to enhance the diversity and quantity of the data set. Each image is cropped into four fragments, which are randomly arranged, spliced ​​and superimposed.

[0038] The target detector is then verified and optimized on multiple rounds of test sets, and the target detector is continuously iterated and adjusted.

[0039] This embodiment can support efficient real-time processing and can quickly identify the on-the-job status of loading and unloading personnel, which is crucial to improving the management efficiency and safety of the logistics transfer center.

[0040] See also Figure 2The second embodiment of the method for real-time on-the-job identification of loading and unloading personnel in the embodiment of the present invention includes: 201. Acquire a site picture of the logistics transfer center, and pre-process the acquired site picture of the logistics transfer center to obtain a pre-processed site picture; In this embodiment, a live image of the logistics transfer center is acquired based on the camera of the logistics transfer center; and the acquired live image information is subjected to denoising, cropping, scaling and normalization processing to obtain a processed live image.

[0041] 202. Construct a data set based on the preprocessed on-site pictures; 203. Build a target detector; In this embodiment, the target detector is an end-to-end target detector based on Transformer, which is used to perform real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center. The Transformer not only has advantages in accuracy and speed, but also does not require post-processing. Therefore, the reasoning speed of the target detector will not be delayed and remain stable, making full use of the advantages of the end-to-end detection pipeline.

[0042] 204. Train the target detector using the data set to obtain a trained target detector; and perform quantization and pruning on the trained target detector to adapt to resource constraints of the edge device.

[0043] In this embodiment, the quantization process is to reduce the model size and computing requirements by reducing the accuracy of model parameters; while pruning is to remove unimportant weights or neurons in the model to simplify the model structure.

[0044] The quantization process includes: integer quantization and floating point quantization; The pruning process includes: structured pruning and unstructured pruning; During the process of quantizing and pruning the trained target detector, experiments and verification are performed on the current target detector to ensure that the quantized and pruned target detector maintains high performance and accuracy.

[0045] This embodiment ensures that the target detector can run efficiently on the edge device by quantizing and pruning the target detector, thereby maintaining good performance in resource-limited scenarios and optimizing the use of hardware resources.

[0046] 205. Deploy the quantized and pruned target detector on the edge device, and use the quantized and pruned target detector deployed on the edge device to perform real-time on-the-job identification and detection of the loading and unloading personnel in the logistics transfer center.

[0047] In this embodiment, a live image is acquired based on a camera of a logistics transfer center, and the acquired live image is subjected to denoising, cropping, scaling and normalization processing to obtain a processed live image; The processed on-site images are input into the target detector deployed on the edge device, and the target detector deployed on the edge device is used to detect and identify the loading and unloading personnel to determine whether the loading and unloading personnel are on duty; At the same time, monitoring tools are used to monitor the performance indicators of target detectors deployed on edge devices, as well as to diagnose and repair faults to ensure the stable operation of the entire device.

[0048] The performance indicators monitored include: reasoning speed, accuracy and recall rate.

[0049] In this embodiment, the real-time on-the-job identification device for loading and unloading personnel is integrated with the management system of the logistics transfer center to realize automated personnel management and monitoring. The job identification results can be fed back to the management system in real time so that the management personnel can conduct personnel scheduling and arrangements. At the same time, the management system can also send instructions to the real-time on-the-job identification device for loading and unloading personnel, such as adjusting the camera angle, activating the alarm, etc., to realize intelligent management and monitoring.

[0050] In the process of performance monitoring, fault diagnosis and repair of the target detector, this embodiment ensures the stable operation of the equipment and reduces the impact that may be caused by system failures.

[0051] This embodiment can promptly detect any absences or violations by real-time monitoring of the on-the-job status of loading and unloading personnel, thereby reducing the risk of accidents and improving on-site safety. Automated on-the-job identification can reduce the need for manual inspections, allowing managers to focus more time and energy on other key tasks, thereby improving overall logistics efficiency. By integrating the identification results with the management system to achieve real-time data feedback, managers can quickly obtain the on-the-job status of personnel, facilitate scheduling and arrangements, and reduce losses caused by lagging human management.

[0052] The above describes the real-time on-the-job identification method for loading and unloading personnel in the embodiment of the present invention. The following describes the real-time on-the-job identification device for loading and unloading personnel in the embodiment of the present invention. Figure 3 In one embodiment of the present invention, a device for real-time on-the-job identification of loading and unloading personnel includes: The picture acquisition module 301 is used to acquire the on-site picture of the logistics transfer center, and pre-process the acquired on-site picture of the logistics transfer center to obtain the pre-processed on-site picture; In this embodiment, the picture acquisition module 301 includes: The on-site picture acquisition submodule 3011 is used to acquire on-site pictures based on the cameras of the logistics transfer center; wherein the cameras are installed at key positions of the logistics transfer center to ensure that different angles and scenes can be covered; The denoising submodule 3012 is used to denoise the acquired on-site pictures of the logistics transfer center to obtain denoised on-site pictures; denoise the acquired on-site pictures using a Gaussian denoising algorithm to obtain denoised on-site pictures; by removing or reducing Gaussian noise in the image, the quality of the image can be improved and the image details can be further retained. The Gaussian denoising algorithm includes mean filtering, median filtering and Gaussian filtering, etc., and the cv2.GaussianBlue() function is used to perform Gaussian filtering on the image.

[0053] The cropping and scaling submodule 3013 is used to crop and scale the denoised scene picture according to preset requirements to obtain a cropped and scaled scene picture; then, according to the input requirements of the target detector, the image is cropped and scaled to ensure that the image size is consistent; The normalization processing submodule 3014 is used to normalize the pixel values ​​of the cropped and scaled scene picture so that the pixel values ​​are distributed within a specific range, thereby obtaining the normalized scene picture.

[0054] A data set construction module 302, used to construct a data set based on the preprocessed on-site pictures; In this embodiment, the preprocessed on-site pictures are annotated to construct a data set.

[0055] A model building module 303, used to build a target detector; In this embodiment, the target detector is an end-to-end target detector based on Transformer, which is used to perform real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center. The Transformer not only has advantages in accuracy and speed, but also does not require post-processing. Therefore, the reasoning speed of the target detector will not be delayed and remain stable, making full use of the advantages of the end-to-end detection pipeline.

[0056] The target detector includes: a backbone network, a hybrid encoder, and a Transformer decoder with an auxiliary prediction head; The backbone network is used to extract features from on-site pictures, and then the output features of the last three stages of the backbone network are extracted and marked as S3, S4 and S5 respectively; the hybrid encoder decouples the intra-scale interaction of the output feature S5, marks its output feature as F5, and performs cross-scale feature fusion of feature F5 with features S3 and S4 to convert them into image feature sequences; the Transformer decoder with an auxiliary prediction head filters a fixed number of image features from the image feature sequence through the IoU-aware query selection technology, regards the filtered image features as the initial objects of the Transformer decoder, and iteratively optimizes the object query in the Transformer decoder to generate bounding boxes and corresponding scores, thereby realizing the prediction of loading and unloading personnel in the logistics transfer center based on on-site pictures.

[0057] The target detector in this embodiment combines the structure of the backbone network, hybrid encoder and Transformer decoder, which can effectively extract and fuse features of different scales and improve the recognition accuracy of loading and unloading personnel. This multi-level feature processing makes the model more adaptable to various scene changes.

[0058] A training module 304 is used to train the target detector using the data set to obtain a trained target detector; In this embodiment, in the process of training the target detector, the Mosaic data enhancement technology is used to enhance the data set; wherein the Mosaic data enhancement technology is a technology widely used in the field of target detection, which increases the diversity and richness of the data set by splicing multiple pictures into a new picture. Mosaic data enhancement technology is a technology that enhances image data sets and improves the generalization ability and robustness of target detectors. Two or four different images are spliced ​​together to form a new image to enhance the diversity and quantity of the data set. Each image is cropped into four fragments, which are randomly arranged, spliced ​​and superimposed.

[0059] The target detector is then verified and optimized on multiple rounds of test sets, and the target detector is continuously iterated and adjusted.

[0060] The quantization and pruning module 305 is used to perform quantization and pruning processing on the trained target detector to adapt to the resource limitations of the edge device.

[0061] In this embodiment, the quantization process is to reduce the model size and computing requirements by reducing the accuracy of model parameters; while pruning is to remove unimportant weights or neurons in the model to simplify the model structure.

[0062] The quantization process includes: integer quantization and floating point quantization; The pruning process includes: structured pruning and unstructured pruning; During the process of quantizing and pruning the trained target detector, experiments and verification are performed on the current target detector to ensure that the quantized and pruned target detector maintains high performance and accuracy.

[0063] The recognition module 306 is used to deploy the target detector after quantization and pruning on the edge device, and use the target detector after quantization and pruning deployed on the edge device to perform real-time on-the-job recognition and detection of the loading and unloading personnel in the logistics transfer center.

[0064] In this embodiment, a live image is acquired based on a camera of a logistics transfer center, and the acquired live image is subjected to denoising, cropping, scaling and normalization processing to obtain a processed live image; The processed on-site images are input into the target detector deployed on the edge device, and the target detector deployed on the edge device is used to detect and identify the loading and unloading personnel to determine whether the loading and unloading personnel are on duty; At the same time, monitoring tools are used to monitor the performance indicators of target detectors deployed on edge devices, as well as to diagnose and repair faults to ensure the stable operation of the entire device.

[0065] The performance indicators monitored include: reasoning speed, accuracy and recall rate.

[0066] In this embodiment, the real-time on-the-job identification device for loading and unloading personnel is integrated with the management system of the logistics transfer center to realize automated personnel management and monitoring. The job identification results can be fed back to the management system in real time so that the management personnel can conduct personnel scheduling and arrangements. At the same time, the management system can also send instructions to the real-time on-the-job identification device for loading and unloading personnel, such as adjusting the camera angle, activating the alarm, etc., to realize intelligent management and monitoring.

[0067] above Figure 3 The real-time on-the-job identification device for loading and unloading vehicle personnel in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The electronic device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0068] Figure 47 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more mass storage devices) storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or permanent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the electronic device 700. Furthermore, the processor 710 may be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the electronic device 700.

[0069] The electronic device 700 may also include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 750, and / or one or more operating systems 731, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 4 The structure of the electronic device shown does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0070] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of a method for real-time on-the-job identification of loading and unloading personnel.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for real-time on-the-job identification of loading and unloading personnel, characterized in that: include: Acquire on-site pictures of the logistics transfer center, and pre-process the acquired on-site pictures of the logistics transfer center to obtain pre-processed on-site pictures; Construct a dataset based on preprocessed on-site images; Build an object detector; Use the dataset to train the target detector, and use the trained target detector to identify the loading and unloading personnel in the logistics transfer center on the job; The target detector is an end-to-end target detector based on Transformer, which is used for real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center.

2. The method for real-time on-the-job identification of loading and unloading personnel according to claim 1 is characterized in that: The obtaining of the on-site pictures of the logistics transfer center and preprocessing of the obtained on-site pictures of the logistics transfer center to obtain the preprocessed on-site pictures include: Obtain on-site pictures based on the cameras in the logistics transfer center; De-noising the acquired on-site pictures of the logistics transfer center to obtain denoised on-site pictures; The denoised scene picture is cropped and scaled according to preset requirements to obtain a cropped and scaled scene picture; The pixel values ​​of the cropped and scaled scene picture are normalized to obtain a normalized scene picture.

3. The method for real-time on-the-job identification of loading and unloading personnel according to claim 2 is characterized in that: The denoising process of the acquired on-site pictures of the logistics transfer center includes: The obtained on-site pictures of the logistics transfer center are denoised by using the Gaussian denoising method to obtain denoised on-site pictures; The Gaussian denoising method includes at least one of mean filtering, median filtering and Gaussian filtering.

4. The method for real-time on-the-job identification of loading and unloading personnel according to claim 1, characterized in that: The target detector includes: a backbone network, a hybrid encoder, and a Transformer decoder with an auxiliary prediction head; The backbone network is used to extract features from on-site pictures, and then the output features of the last three stages of the backbone network are extracted and marked as S3, S4 and S5 respectively; the hybrid encoder decouples the intra-scale interaction of the output feature S5, marks its output feature as F5, and performs cross-scale feature fusion of feature F5 with features S3 and S4 to convert them into image feature sequences; the Transformer decoder with an auxiliary prediction head filters a fixed number of image features from the image feature sequence through the IoU-aware query selection technology, regards the filtered image features as the initial objects of the Transformer decoder, and iteratively optimizes the object query in the Transformer decoder to generate bounding boxes and corresponding scores, thereby realizing the prediction of loading and unloading personnel in the logistics transfer center based on on-site pictures.

5. The method for real-time on-the-job identification of loading and unloading personnel according to claim 1, characterized in that: In the process of using the data set to train the target detector, the data set is enhanced by using the Mosaic data enhancement technology to obtain a data set after data enhancement; The target detector is trained using the data set processed with data enhancement to obtain a trained target detector.

6. The method for real-time on-the-job identification of loading and unloading personnel according to claim 1, characterized in that: The method further comprises: Performing quantization and pruning processing on the trained target detector; The quantization process includes: integer quantization and floating point quantization; The pruning process includes: structured pruning and unstructured pruning; During the process of quantizing and pruning the trained target detector, experiments and verification are performed on the current target detector to ensure that the quantized and pruned target detector meets preset requirements.

7. The method for real-time on-the-job identification of loading and unloading personnel according to claim 6 is characterized in that: The method of using the trained target detector to identify the loading and unloading personnel of the logistics transfer center on the job includes: using the trained target detector deployed on the edge device to identify the loading and unloading personnel of the logistics transfer center on the job; The on-site image is acquired based on the camera of the logistics transfer center, and the acquired on-site image is subjected to denoising, cropping, scaling and normalization processing to obtain a processed on-site image; The processed on-site images are input into the target detector deployed on the edge device, and the target detector deployed on the edge device is used to detect and identify the loading and unloading personnel to determine whether the loading and unloading personnel are on duty; At the same time, monitoring tools are used to monitor the performance indicators of target detectors deployed on edge devices and to diagnose and repair faults; wherein the performance indicator monitoring includes: inference speed, accuracy and recall rate.

8. A real-time on-the-job identification device for loading and unloading personnel, characterized in that: include: The picture acquisition module is used to acquire the on-site pictures of the logistics transfer center and pre-process the acquired on-site pictures of the logistics transfer center to obtain the pre-processed on-site pictures; A dataset construction module, used to construct a dataset based on preprocessed on-site images; Model building module for building object detectors; The target recognition module is used to train the target detector using the data set, and use the trained target detector to perform on-the-job recognition of the loading and unloading personnel in the logistics transfer center; The target detector is an end-to-end target detector based on Transformer, which is used for real-time on-the-job recognition and detection of loading and unloading personnel in a logistics transfer center.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for real-time on-the-job identification of loading and unloading personnel described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the method for real-time on-the-job identification of loading and unloading personnel described in any one of claims 1 to 7 are implemented.

Images