Intelligent logistics sorting method and device, terminal and storage medium

Abstract

The invention discloses an intelligent logistics sorting method and device, a terminal and a storage medium, and relates to the field of logistics sorting.The intelligent logistics sorting method is based on a multi-face scanning system, the system comprises a plurality of industrial camera sets, an entrance photoelectric sensor and an exit photoelectric sensor, and the method comprises the steps that flight information is pulled from a warehouse management system, and large package distribution information is obtained from a logistics distribution system; the control unit is used for controlling the camera set to collect cargo box images and record information when receiving signals of the entrance photoelectric sensor, carrying out bar code decoding and OCR identification on the collected images and fusing and matching with flight information, carrying out photoelectric verification when receiving information of the exit photoelectric sensor, and outputting main order numbers and large package numbers for sorting after verification is passed. According to the logistics container sorting device, the technical effects that the logistics containers are efficiently and accurately sorted, the sorting errors are reduced, and the logistics sorting efficiency and accuracy are improved are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of logistics automation sorting, in particular to an intelligent logistics sorting method, device, terminal and storage medium. BACKGROUND

[0002] In today's competitive market environment, with the rapid development of e-commerce and express industry, the importance of logistics sorting is increasingly prominent, and the performance of the logistics sorting system has become one of the important factors for enterprises to improve competitiveness.

[0003] In the field of logistics sorting, in order to solve the problems of package sorting and information acquisition, various technical means are commonly used. Cross-belt sorting machine is a common one, which is composed of transmission belt, package supply table, sorting trolley, code reading system and package falling system, etc. By the coordinated operation of the transmission belt and the sorting trolley, it can efficiently handle the sorting and transportation of large quantities of packages.

[0004] However, the existing logistics sorting system has many defects. In the process of handling package sorting, a large amount of manual intervention is required, such as manual adjustment of package placement position to ensure that the face sheet is facing up, which undoubtedly increases the labor cost. Moreover, the traditional sorting system has low sorting efficiency when facing complex orders or large quantities of packages, which is difficult to meet the rapid growth demand of e-commerce and express industry. At the same time, the existing system has deficiencies in information processing, which cannot effectively integrate bar code information and OCR information, resulting in low matching accuracy. And there are deficiencies in automatic matching of flight information, which requires manual intervention, increasing the error rate. In addition, the traditional system can only identify part of the information of the box, cannot fully obtain all the key information on the box, and has no verification mechanism, and the discovery ability of the error box is insufficient. The existing volume measurement system also has deficiencies in accuracy, which cannot meet the demand of high-precision sorting, and the system also has difficulties in integration with other systems such as WMS, resulting in low efficiency of data sharing and process collaboration. SUMMARY

[0005] In order to achieve the effect of improving the efficiency and accuracy of logistics sorting, the present application provides an intelligent logistics sorting method, device, terminal and storage medium.

[0006] In the first aspect, the present application provides an intelligent logistics sorting method, which adopts the technical scheme as follows: An intelligent logistics sorting method based on a multi-scan system, the multi-scan system comprising a plurality of industrial camera groups arranged in a predetermined scanning area, the entrance and exit of the scanning area being provided with an entrance photoelectric sensor and an exit photoelectric sensor respectively, the method comprising the following steps: Pulling flight information from a pre-established warehouse management system, the flight information including the master number and the large package number of all cargo boxes; When receiving the first photoelectric trigger signal sent by the entrance photoelectric sensor, control all the industrial camera groups to collect images of the box passing through the entrance photoelectric sensor to obtain first collected images, and record the first photoelectric trigger information; Based on the first collected images, perform barcode decoding and OCR identification, fuse and match the obtained barcode decoding information, OCR identification information, and the pulled flight information to obtain the corresponding master number and the package number; When receiving the second photoelectric trigger information recorded by the exit photoelectric sensor, perform photoelectric verification combined with the second photoelectric trigger information, and output the master number and the package number for sorting when the photoelectric verification is passed.

[0007] By adopting the above technical solution, the flight information can be automatically pulled from the warehouse management system, the package allocation information can be obtained from the logistics distribution system, the entrance photoelectric sensor can be used to trigger the industrial camera group to collect box images, the information can be fused and matched to obtain the master number and the package number after barcode decoding and OCR identification, and the photoelectric verification can be performed through the exit photoelectric sensor. Finally, the information is output for sorting, which realizes the automation of logistics sorting, reduces manual intervention, improves sorting efficiency, improves the accuracy of flight information matching through information fusion and matching and photoelectric verification, and can obtain key information such as barcodes and OCR text of the box at one time. The integrated process from information acquisition to sorting is realized through the cooperation of each step.

[0008] Preferably, the flight information is pulled from the pre-established warehouse management system, and the flight information includes master numbers and package numbers of all boxes, and specifically includes the following steps: According to the flight number, the corresponding flight information is pulled from the database of the pre-established warehouse management system, the flight information includes master numbers and package numbers of all boxes, each flight includes a plurality of master numbers, each master number includes a plurality of package numbers, and each box has a unique corresponding package number; According to the flight number, the package allocation information and the transportation plan are obtained from the pre-established logistics distribution system; By adopting the above technical solution, the master numbers and the package numbers of all boxes can be accurately pulled from the database of the warehouse management system according to the flight number, and the package allocation information and the transportation plan can be obtained from the logistics distribution system, which provides comprehensive and accurate flight-related information for subsequent sorting, thereby improving the efficiency and accuracy of logistics sorting.

[0009] Preferably, when receiving the first photoelectric trigger signal sent by the entrance photoelectric sensor, control all the industrial camera groups to collect images of the box passing through the entrance photoelectric sensor to obtain first collected images, and record the first photoelectric trigger information, specifically including the following steps: the entrance photoelectric sensor emits a first photoelectric trigger signal when the entrance photoelectric sensor is blocked by the passing container; When the first photoelectric trigger signal is received, a temporary container number of the container is generated, and a first-in-first-out queue is maintained, the temporary container number is stored in the first-in-first-out queue, and first photoelectric trigger information including the temporary container number and an entering time stamp of the current container is recorded; According to the first photoelectric trigger signal, the industrial camera group continuously performs image acquisition on each face of the container in the scanning area to obtain a plurality of first acquisition images; The temporary container number is bound to the first acquisition images.

[0010] By adopting the above technical scheme, when the entrance photoelectric sensor emits a first photoelectric trigger signal when the entrance photoelectric sensor is blocked by the passing container, the signal can be received in time, the temporary container number of the container is generated, and the first-in-first-out queue is maintained, the number is stored in the queue, and the first photoelectric trigger information including the entering time stamp is recorded. According to the signal, the industrial camera group continuously acquires images of each face of the container to obtain a plurality of first acquisition images, and the temporary container number is bound to the images, which realizes accurate recording of container information and comprehensive acquisition of images, provides accurate and complete data basis for subsequent barcode decoding, OCR recognition, information fusion matching and sorting operation, and improves the accuracy and efficiency of intelligent logistics sorting.

[0011] Preferably, a spatial coordinate system is established in the scanning area, and when the image acquisition is performed, the industrial camera group in the scanning area continuously performs high-speed photographing on the six faces of the container; the industrial camera group includes industrial cameras located in the positive direction of the X-axis, the negative direction of the X-axis, the positive direction of the Y-axis, the negative direction of the Y-axis, the positive direction of the Z-axis and the negative direction of the Z-axis of the spatial coordinate system, wherein the industrial camera located in the negative direction of the Z-axis is a bottom scanning camera, the bottom scanning camera is located below the conveying belt conveying the container, a reflector is arranged in the field of view of the bottom scanning camera, and the reflector faces the gap between the two conveying belts.

[0012] By adopting the above technical scheme, the spatial coordinate system is established in the scanning area, so that the industrial camera group can continuously and high-speed photograph the six faces of the container, and the bottom scanning camera located below the conveying belt and having the reflector in the field of view facing the gap between the conveying belts can comprehensively acquire image information of each face of the container, improve the comprehensiveness of information acquisition, and further improve the accuracy of flight information matching.

[0013] Preferably, the barcode decoding and OCR identification based on the first acquisition image are performed, the obtained barcode decoding information and OCR identification information are matched with the pulled flight information, and the corresponding main number and large package number are obtained, specifically including the following steps: According to the first acquisition image, it is judged whether the box is one when entering the scanning area, if yes, the box face sheet of the current box is obtained based on the first acquisition image; The barcode decoding information is obtained by performing barcode decoding on the box face sheet, and the OCR identification information is obtained by performing OCR identification on the plurality of first acquisition images, the barcode decoding information includes the main number, the large package number and the flight number, and the OCR identification information includes the address of sending and receiving goods and the description of goods; The semantic feature is obtained by performing entity extraction on the OCR identification information based on the preset BERT model, the semantic similarity between the semantic feature and the preset semantic field of the flight information is calculated, and the semantic similarity exceeding the semantic matching threshold is screened and summarized as a candidate matching set; The structured character data of each flight information is extracted from the candidate matching set, the edit distance between the structured character data and the corresponding character data in the barcode decoding information is calculated by using the edit distance matching algorithm, and the flight information with the smallest edit distance and less than the edit distance threshold is selected as the final matching result; If there are multiple pieces of data with the same edit distance, secondary sorting is performed in combination with the semantic similarity, and the record with the highest semantic similarity is selected as the final matching result.

[0014] By using the above technical solutions, it can be judged whether the box in the first acquisition image is one, ensuring that the subsequent processing is for a single box and avoiding interference of multiple boxes; through face sheet extraction, barcode decoding and OCR identification, key information such as box barcode, main number, large package number, flight number, address of sending and receiving goods and description of goods can be obtained; by using the BERT model to perform entity extraction and semantic similarity calculation on the OCR identification information, in combination with the edit distance matching algorithm, flight information with high matching degree with the barcode decoding information can be screened out, even if there are multiple pieces of data with the same edit distance, the final matching result can also be obtained through secondary sorting in combination with the semantic similarity, the accuracy of flight information matching is improved, and effective fusion and matching of box information and flight information are realized.

[0015] Preferably, when the second photoelectric trigger information recorded by the exit photoelectric sensor is received, photoelectric verification is performed in combination with the second photoelectric trigger information, specifically including the following steps: receiving a second photoelectric trigger signal emitted by the exit photoelectric sensor and generating second photoelectric trigger information when the exit photoelectric sensor is blocked by the passing box, the second photoelectric trigger information including an egress timestamp of the current box; controlling the industrial camera group to perform image acquisition on the current box, to obtain a second acquisition image; taking out the temporary box number stored first in the current queue from the head of the first-in-first-out queue, and searching and obtaining the main single number and the large package number bound to the temporary box number according to the taken-out temporary box number, as the sorting information to be output; performing similarity comparison on the first acquisition image and the second acquisition image, and calculating an image matching score; calculating a time difference between the egress timestamp in the second photoelectric trigger information and the entry timestamp in the first photoelectric trigger information, and determining whether the time difference is within a preset reasonable time threshold range; If the image matching score exceeds a preset matching confidence score, and the time difference ΔT is within the reasonable time threshold range, the photoelectric verification passes, and the sorting information to be output is output to the logistics distribution system; otherwise, arbitration verification is performed, and if the arbitration verification fails, a backflow operation of the box is triggered.

[0016] By using the above technical solutions, the similarity of the images captured by the entrance and exit photoelectric sensors, and the time difference calculation and verification between the second photoelectric trigger information and the first photoelectric trigger information can effectively determine the state of the box, realize photoelectric verification, perform arbitration verification when the verification fails, and trigger the backflow operation when necessary, thereby improving the accuracy and reliability of the box sorting and reducing the occurrence of error sorting.

[0017] Preferably, after continuously performing image acquisition on each side of the box to obtain a plurality of first acquisition images, the method further comprises the following steps: real-time analyzing the depth information of the plurality of first acquisition images acquired by the industrial camera group, and when detecting that the distance between a plurality of boxes continuously is lower than a preset distance threshold, determining that a collision event occurs between the plurality of boxes, recording the plurality of boxes as a collision box group and recording a collision timestamp; obtaining the temporary box numbers of all boxes in the collision box group, denoted as a collision number set; For each box in the collision number set, a first position fingerprint is generated by identifying and extracting the face on which the box face single is located and the two-dimensional coordinates on the current face from the first acquisition image corresponding to the box. extracting the latest frame of the first acquisition image after the collision timestamp and before the case group leaves the scanning area, identifying and extracting the current face of the case face sheet on which each case is located and the two-dimensional coordinates on the current face, and generating a second location fingerprint; If the case face sheet of all cases is successfully identified and the second location fingerprint has a unique matching relationship with the first location fingerprint, the collision number set in the first-in-first-out queue is maintained, and the first-in-first-out queue is updated according to the matching result; If the case face sheet cannot be identified or the matching relationship is not unique, the collision number set is removed from the first-in-first-out queue, and a backflow operation of the case group is triggered.

[0018] By using the above technical solution, the depth information of the industrial camera group acquisition image is analyzed in real time, the case collision event can be determined in time and the related information can be recorded; the first location fingerprint and the second location fingerprint are generated, the collision whether causes the case identity to be lost can be accurately judged; according to the judgment result, the queue order is maintained or the backflow operation is triggered, the accuracy of the case information and the reliability of the sorting process are guaranteed, the sorting error caused by the case collision identity loss is reduced, and the accuracy and efficiency of the logistics sorting are improved.

[0019] In a second aspect, the present application provides an intelligent logistics sorting device, which adopts the following technical solution: An information pulling module is configured to pull flight information including master serial numbers and large package numbers of all cases from a pre-established warehouse management system; An image acquisition module is configured to control all the industrial camera groups to acquire images of the cases passing through the entrance photoelectric sensor to obtain first acquisition images and record first photoelectric trigger information when receiving a first photoelectric trigger signal sent by the entrance photoelectric sensor; An information fusion matching module is configured to perform barcode decoding and OCR identification based on the first acquisition images, fuse and match the obtained barcode decoding information, OCR identification information and the pulled flight information, and obtain corresponding master serial numbers and large package numbers; A photoelectric verification module is configured to perform photoelectric verification in combination with second photoelectric trigger information recorded by the exit photoelectric sensor when receiving the second photoelectric trigger information, and output the master serial numbers and the large package numbers for sorting when the photoelectric verification is passed.

[0020] In a third aspect, the present application provides a terminal, which adopts the following technical solution: A terminal comprises a memory and a processor, the memory stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to realize the intelligent logistics sorting method as described above.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium, the readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to realize the intelligent logistics sorting method as described above.

[0022] In summary, the present application includes at least one of the following beneficial technical effects: (1) The present application can automatically pull flight information from the warehouse management system, and fuse and match the information obtained by barcode decoding and OCR recognition to obtain key information such as barcodes and OCR text on the box at one time, and realize comprehensive information acquisition; (2) The present application reduces manual intervention through an automatic process, and cooperates with the entrance and exit photoelectric sensors to perform image acquisition, counting, verification and other operations, thereby improving the sorting efficiency; (3) The present application improves the accuracy of flight information matching through information fusion matching technology and multiple verification mechanisms such as photoelectric verification. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of an intelligent logistics sorting method; Figure 2 is a structural diagram of an intelligent logistics sorting device. DETAILED DESCRIPTION

[0024] The present application provides an intelligent logistics sorting method, device, terminal and storage medium, in order to make the purpose, technical scheme and advantages of the present application more clear, the following will be further detailed description of the embodiments of the present application.

[0025] The embodiments of the intelligent logistics sorting method of the present application are further described in detail below in combination with the drawings of the specification.

[0026] The intelligent logistics sorting method of the present embodiment is based on a multi-scan system, which comprises a plurality of industrial camera groups arranged in a predetermined scanning area P2, and the entrance and exit of the scanning area P2 are respectively provided with an entrance photoelectric sensor and an exit photoelectric sensor.

[0027] A spatial coordinate system is established in the scanning area. During image acquisition, the industrial camera group in the scanning area continuously takes high-speed photos of the six sides of the cargo box.

[0028] The industrial camera group includes industrial cameras located in the positive X-axis direction, negative X-axis direction, positive Y-axis direction, negative Y-axis direction, positive Z-axis direction, and negative Z-axis direction of the spatial coordinate system.

[0029] The industrial camera located in the negative Z-axis direction is a bottom-scan camera. It is positioned below the conveyor belt of the transport container, and a reflector is positioned within its field of view, facing the gap between the two conveyor belts. Specifically, the camera in the positive Z-axis direction photographs the top surface of the container vertically downwards.

[0030] A camera positioned vertically upwards along the negative Z-axis captures images of the bottom of the cargo container. Because the conveyor belt obstructs the view, this camera is a bottom-scan camera; it is mounted below the conveyor belt and observes the bottom of the container through a reflector facing the gap between the two belts.

[0031] The cameras, positioned horizontally in the positive and negative X-axis directions, photograph the two sides of the cargo box.

[0032] The camera on the positive Y-axis is positioned at the end of the aisle, facing the entrance, to photograph the front of the cargo container.

[0033] The camera on the negative Y-axis is positioned at the beginning of the passageway, facing the exit, to photograph the back of the cargo container.

[0034] All cameras are equipped with bar lights and diffusers to ensure that images with uniform lighting and no overexposure of highlights are obtained under any ambient light conditions.

[0035] This setup ensures comprehensive image capture of all six sides of the cargo box.

[0036] like Figure 1 As shown, the method of this application includes the following steps: S1. Retrieve flight information from the pre-established warehouse management system. The flight information includes the master manifest number and package number of all containers. The specific steps are as follows: S11. Retrieve the corresponding flight information from the pre-established warehouse management system database based on the flight number. The flight information includes the master bill of lading number and the package number of all cargo containers. Each flight includes several master bill of lading numbers, and each master bill of lading number includes several package numbers. Each cargo container has a unique corresponding package number.

[0037] The warehouse management system can be a common WMS system, storing cargo container information related to flights. The database used can be a relational database, such as MySQL or Oracle, or a non-relational database, such as MongoDB.

[0038] S12, according to the flight number from the pre-established logistics distribution system to obtain large package distribution information and transportation plan, the logistics distribution system manages the distribution of large packages.

[0039] The information pulling of the embodiment is a timing task, and incremental data synchronization is performed every minute. First, the flight information table (flight_info) of the database of the WMS system is queried to pull all records with a planned takeoff time within the next 2 hours. Each flight record contains a flight number (flight_number), a master waybill number (master_waybill_number), and its status. For each master waybill number, the associated package information table (packages) is further queried to obtain all house waybill numbers (house_waybill_number) associated with the master waybill number.

[0040] The embodiment adopts a double verification mechanism to ensure data consistency: first, the master waybill number format is checked, and the standard format is, for example, 2 letters + 10 digits, and second, it is verified whether each house waybill number exists in the TMS loading manifest. If it is found that the WMS and TMS data are inconsistent, for example, a certain house waybill number does not exist in the TMS, the record will be marked as to be confirmed, and will not be added to the matching library at the moment, and an alarm will be triggered to notify the administrator.

[0041] S2, when receiving the first photoelectric trigger signal sent by the entrance photoelectric sensor, control the whole industrial camera group to collect the image of the case passing through the entrance photoelectric sensor to obtain the first collection image, and record the first photoelectric trigger information. The entrance photoelectric sensor can adopt a reflection type photoelectric sensor, which emits a trigger signal when the case blocks the light. Specifically, the following steps are included: S21, when the entrance photoelectric sensor is blocked by the passing case, the entrance photoelectric sensor emits a first photoelectric trigger signal.

[0042] S22, when receiving the first photoelectric trigger signal, a temporary case number of the case is generated and a first-in-first-out queue is maintained, the temporary case number is stored in the first-in-first-out queue, and the first photoelectric trigger information is recorded, the first photoelectric trigger information including the temporary case number and the entering timestamp of the current case.

[0043] The system maintains a first-in-first-out (FIFO) queue, which is used to store all temporary case numbers in the order of the cases passing through the entrance. Whenever a temporary case number is generated for a case at the entrance photoelectric sensor, the temporary case number is immediately pushed to the tail of the queue.

[0044] S23, according to the first photoelectric trigger signal, control the industrial camera group of the scanning area to continuously collect images of each side of the case in the scanning area to obtain multiple first collection images; S24, bind the temporary box number with the first acquisition image.

[0045] In one specific implementation, when the entrance photoelectric sensor is triggered, the PLC sends a rising edge signal to the image server. A daemon process on the server listens for this signal and, once captured, performs the following operations: Generate a temporary box number, record the first photoelectric trigger information, including the temporary number and the entry timestamp accurate to milliseconds, and store it as a data object in the Redis cache.

[0046] Trigger all industrial camera groups for synchronous acquisition through the GPIO port or Gigabit network.

[0047] Each set of photos (six photos per set, one for each side) collected by the camera group is immediately uploaded to the file server, and the temporary box number is strongly associated with the URL path of the six picture files in a new record in the database.

[0048] S3, based on the first acquisition image, perform barcode decoding and OCR recognition, and fuse and match the obtained barcode decoding information, OCR recognition information and pulled flight information to obtain the corresponding master number and package number, specifically including the following steps: S31, determine whether the box is one based on the first acquisition image when entering the scanning area, if so, perform face sheet extraction based on the first acquisition image to obtain the current box face sheet.

[0049] In this embodiment, whether the box is one based on the first acquisition image when entering the scanning area is determined by determining whether the box in the image at the time corresponding to the entry timestamp of the box is one based on the first acquisition image, if not, all the boxes currently entering the scanning area are returned.

[0050] S32, barcode decoding of the box face sheet to obtain barcode decoding information, and OCR recognition of multiple first acquisition images to obtain OCR recognition information, the barcode decoding information including the master number, the package number and the flight number, and the OCR recognition information including the sender and receiver addresses and the item description.

[0051] S33, based on the pre-set BERT model, perform entity extraction on the OCR recognition information to obtain semantic features, calculate the semantic similarity between the semantic features and the pre-set semantic fields of the flight information, and filter the semantic similarity exceeding the semantic matching threshold to obtain a candidate matching set.

[0052] In this embodiment, the BERT model is a pre-trained language model used for effective semantic understanding and feature extraction of text.

[0053] From the full text recognized by OCR, use the pre-trained BERT model to extract key entities such as [DEST:PEK] (destination capital airport) and [MAWB:1234567890].

[0054] Calculate the semantic similarity of the destination field extracted by OCR and the destination code in the WMS flight information. For example, the destination field "Beijing Capital" and the destination code "PEK".

[0055] In this embodiment, all text is projected into the 768-dimensional vector space of BERT, and the cosine similarity is calculated. This embodiment sets the semantic matching threshold to 0.85, and selects all flight records with a similarity higher than this value to form a candidate matching set A.

[0056] S34, extract the structured character data of each flight information from the candidate matching set, and use the edit distance matching algorithm to calculate the edit distance between the corresponding character data in the barcode decoding information, and select the flight information with the smallest edit distance and less than the edit distance threshold as the final matching result.

[0057] The edit distance matching algorithm in this embodiment is the Levenshtein distance algorithm. Calculate the edit distance between the main single number and the large package number recognized by OCR and the corresponding number in WMS, and normalize the distance. Normalized distance = edit distance / field length.

[0058] The edit distance threshold of this embodiment is set to 0.2, and all records with a normalized distance lower than this value are selected to form a candidate matching set B.

[0059] Take the intersection of candidate sets A and B as the final candidate set. In general, there will be only one record in the final candidate set, which is the final matching result. If the intersection is empty or there are multiple records, take the union to form the final candidate set.

[0060] Calculate a comprehensive confidence score for each record in the final candidate set: comprehensive confidence score = 0.6 x semantic similarity + 0.4 x (1-normalized distance). Select the record with the highest score and greater than 0.75 to determine the final matching result.

[0061] S35, if there are multiple data with the same edit distance, then perform secondary sorting combined with semantic similarity, and select the record with the highest semantic similarity as the final matching result.

[0062] S4, in a specific implementable way, after continuously capturing multiple first capture images on each side of the cargo box, it further includes a collision detection and processing step, which specifically includes the following steps: S41, analyze the depth information of the plurality of first acquisition images collected by the industrial camera group in real time, when it is detected that the distance between the plurality of containers continuously is lower than the preset distance threshold, it is determined that a collision event occurs between the plurality of containers, the plurality of containers are recorded as a collision container group and a collision timestamp is recorded.

[0063] The depth information analysis of the embodiment is realized by binocular vision technology, which can accurately determine the distance between the containers.

[0064] When the containers travel in the scanning area, the system analyzes the depth map point cloud data generated by the camera in real time, determines the collision by calculating the minimum distance between adjacent containers, if the distance is less than 5cm for 5 consecutive frames (100ms), it is determined that a collision event occurs, the numbers of the two containers involved are marked, and they are recorded in the collision event record table.

[0065] S42, obtain the temporary container numbers of all containers in the collision container group, denoted as a collision number set.

[0066] S43, for each container in the collision number set, identify and extract the face and two-dimensional coordinates on the current face of the container face sheet from the first acquisition image corresponding to the container, to generate a first position fingerprint; S44, extract the latest frame of the first acquisition image after the collision timestamp and before the container group leaves the scanning area, identify and extract the face and two-dimensional coordinates on the current face of the container face sheet of each container, to generate a second position fingerprint; S45, if the container face sheet of all containers is successfully identified and the second position fingerprint has a unique matching relationship with the first position fingerprint, it is determined that the collision does not cause identity loss, the collision number set in the first-in-first-out queue is maintained, and the first-in-first-out queue is updated according to the matching result. S46, if there is a container face sheet that cannot be identified or the matching relationship is not unique, remove the collision number set from the first-in-first-out queue, and trigger the backflow operation of the container group.

[0067] The implementation principle of the embodiment is: a collision detection and processing step is added, which further improves the stability and reliability of the logistics sorting system. It can timely find the collision between the containers, and judge whether the identity loss is caused according to the position information of the container face sheet after the collision. The backflow operation is performed on the container group with identity loss, which avoids the occurrence of error sorting.

[0068] In a specific implementable way, if the temporary number of the container is marked as involving collision, assuming that the numbers of the two containers involved in the collision are M456 and N123, start the recovery program: A: The first location fingerprint of the face sheet of each container in the collision box group is generated when the container is taken out of the buffer. For example: M456: ('top', 0.2, 0.8); N123: ('side', 0.5, 0.5).

[0069] B: From the continuously collected video stream, the latest clear image after the collision occurs and before the exit trigger is extracted.

[0070] C: Try to identify the face sheet of each container in the frame image and generate the second location fingerprint. If successful, for example, the face sheet of the current container is found to be at ('side', 0.5, 0.5), it is matched with the first fingerprint, and the true identity of the current container is confirmed to be N123.

[0071] D: If the match is successful, update the queue order according to the location of the current container, and exchange the temporary number of the container closer to the exit sensor to the front of the queue.

[0072] E: If the match fails, delete the collision number set of the collision box group involved from the collision event record table, and force all the collision box group to return. The reasons for the match failure include not finding a unique matching relationship or face sheet obstruction.

[0073] The judgment condition for face sheet obstruction causing match failure is that when the visible area or visibility of the face sheet area in all perspectives is less than the preset visibility threshold, the return is directly triggered; Wherein, the visibility=(current face sheet pixel number) / (prestored face sheet reference pixel number).

[0074] S5, when receiving the second photoelectric trigger information recorded by the exit photoelectric sensor, combining the second photoelectric trigger information for photoelectric verification, when the photoelectric verification passes, outputting the main single number and the big bag number for sorting, specifically including the following steps: S51, the exit photoelectric sensor also uses a through-beam photoelectric sensor, when the exit photoelectric sensor is blocked by the passing container, receiving the second photoelectric trigger signal emitted by the exit photoelectric sensor and generating the second photoelectric trigger information, the second photoelectric trigger information including the moving-out timestamp of the current container.

[0075] S52, controlling the industrial camera group to collect images of the current container to obtain the second collection image; S53, taking out the temporary container number stored first in the current queue from the head of the first-in-first-out queue, and searching and obtaining the main single number and the big bag number bound to the temporary container number according to the taken-out temporary container number, as the output sorting information.

[0076] When the exit photoelectric sensor is triggered, the system takes the oldest temporary bin number from the head of the first-in-first-out queue. This number is considered as the temporary bin number of the current exiting bin.

[0077] S54, similarity comparison is performed on the first and second acquisition images to calculate an image matching score.

[0078] In one specific embodiment, ORB feature points and their feature descriptors in the first and second acquisition images are extracted. Feature point matching is performed using a K-nearest neighbor algorithm, and a ratio test is applied to filter out high-quality matching pairs. The image matching score is the ratio of the number of high-quality matching pairs to the total number of feature points.

[0079] S55, the time difference between the removal timestamp in the second photoelectric trigger information and the entry timestamp in the first photoelectric trigger information is calculated, and it is determined whether the time difference is within a preset reasonable time threshold range.

[0080] The time difference ΔT = T2 - T1 is calculated, and a reasonable time threshold range is set according to the conveyor belt speed and the scanning area length. For example, if the conveyor belt speed is 0.5 m / s and the scanning area length is 2 m, the theoretical passing time is 4 seconds, and a reasonable time threshold range of [3.5, 5] seconds can be set.

[0081] S56, if the image matching score exceeds the preset matching confidence value and the time difference ΔT is within the reasonable time threshold range, the photoelectric verification passes, and the output sorting information is output to the logistics distribution system; otherwise, arbitration verification is performed, and if the arbitration verification fails, the reflow operation of the bin is triggered.

[0082] In one specific embodiment, if the image matching score is lower than the preset matching confidence value, arbitration verification is performed, and a backup matching strategy is started to extract visual features of the bin non-face sheet area for secondary matching. The current second acquisition image is matched with the first acquisition images bound to the subsequent multiple temporary bin numbers in the first-in-first-out queue. If the image matching with the Nth number in the queue is successful, it is inferred that the bins from the head of the queue to the N-1th number are the stagnant bins, and the current exit bin is the Nth bin. The corresponding sorting information is output and the first-in-first-out queue is updated. If it is still not matched, the arbitration verification fails, and the corresponding bin is triggered to reflow.

[0083] In another specific embodiment, if ΔT exceeds the preset time threshold range, multiple bin collisions may occur, and arbitration verification is performed: The collision event record table is queried to determine whether the current temporary bin number is recorded in the current collision event record table. If the temporary bin number is recorded in the collision event, and the time difference between the collision timestamp and the removal timestamp does not exceed the preset collision time tolerance value, the arbitration check passes, it is determined that the photoelectricity check passes, and the updated sorting information is output; otherwise, the corresponding bin backflow is triggered.

[0084] If the temporary bin number is not recorded in the collision event or the time difference between the collision timestamp and the removal timestamp exceeds the preset collision time tolerance value, the arbitration check does not pass, it is finally determined that the photoelectricity check fails, and the backflow operation is triggered.

[0085] In a specific implementable manner, after the bin passes through the outlet photoelectric sensor, it passes through the camera for obtaining the volume of the bin. The camera summarizes the length, width and height information of the bin into the sorting information of the bin, facilitating the subsequent palletizing process.

[0086] S57, in combination with the large package number, the main order number in the sorting information of the current bin, and the previously obtained large package allocation information and the transportation plan, the transportation allocation is performed for the bin that successfully passes the check.

[0087] Through the multi-scan system, the bar code information and the OCR information of the bin are obtained, and the information is fused and matched with the pulled flight information, and the photoelectricity check is performed, which greatly improves the accuracy and efficiency of the logistics sorting. The high cost and low efficiency problem caused by manual intervention is avoided, and the problems of insufficient information fusion and low automation degree of the existing system are solved.

[0088] Based on the same inventive concept as described above, the embodiments of the present application also disclose an intelligent logistics sorting device, as shown in Figure 2 The device comprises the following modules: An information pulling module is configured to pull flight information from a pre-established warehouse management system, and the flight information comprises main order numbers and large package numbers of all bins; An image acquisition module is configured to, when receiving a first photoelectric trigger signal sent by an entrance photoelectric sensor, control all industrial camera groups to acquire images of the bins passing through the entrance photoelectric sensor to obtain first acquisition images, and record first photoelectric trigger information; An information fusion and matching module is configured to perform bar code decoding and OCR recognition based on the first acquisition images, fuse and match the obtained bar code decoding information and OCR recognition information with the pulled flight information, and obtain corresponding main order numbers and large package numbers; A photoelectricity check module is configured to, when receiving second photoelectric trigger information recorded by an exit photoelectric sensor, perform photoelectricity check in combination with the second photoelectric trigger information, and output the main order numbers and the large package numbers for sorting when the photoelectricity check passes.

[0089] In one specific implementation, the information pulling module comprises the following units: a first information pulling unit for pulling corresponding flight information from a pre-established database of the warehouse management system according to the flight number, the flight information including master order numbers and package numbers of all the cargo boxes, each flight including a plurality of master order numbers, each master order number including a plurality of package numbers, and each cargo box having a unique corresponding package number; a second information pulling unit for obtaining package allocation information and transportation plans from a pre-established logistics distribution system according to the flight number.

[0090] In one specific implementation, the image acquisition module comprises the following units: a first image acquisition unit for emitting a first photoelectric trigger signal when the entrance photoelectric sensor is blocked by a passing cargo box; a second image acquisition unit for generating a temporary cargo box number of the cargo box and maintaining a first-in-first-out queue when the first photoelectric trigger signal is received, storing the temporary cargo box number in the first-in-first-out queue and recording first photoelectric trigger information, the first photoelectric trigger information including the temporary cargo box number and an entering timestamp of the current cargo box; a third image acquisition unit for controlling the industrial camera group to continuously acquire images of each side of the cargo box in the scanning area according to the first photoelectric trigger signal, to obtain a plurality of first acquisition images; binding the temporary cargo box number with the first acquisition images.

[0091] In one specific implementation, the information fusion matching module comprises the following units: a first information fusion matching unit for determining whether the cargo box is one when entering the scanning area according to the first acquisition images, and if so, extracting the cargo box face label of the current cargo box based on the first acquisition images; a second information fusion matching unit for decoding the barcode of the cargo box face label to obtain barcode decoding information, and performing OCR recognition on the plurality of first acquisition images to obtain OCR recognition information, the barcode decoding information including master order numbers, package numbers, and flight numbers, and the OCR recognition information including shipping and receiving addresses and item descriptions; a third information fusion matching unit for extracting entities from the OCR recognition information based on a pre-set BERT model to obtain semantic features, calculating semantic similarity of the semantic features with pre-set semantic fields of the flight information, and screening semantic similarity exceeding a semantic matching threshold and summarizing as a candidate matching set; The fourth information fusion matching unit is configured to extract structured character data of each flight information from the candidate matching set, calculate an edit distance between the structured character data and corresponding character data in the barcode decoding information by using an edit distance matching algorithm, and select flight information with the smallest edit distance and less than an edit distance threshold as a final matching result. The fifth information fusion matching unit is configured to, if there are multiple pieces of data with the same edit distance, perform secondary sorting in combination with semantic similarity, and select a record with the highest semantic similarity as the final matching result.

[0092] In one specific implementation, the photoelectric verification module comprises the following units: The first photoelectric verification unit is configured to receive a second photoelectric trigger signal sent by the outlet photoelectric sensor and generate second photoelectric trigger information when the outlet photoelectric sensor is blocked by a passing container, the second photoelectric trigger information comprising an egress timestamp of the current container; The second photoelectric verification unit is configured to control the industrial camera group to perform image acquisition on the current container to obtain a second acquisition image, take out a temporary container number stored first in the current queue from the head of the first-in-first-out queue, and search for and obtain a master number and a large package number bound to the temporary container number according to the taken-out temporary container number, as the output sorting information to be output; The third photoelectric verification unit is configured to perform similarity comparison on the first acquisition image and the second acquisition image, calculate an image matching score, and calculate a time difference between the egress timestamp in the second photoelectric trigger information and the entry timestamp in the first photoelectric trigger information, to determine whether the time difference is within a preset reasonable time threshold range; The fourth photoelectric verification unit is configured to, if the image matching score exceeds a preset matching confidence value and the time difference ΔT is within the reasonable time threshold range, determine that the photoelectric verification is passed, and output the output sorting information to be output to the logistics distribution system; otherwise, perform arbitration verification, and if the arbitration verification is not passed, trigger a backflow operation of the container.

[0093] In one specific implementation, the device further comprises a collision detection module comprising the following units: The first collision detection unit is configured to analyze depth information of a plurality of first acquisition images collected by the industrial camera group in real time, determine that a collision event occurs between a plurality of containers when it is detected that a distance between the plurality of containers is continuously lower than a preset distance threshold, record the plurality of containers as a collision container group and record a collision timestamp, and obtain temporary container numbers of all containers in the collision container group, as a collision number set; The second collision detection unit is configured to, for each container in the collision number set, identify and extract a face on which a container face label is located and a two-dimensional coordinate on the current face from the first acquisition image corresponding to the container, and generate a first position fingerprint; A third collision detection unit is configured to extract the latest frame of the first acquisition image after the collision timestamp and before the box group leaves the scanning area, identify and extract the current box face of each box and the two-dimensional coordinates on the current face, and generate a second location fingerprint; A fourth collision detection unit is configured to, if the box face of all the boxes is successfully identified and the second location fingerprint has a unique matching relationship with the first location fingerprint, maintain the collision number set in the first-in-first-out queue and update the first-in-first-out queue according to the matching result; if the box face cannot be identified or the matching relationship is not unique, remove the collision number set from the first-in-first-out queue and trigger a backflow operation of the box group.

[0094] Based on the same inventive concept, the embodiments of the present application further disclose a terminal, comprising a memory and a processor, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to realize the intelligent logistics sorting method as described above.

[0095] Also based on the same inventive concept, the embodiments of the present application further disclose a computer readable storage medium, the readable storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, at least one program, code set or instruction set being loaded and executed by the processor to realize the intelligent logistics sorting method as described above.

[0096] The method steps described in the present application do not necessarily have to be strictly implemented in the order indicated by the reference numerals. The order of the steps can be adjusted, changed or executed in parallel according to actual needs. Unless the dependency between the steps is explicitly stated (for example, step S3 needs to be performed on the basis of the execution result of step S2), the serial number should not be interpreted as a strict limitation on the order.

[0097] Those of ordinary skill in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by programs instructing related hardware. The above programs can be stored in a computer readable storage medium, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0098] The above are preferred embodiments of the present application, which do not limit the protection scope of the present application, therefore: all equivalent changes made on the basis of the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. An intelligent logistics sorting method, characterized in that, The method based on a multi-scan system including a plurality of industrial camera groups arranged in a preset scanning area, and an entrance photoelectric sensor and an exit photoelectric sensor arranged at the entrance and the exit of the scanning area, comprises the following steps: Pull flight information from a pre-established warehouse management system, the flight information including master order numbers and large package numbers of all the cargo boxes; When receiving a first photoelectric trigger signal sent by the entrance photoelectric sensor, control all the industrial camera groups to collect images of the cargo box passing through the entrance photoelectric sensor to obtain first collection images, and record first photoelectric trigger information; Based on the first collection images, perform barcode decoding and OCR identification, fuse and match the obtained barcode decoding information, OCR identification information and the pulled flight information to obtain corresponding master order numbers and large package numbers; When receiving second photoelectric trigger information recorded by the exit photoelectric sensor, perform photoelectricity check combined with the second photoelectric trigger information, and output the master order numbers and the large package numbers for sorting when the photoelectricity check is passed.

2. The intelligent logistics sorting method according to claim 1, characterized in that, The step of pulling flight information from a pre-established warehouse management system, the flight information including master order numbers and large package numbers of all the cargo boxes, specifically comprises the following steps: Pull corresponding flight information from a database of a pre-established warehouse management system according to a flight number, the flight information including master order numbers and large package numbers of all the cargo boxes, each flight including a plurality of master order numbers, each master order number including a plurality of large package numbers, and each cargo box having a unique corresponding large package number; Obtain large package allocation information and transportation plans from a pre-established logistics allocation system according to the flight number.

3. The intelligent logistics sorting method according to claim 2, characterized in that, The step of, when receiving a first photoelectric trigger signal sent by the entrance photoelectric sensor, controlling all the industrial camera groups to collect images of the cargo box passing through the entrance photoelectric sensor to obtain first collection images, and recording first photoelectric trigger information, specifically comprises the following steps: When the entrance photoelectric sensor is blocked by the passing cargo box, the entrance photoelectric sensor sends a first photoelectric trigger signal; When receiving the first photoelectric trigger signal, generate a temporary cargo box number of the cargo box and maintain a first-in-first-out queue, store the temporary cargo box number in the first-in-first-out queue and record first photoelectric trigger information, the first photoelectric trigger information including the temporary cargo box number and an entering time stamp of the current cargo box; Control the industrial camera groups to continuously collect images of each side of the cargo box in the scanning area according to the first photoelectric trigger signal to obtain a plurality of first collection images; Bind the temporary cargo box number with the first collection images. 4.The intelligent logistics sorting method according to claim 1, characterized in that: A spatial coordinate system is established in the scanning area, and when the image is collected, the industrial camera group in the scanning area continuously takes high-speed photos of the six faces of the box; the industrial camera group includes industrial cameras located in the positive direction of the X-axis, the negative direction of the X-axis, the positive direction of the Y-axis, the negative direction of the Y-axis, the positive direction of the Z-axis and the negative direction of the Z-axis of the spatial coordinate system, wherein the industrial camera located in the negative direction of the Z-axis is a bottom scanning camera, the bottom scanning camera is located below the conveying belt conveying the box, a mirror is arranged in the field of view of the bottom scanning camera, and the mirror faces the gap between the two conveying belts. 5.The intelligent logistics sorting method according to claim 3, characterized in that, The barcode decoding and OCR identification based on the first collected image are performed, the obtained barcode decoding information and OCR identification information are fused and matched with the pulled flight information, and the corresponding master number and large package number are obtained, specifically including the following steps: It is judged according to the first collected image whether the box is one when entering the scanning area, if yes, the face sheet of the current box is obtained based on the first collected image; The barcode decoding information includes the master number, the large package number and the flight number, and the OCR identification information includes the address of sending and receiving goods and the description of goods; The semantic features are obtained by performing entity extraction on the OCR identification information based on a preset BERT model, the semantic similarity between the semantic features and a preset semantic field of the flight information is calculated, and the semantic similarity exceeding a semantic matching threshold is screened and summarized as a candidate matching set; If there are multiple pieces of data with the same edit distance, secondary sorting is performed in combination with the semantic similarity, and the record with the highest semantic similarity is selected as the final matching result. When the second photoelectric trigger information recorded by the exit photoelectric sensor is received, photoelectric verification is performed in combination with the second photoelectric trigger information, specifically including the following steps: 6.The intelligent logistics sorting method according to claim 3, characterized in that, When the exit photoelectric sensor is blocked by the box passing through, the second photoelectric trigger signal emitted by the exit photoelectric sensor is received and second photoelectric trigger information is generated, the second photoelectric trigger information includes the moving-out timestamp of the current box; The industrial camera group is controlled to collect images of the current box and obtain second collected images; The temporary box number first stored in the current queue is taken out from the head of the first-in-first-out queue, and the master number and the large package number bound to the temporary box number are found and obtained according to the taken-out temporary box number, as the output sorting information to be output; The first collected image and the second collected image are compared in similarity, and an image matching score is calculated; ​ calculating a time difference between the moving-out timestamp in the second photoelectric trigger information and the moving-in timestamp in the first photoelectric trigger information, and determining whether the time difference is within a preset reasonable time threshold range; if the image matching score exceeds a preset matching score threshold and the time difference ΔT is within the reasonable time threshold range, the photoelectric verification is passed, and the to-be-output sorting information is output to the logistics distribution system; otherwise, arbitration verification is performed, and if the arbitration verification fails, a backflow operation of the container is triggered. 7.The intelligent logistics sorting method according to claim 5, characterized in that, After continuously performing image acquisition on each face of the container to obtain a plurality of first acquisition images, the following steps are further included: real-time analysis of depth information of the plurality of first acquisition images acquired by the industrial camera group, when it is detected that the distance between a plurality of containers continuously falls below a preset distance threshold, it is determined that a collision event occurs between the plurality of containers, the plurality of containers are recorded as a collision container group and a collision timestamp is recorded; obtaining the temporary container numbers of all containers in the collision container group, denoted as a collision number set; for each container in the collision number set, identifying and extracting the face and two-dimensional coordinates on the current face of the container face sheet from the first acquisition image corresponding to the container, to generate a first location fingerprint; extracting the latest first acquisition image after the collision timestamp and before the container group leaves the scanning area, identifying and extracting the face and two-dimensional coordinates on the current face of the container face sheet of each container, to generate a second location fingerprint; if the container face sheet of all containers is successfully identified and the second location fingerprint has a unique matching relationship with the first location fingerprint, the collision number set is maintained in the first-in-first-out queue, and the first-in-first-out queue is updated according to the matching result; if the container face sheet cannot be identified or the matching relationship is not unique, the collision number set is removed from the first-in-first-out queue, and a backflow operation of the container group is triggered.

8. An intelligent logistics sorting device, characterized in that, The following modules are included: an information pulling module for pulling flight information from a pre-established warehouse management system, the flight information including master numbers and package numbers of all containers; an image acquisition module for controlling all industrial camera groups to perform image acquisition on the containers passing through the entrance photoelectric sensor to obtain first acquisition images and record first photoelectric trigger information when receiving a first photoelectric trigger signal from the entrance photoelectric sensor; an information fusion and matching module for performing barcode decoding and OCR identification based on the first acquisition images, fusing and matching the obtained barcode decoding information, OCR identification information, and the pulled flight information to obtain corresponding master numbers and package numbers; a photoelectric verification module for performing photoelectric verification in combination with second photoelectric trigger information recorded by an exit photoelectric sensor, and outputting the master numbers and package numbers for sorting when the photoelectric verification is passed.

9. A terminal, characterized by comprising: The application also discloses a computer readable storage medium, which comprises at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to realize the intelligent logistics sorting method.

10. A computer-readable storage medium, characterized in that, The application also discloses a computer readable storage medium, which comprises at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to realize the intelligent logistics sorting method.

Images