A compliance intelligent identification method based on express package signing position photos
By combining the YOLO architecture with the Qwen2.5-VL multimodal large model, the problem of automated compliance review in the last-mile delivery signing process of express logistics is solved. This enables efficient and accurate compliance identification of express parcel location photos, improves system transparency and adaptability, and forms a closed-loop feedback mechanism.
Patent Information
- Application Number
- CN202610036293.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-13
- Publication Date
- 2026-06-09
Smart Images
Figure CN122175500A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent express logistics technology, specifically to a compliance intelligent identification method based on a photo of the location of a signed package. Background Technology
[0002] In the last mile of the express delivery and logistics industry, the signing process is crucial for service loop and quality control. Currently, the common practice is to manually take photos of the packages for signing, which are then reviewed by the back-end system to confirm whether the packages have been placed in compliant locations. However, with the surge in business volume, the manual review model faces problems such as low efficiency, high cost, inconsistent standards, strong subjectivity, and poor real-time performance, making it difficult to meet the needs of modern logistics. Existing automated solutions often rely on traditional image processing, single-object detection, or simple classification rules, which are poorly adaptable to complex and ever-changing real-world shooting environments. They cannot understand the deep semantics required for location compliance, lack the ability to identify undefined violation scenarios, and mostly rely on single visual modal analysis, failing to combine contextual information for comprehensive reasoning. Summary of the Invention
[0003] To address the aforementioned technical challenges, this paper presents a compliance-based intelligent identification method using photos of express parcel delivery locations. This solution addresses the urgent need for an intelligent identification method that deeply integrates visual perception and semantic understanding, possesses high accuracy, robustness, and good generalization ability, in order to achieve efficient and accurate automated compliance review of express parcel delivery location photos.
[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A compliance-based intelligent identification method for express parcel delivery location photos includes: S1. Receive the photo of the delivery location of the parcel to be reviewed. Utilize a lightweight object detection model based on the YOLO architecture to calculate the bounding box and confidence score of the photo of the delivery location of the parcel to be reviewed. Set a confidence threshold and detect whether there is a parcel in the photo of the delivery location of the parcel to be reviewed. S2. Based on the judgment that the location of the parcel to be inspected is compliant, design a structured system prompt word engineering, train the Qwen2.5-VL open source multimodal large model, and generate the result of the compliance judgment of the parcel placement location. S3. Based on the compliance determination result of the parcel placement location, the system analyzes the data and generates the final review conclusion. If the review conclusion is compliant, the review approval record is automatically completed. If the review conclusion is non-compliant, the reason for non-compliance is automatically fed back to the business front end, prompting the courier to take a photo of the parcel location correctly, thus realizing intelligent recognition of the compliance of the parcel signing location photo.
[0005] Preferably, step S1 specifically includes: Based on the back-end signing system for express parcels, several RGB images of the signing location of express parcels awaiting review are obtained. The minimum image size of the images of the signing location of express parcels awaiting review is greater than 200 pixels and the maximum image size is less than 4096 pixels, which are used as the image size standard. The images of the signing location of RGB express parcels awaiting review that meet the image size standard are then selected. Based on the RGB images of the unapproved express parcels' signature locations that meet the image size standards, the images are resized and uniformly scaled to a fixed image size of 640×640, and the pixel values are standardized.
[0006] Preferably, step S1 further includes: Using RGB photos of the signed location of parcels awaiting review as input, a lightweight object detection model based on the YOLO architecture is trained. The backbone network is used to extract image features of the RGB photos of the signed location of parcels awaiting review, and multi-scale fusion is performed through the neck network to enhance the model's ability to recognize objects of different image sizes. Based on the RGB image features of the delivery location photo of the parcel to be reviewed, the bounding box and category of the delivery location photo of the parcel to be reviewed are predicted by the decoupling head, and its confidence score is calculated.
[0007] Preferably, step S1 further includes: Set a confidence threshold of 0.85 to check whether the location photos of the signed packages to be reviewed are compliant; If the confidence score of the delivery location photo of the parcel to be reviewed is less than the threshold, the photo is deemed non-compliant, a non-compliant conclusion is output, a standardized prompt is generated indicating that no clearly visible parcel was detected in the photo, and the process ends. If the location photo of the parcel to be signed for is classified as a parcel and the confidence score is greater than or equal to the threshold, the photo is deemed compliant, the initial screening is completed, and the process proceeds to the next step.
[0008] Preferably, step S2 specifically includes: The system design includes a structured prompt word project, defining the role as a courier delivery compliance review expert. This expert identifies whether the core object in the delivery location photo of the parcel to be reviewed is a courier parcel. The criteria for identifiable location photos are clear location markers, distinctive background features, stable placement, and no severe obstruction. The system then determines the compliance of the parcel placement location and performs supervised fine-tuning of the Qwen2.5-VL open-source multimodal large model.
[0009] Preferably, step S2 further includes: Based on the compliance of the signature location photo of the parcel to be reviewed and the structured system prompt words as input, the signature location photo of the parcel to be reviewed is converted into a visual feature vector by a visual encoder, and the structured system prompt words are converted into word embedding vectors. The visual feature vector of the delivery location photo of the parcel to be reviewed is fused with the word embedding vector in a cross-modal manner to generate a unified multimodal representation of the delivery location photo of the parcel to be reviewed. Based on the unified multimodal representation of the delivery location of the parcel to be reviewed, the Qwen2.5-VL open-source multimodal model with minor adjustments is used for inference to generate a structured JSON text format for determining the compliance of the parcel placement location.
[0010] Preferably, step S3 specifically includes: Based on the results of the compliance assessment of the placement location of express parcels, and combined with the location photo identification standard, the system generates the final review conclusion. If the parsing is deemed compliant, the system automatically generates a final conclusion of approval and extracts the location features of the express package; if the parsing is deemed non-compliant, the system automatically generates a final conclusion of disapproval and points out the reason for non-compliance. The reasons for non-compliance include: missing location references, blurred environmental features, and failure to identify express parcels.
[0011] Preferably, step S3 further includes: Upon final approval, the system automatically records the status of the parcel receipt approval and extracts the timestamp, tracking number, and location feature text of the parcel, storing them in the approval log database. For cases where the review fails, the reason for non-compliance is pushed to the mobile APP and business front-end system of the courier corresponding to the parcel order in real time through the message push interface, prompting the courier to take a photo of the parcel's location correctly, thus realizing intelligent recognition of the compliance of the parcel's signature location photo.
[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes a compliance intelligent recognition scheme based on photos of express parcel signature locations. Its advantages include: a lightweight object detection model based on the YOLO architecture quickly filters out invalid photos without parcels, significantly reducing the need for computationally expensive large models and optimizing resources; it combines the accuracy of YOLO in object detection with the semantic understanding capabilities of multimodal large models for complex scenes, making it more robust to challenges such as lighting, angle, background complexity, and partial occlusion; the natural language explanations output by the Qwen2.5-VL open-source multimodal large model provide intuitive and user-friendly review criteria, facilitating manual review and customer service communication, and improving system transparency; by modifying or enriching the prompts, this framework can be easily extended to other compliance review scenarios without retraining the core visual model; through two-stage collaborative intelligent processing, it achieves a leap from seeing the parcel to understanding the scene, providing a practical and automated solution for the quality management of parcel signatures in the express logistics industry. Attached Figure Description
[0013] Figure 1 This is a flowchart of a compliance-based intelligent identification method for express parcels with signed locations. Detailed Implementation
[0014] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.
[0015] Reference Figure 1 As shown, a compliance intelligent identification method based on a photo of the delivery location of a courier package includes: S1. Receive the photo of the delivery location of the parcel to be reviewed. Utilize a lightweight object detection model based on the YOLO architecture to calculate the bounding box and confidence score of the photo of the delivery location of the parcel to be reviewed. Set a confidence threshold and detect whether there is a parcel in the photo of the delivery location of the parcel to be reviewed. Step S1 specifically includes: Based on the back-end signing system for express parcels, several RGB images of the signing location of express parcels awaiting review are obtained. The minimum image size of the images of the signing location of express parcels awaiting review is greater than 200 pixels and the maximum image size is less than 4096 pixels, which are used as the image size standard. The images of the signing location of RGB express parcels awaiting review that meet the image size standard are then selected. Based on the RGB images of the unapproved express parcels' signature locations that meet the image size standards, the images are resized and uniformly scaled to a fixed image size of 640×640, and the pixel values are standardized.
[0016] Step S1 also includes: Using RGB photos of the signed location of parcels awaiting review as input, a lightweight object detection model based on the YOLO architecture is trained. The backbone network is used to extract image features of the RGB photos of the signed location of parcels awaiting review, and multi-scale fusion is performed through the neck network to enhance the model's ability to recognize objects of different image sizes. Based on the RGB image features of the delivery location photo of the parcel to be reviewed, the bounding box and category of the delivery location photo of the parcel to be reviewed are predicted by the decoupling head, and its confidence score is calculated.
[0017] Step S1 also includes: Set a confidence threshold of 0.85 to check whether the location photos of the signed packages to be reviewed are compliant; If the confidence score of the delivery location photo of the parcel to be reviewed is less than the threshold, the photo is deemed non-compliant, a non-compliant conclusion is output, a standardized prompt is generated indicating that no clearly visible parcel was detected in the photo, and the process ends. If the location photo of the parcel to be signed for is classified as a parcel and the confidence score is greater than or equal to the threshold, the photo is deemed compliant, the initial screening is completed, and the process proceeds to the next step.
[0018] When using it, please refer to the steps outlined above: Existing technologies primarily rely on manual review of photos showing the location of parcel signatures, which suffers from inefficiency, strong subjectivity, and difficulty in standardization. Traditional object detection models often have high computational complexity and deployment costs, making it difficult to achieve efficient and accurate automatic detection in real-time review scenarios. This step adopts a lightweight object detection model based on the YOLO architecture, combined with preset image size filtering and standardized preprocessing, and sets strict confidence thresholds for automated judgment, improving detection speed and accuracy. It achieves rapid, objective, and standardized compliance review of parcel signature location photos, effectively reducing labor costs and avoiding disputes caused by blurry images or missing packages. The lightweight design facilitates integration into existing express delivery backend systems, enhancing the automation and reliability of the overall process.
[0019] S2. Based on the judgment that the location of the parcel to be inspected is compliant, design a structured system prompt word engineering, train the Qwen2.5-VL open source multimodal large model, and generate the result of the compliance judgment of the parcel placement location. Step S2 specifically includes: The system design includes a structured prompt word project, defining the role as a courier delivery compliance review expert. This expert identifies whether the core object in the delivery location photo of the parcel to be reviewed is a courier parcel. The criteria for identifiable location photos are clear location markers, distinctive background features, stable placement, and no severe obstruction. The system then determines the compliance of the parcel placement location and performs supervised fine-tuning of the Qwen2.5-VL open-source multimodal large model.
[0020] Step S2 also includes: Based on the compliance of the signature location photo of the parcel to be reviewed and the structured system prompt words as input, the signature location photo of the parcel to be reviewed is converted into a visual feature vector by a visual encoder, and the structured system prompt words are converted into word embedding vectors. The visual feature vector of the delivery location photo of the parcel to be reviewed is fused with the word embedding vector in a cross-modal manner to generate a unified multimodal representation of the delivery location photo of the parcel to be reviewed. Based on the unified multimodal representation of the delivery location of the parcel to be reviewed, the Qwen2.5-VL open-source multimodal model with minor adjustments is used for inference to generate a structured JSON text format for determining the compliance of the parcel placement location.
[0021] When using it, please refer to the steps outlined above: In existing technologies, compliance verification of express delivery receipts mainly relies on manual visual inspection or traditional image rule matching. This approach suffers from low efficiency, high cost, and significant subjective bias, and traditional visual methods are difficult to generalize to handle complex and ever-changing real-world scenarios. This step, through the design of structured prompt word engineering and multimodal supervised fine-tuning, combined with visual coding and cross-modal fusion technology, achieves automated and accurate compliance determination of express delivery package receipt location photos, improving verification efficiency and accuracy. Relying on unified multimodal representation and structured JSON output, it enhances the system's interpretability, integrability, and scenario adaptability.
[0022] S3. Based on the compliance judgment result of the parcel placement location, the system generates the final review conclusion. If the review conclusion is compliant, the review approval record is automatically completed. If the review conclusion is non-compliant, the reason for non-compliance is automatically fed back to the business front end, prompting the courier to take a photo of the parcel location correctly, so as to realize the intelligent recognition of the compliance of the parcel signing location photo. Step S3 specifically includes: Based on the results of the compliance assessment of the placement location of express parcels, and combined with the location photo identification standard, the system generates the final review conclusion. If the parsing is deemed compliant, the system automatically generates a final conclusion of approval and extracts the location features of the express package; if the parsing is deemed non-compliant, the system automatically generates a final conclusion of disapproval and points out the reason for non-compliance. The reasons for non-compliance include: missing location references, blurred environmental features, and failure to identify express parcels.
[0023] Step S3 also includes: Upon final approval, the system automatically records the status of the parcel receipt approval and extracts the timestamp, tracking number, and location feature text of the parcel, storing them in the approval log database. For cases where the review fails, the reason for non-compliance is pushed to the mobile APP and business front-end system of the courier corresponding to the parcel order in real time through the message push interface, prompting the courier to take a photo of the parcel's location correctly, thus realizing intelligent recognition of the compliance of the parcel's signature location photo.
[0024] When using it, please refer to the steps outlined above: In existing technologies, the compliance determination of parcel delivery location photos relies heavily on manual review or simple rule matching, resulting in low automation, delayed feedback, and a single recognition standard. Furthermore, the lack of a systematic judgment analysis and closed-loop feedback mechanism leads to low review efficiency, high misjudgment rates, and difficulty in providing real-time guidance to couriers to correct shooting issues, impacting user experience and delivery quality. This step introduces an intelligent analysis and closed-loop feedback mechanism based on identifiable location photo standards, enabling automatic generation and differentiated processing of review conclusions. The system can automatically generate compliant or non-compliant review conclusions, automatically recording and archiving features for compliant scenarios, improving processing efficiency and data traceability. For non-compliant cases, the system directly feeds back the specific reasons to the courier's mobile device and the business system via real-time message push, accurately guiding re-shooting and effectively improving the compliance rate and recognition accuracy of location photos, achieving full automation and intelligence from judgment to feedback.
[0025] Based on the above, the specific implementation method is as follows: Starting with a photo of a package signed for by a courier, after completing the delivery, courier Xiao Li took a photo of the signing location at the customer's doorstep using a mobile app. This photo was uploaded to the back-end review platform through the courier company's signing system. The system first preprocesses the photo, automatically scaling and standardizing its size to 640x640 pixels. It then immediately launches a pre-trained lightweight YOLO object detection model to scan the photo. The model quickly locates a bounding box with a high confidence score of 0.92 and determines that the object in the box is a courier package. Since 0.92 is much higher than the system's preset threshold of 0.85, the system determines that the photo is initially compliant within milliseconds, meaning that the photo does indeed contain a clearly visible package. If the confidence level is lower than the threshold, the system will immediately end the process and automatically send a standardized prompt to Xiao Li's APP: No clear express package was detected in the photo. Please take a new photo. This quickly filters out invalid or blurry images and avoids the huge workload of manually reviewing such photos. Based on the rapid filtering out of invalid or blurry images, the system enters the expert-level scene analysis stage. The system calls the Qwen2.5-VL open-source multimodal large model, which has been supervised and fine-tuned, and injects pre-designed structured instructions: You are an expert in the compliance review of express delivery receipts. Please judge whether the placement of the package in the photo is identifiable. The criteria include: whether there are clear location markers, such as door signs or company logos at the front desk; whether the background environment has significant features, such as corridors or front desks; and whether the package is placed stably and without serious obstruction. After receiving the photo and instructions, the model encodes the visual content and performs cross-modal fusion understanding with the text instructions. It recognizes that the package is stably placed at the resident's door, with a door number 302 clearly visible next to it. The background is a typical apartment corridor environment with no obstructions. The model infers and generates a structured JSON judgment result: Compliance judgment: Compliance, Judgment basis: The package is placed at the door of resident number 302, the location marker is clear, the background environment is an apartment corridor, the features are significant and the package is unobstructed; The system automatically parses the JSON results returned by the large model. If it is determined to be compliant, the system immediately generates a final approval conclusion and archives the timestamp of this review, the waybill number, and the location feature text extracted from the JSON, which is placed at the door of resident No. 302, into the review log database to complete the record. The whole process does not require manual intervention. If the model determines that the delivery is non-compliant, the returned JSON result is: Compliance judgment: Non-compliant, Reason for non-compliance: Missing location reference, Details: Only the package is on the ground in the photo, and the specific location cannot be identified. The system automatically triggers the feedback mechanism and sends this reason for non-compliance to the courier Xiao Li's APP and business management backend in real time through the message push interface. The specific prompt is: Review failed: Missing location reference, unable to identify the specific delivery location. Please take a photo of the package together with the doorplate, front desk, and other landmarks. Xiao Li can immediately know the problem and take the correct photo in the next delivery, forming an automated closed loop from intelligent recognition to precise guidance, which systematically improves the overall quality and review efficiency of the signed photos.
[0026] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.
Claims
1. A compliance-based intelligent identification method for express parcel delivery location photos, characterized in that, include: S1. Receive the photo of the delivery location of the parcel to be reviewed. Utilize a lightweight object detection model based on the YOLO architecture to calculate the bounding box and confidence score of the photo of the delivery location of the parcel to be reviewed. Set a confidence threshold and detect whether there is a parcel in the photo of the delivery location of the parcel to be reviewed. S2. Based on the judgment that the location of the parcel to be inspected is compliant, design a structured system prompt word engineering, train the Qwen2.5-VL open source multimodal large model, and generate the result of the compliance judgment of the parcel placement location. S3. Based on the compliance determination result of the parcel placement location, the system analyzes the data and generates the final review conclusion. If the review conclusion is compliant, the review approval record is automatically completed. If the review conclusion is non-compliant, the reason for non-compliance is automatically fed back to the business front end, prompting the courier to take a photo of the parcel location correctly, thus realizing intelligent recognition of the compliance of the parcel signing location photo.
2. The compliance intelligent identification method based on the location photo of the express parcel signature as described in claim 1, characterized in that, Step S1 specifically includes: Based on the back-end signing system for express parcels, several RGB images of the signing location of express parcels awaiting review are obtained. The minimum image size of the images of the signing location of express parcels awaiting review is greater than 200 pixels and the maximum image size is less than 4096 pixels, which are used as the image size standard. The images of the signing location of RGB express parcels awaiting review that meet the image size standard are then selected. Based on the RGB images of the unapproved express parcels' signature locations that meet the image size standards, the images are resized and uniformly scaled to a fixed image size of 640×640, and the pixel values are standardized.
3. The compliance intelligent identification method based on the location photo of the express parcel signature as described in claim 2, characterized in that, Step S1 also includes: Using RGB photos of the signed location of parcels awaiting review as input, a lightweight object detection model based on the YOLO architecture is trained. The backbone network is used to extract image features of the RGB photos of the signed location of parcels awaiting review, and multi-scale fusion is performed through the neck network to enhance the model's ability to recognize objects of different image sizes. Based on the RGB image features of the delivery location photo of the parcel to be reviewed, the bounding box and category of the delivery location photo of the parcel to be reviewed are predicted by the decoupling head, and its confidence score is calculated.
4. The compliance intelligent identification method based on the location photo of the express parcel signature as described in claim 3, characterized in that, Step S1 also includes: Set a confidence threshold of 0.85 to check whether the location photos of the signed packages to be reviewed are compliant; If the confidence score of the delivery location photo of the parcel to be reviewed is less than the threshold, the photo is deemed non-compliant, a non-compliant conclusion is output, a standardized prompt is generated indicating that no clearly visible parcel was detected in the photo, and the process ends. If the location photo of the parcel to be signed for is classified as a parcel and the confidence score is greater than or equal to the threshold, the photo is deemed compliant, the initial screening is completed, and the process proceeds to the next step.
5. The compliance intelligent identification method based on the location photo of a signed parcel as described in claim 4, characterized in that, Step S2 specifically includes: The system design includes a structured prompt word project, defining the role as a courier delivery compliance review expert. This expert identifies whether the core object in the delivery location photo of the parcel to be reviewed is a courier parcel. The criteria for identifiable location photos are clear location markers, distinctive background features, stable placement, and no severe obstruction. The system then determines the compliance of the parcel placement location and performs supervised fine-tuning of the Qwen2.5-VL open-source multimodal large model.
6. The compliance intelligent identification method based on the location photo of a signed parcel as described in claim 5, characterized in that, Step S2 also includes: Based on the compliance of the signature location photo of the parcel to be reviewed and the structured system prompt words as input, the signature location photo of the parcel to be reviewed is converted into a visual feature vector by a visual encoder, and the structured system prompt words are converted into word embedding vectors. The visual feature vector of the delivery location photo of the parcel to be reviewed is fused with the word embedding vector in a cross-modal manner to generate a unified multimodal representation of the delivery location photo of the parcel to be reviewed. Based on the unified multimodal representation of the delivery location of the parcel to be reviewed, the Qwen2.5-VL open-source multimodal model with minor adjustments is used for inference to generate a structured JSON text format for determining the compliance of the parcel placement location.
7. The compliance intelligent identification method based on the location photo of a signed parcel as described in claim 6, characterized in that, Step S3 specifically includes: Based on the results of the compliance assessment of the placement location of express parcels, and combined with the location photo identification standard, the system generates the final review conclusion. If the parsing is deemed compliant, the system automatically generates a final conclusion of approval and extracts the location features of the express package; if the parsing is deemed non-compliant, the system automatically generates a final conclusion of disapproval and points out the reason for non-compliance. The reasons for non-compliance include: missing location references, blurred environmental features, and failure to identify express parcels.
8. The compliance intelligent identification method based on the location photo of a signed parcel as described in claim 7, characterized in that, Step S3 also includes: Upon final approval, the system automatically records the status of the parcel receipt approval and extracts the timestamp, tracking number, and location feature text of the parcel, storing them in the approval log database. For cases where the review fails, the reason for non-compliance is pushed to the mobile APP and business front-end system of the courier corresponding to the parcel order in real time through the message push interface, prompting the courier to take a photo of the parcel's location correctly, thus realizing intelligent recognition of the compliance of the parcel's signature location photo.