Door-to-door delivery sign-in photo recognition method and device, equipment and storage medium

Abstract

The invention relates to the technical field of information processing, in particular to a door-to-door signed photo recognition method, device and equipment and a storage medium, data augmentation is performed on a training set by using Cutout and mix to obtain an augmented training set, an AlexNet network model is constructed by using a Dropout regularization technology, the augmented training set is input into the AlexNet network model for model training, and finally, the AlexNet network model is constructed. The method comprises the steps of obtaining a signed photo recognition model, inputting a real-time home signed photo into the signed photo recognition model for compliance recognition, and generating corresponding signed early warning prompt information according to a recognition result of the signed photo recognition model, so that the compliance of the home signed photo can be automatically verified, the intelligent degree is improved, and the success rate of signing is improved. The quality and efficiency of express service are improved, the burden and dependence of manual auditing are reduced, the auditing accuracy is improved, and the enterprise operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a method, device, equipment and storage medium for door-to-door delivery signature photo recognition. Background Art

[0002] At present, with the rapid development of e-commerce, the express logistics industry has also ushered in explosive growth. However, the quality problems of express services have gradually become prominent, especially in the door-to-door delivery and receipt link. In order to improve efficiency, some couriers may take non-compliant photos as receipt proof, which not only damages the rights and interests of consumers, but also affects the service quality and reputation of the express company. The express delivery receipt process only relies on the self-discipline of the courier, lacks effective technical means to ensure the compliance of the receipt photos, cannot automatically identify and verify the receipt photos, has a low degree of intelligence, and requires manual review afterwards, which reduces the efficiency of receipt photo recognition and is prone to review and judgment errors, which increases the burden of manual review and reduces accuracy. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a door-to-door delivery signature photo recognition method, device, equipment and storage medium that automatically verifies the compliance of door-to-door delivery signature photos, improves the degree of intelligence, improves the quality and efficiency of express delivery services, reduces the burden and reliance on manual review, improves review accuracy, and reduces enterprise operating costs.

[0004] The first aspect of the present invention provides a method for identifying door-to-door delivery receipt photos, comprising: collecting historical receipt image sample data, dividing the historical receipt image sample data into a training set, a validation set and a test set according to a preset ratio; using Cutout and mixup to perform data augmentation on the training set to obtain an augmented training set, and using Dropout regularization technology to build an AlexNet network model; inputting the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and tuning the receipt photo recognition model through the validation set; evaluating the receipt photo recognition model through the test set, and receiving real-time door-to-door receipt photos uploaded by a courier terminal; inputting the real-time door-to-door receipt photos into the receipt photo recognition model for compliance recognition, and generating corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model.

[0005] Optionally, in a first implementation method of the first aspect of the present invention, the collecting of historical receipt image sample data and dividing the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio include: collecting historical receipt image sample data; cleaning the historical receipt image sample data to obtain image sample cleaned data; dividing the image sample cleaned data into a training set, a verification set and a test set according to a preset ratio.

[0006] Optionally, in a second implementation manner of the first aspect of the present invention, the training set is augmented using Cutout and mixup to obtain an augmented training set, and the AlexNet network model is constructed using the Dropout regularization technology, including: using AutoAugment to mirror, rotate, scale, translate and brightness adjust the training set to obtain a first training set; using Cutout to crop the training set to obtain a second training set; using mixup to mix the training set, the first training set and the second training set to obtain an augmented training set; and constructing the AlexNet network model using the Dropout regularization technology.

[0007] Optionally, in a third implementation method of the first aspect of the present invention, the augmented training set is input into the AlexNet network model for model training to obtain a receipt photo recognition model, and the receipt photo recognition model is tuned using the verification set, including: inputting the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model; using the verification set to evaluate the performance of the receipt photo recognition model to obtain an evaluation result; judging whether the receipt photo recognition model is overfitting based on the evaluation result; and if so, adjusting the hyperparameters of the receipt photo recognition model.

[0008] Optionally, in a fourth implementation method of the first aspect of the present invention, the signing photo recognition model is evaluated by the test set, and the real-time door-to-door signing photos uploaded by the courier terminal are received, including: comprehensively evaluating the final performance indicators of the signing photo recognition model by the test set, and the final performance indicators include calculation accuracy, recall rate, F1-score and confusion matrix; creating a listener, and using the listener to listen to the real-time door-to-door signing photo information uploaded by the courier terminal; when the listener listens to the real-time door-to-door signing photo information, receiving the real-time door-to-door signing photo information uploaded by the courier terminal; parsing the real-time door-to-door signing photo information to obtain signing photo parsing information; pre-processing the signing photo parsing information to obtain real-time door-to-door signing photos.

[0009] Optionally, in a fifth implementation method of the first aspect of the present invention, the real-time on-site receipt photo is input into the receipt photo recognition model for compliance identification, and corresponding receipt warning prompt information is generated according to the recognition result of the receipt photo recognition model, including: inputting the real-time on-site receipt photo into the receipt photo recognition model for compliance identification; outputting the recognition result of the receipt photo recognition model, and calling the warning prompt statement library; matching the prompt words in the warning prompt statement library according to the mapping relationship between the recognition result and the warning prompt statement library; generating receipt warning prompt information according to the prompt words, and the receipt warning prompt information includes compliance prompt information and re-upload prompt information.

[0010] Optionally, in a sixth implementation method of the first aspect of the present invention, after the real-time door-to-door receipt photo is input into the receipt photo recognition model for compliance identification, and corresponding receipt warning prompt information is generated according to the recognition result of the receipt photo recognition model, it also includes: determining a protocol method for communicating with the courier terminal; converting the receipt warning prompt information into a standard format corresponding to the protocol method; and sending the receipt warning prompt information to the courier terminal according to the protocol method, so that the courier terminal generates and displays a prompt page based on the receipt warning prompt information.

[0011] The second aspect of the present invention provides a door-to-door delivery receipt photo recognition device, including: an acquisition and division module, used to acquire historical receipt image sample data, and divide the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio; an augmentation construction module, used to use Cutout and mixup to perform data augmentation on the training set to obtain an augmented training set, and use Dropout regularization technology to build an AlexNet network model; a training and tuning module, used to input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and tune the receipt photo recognition model through the verification set; an evaluation and receiving module, used to evaluate the receipt photo recognition model through the test set, and receive real-time door-to-door receipt photos uploaded by the courier terminal; an identification generation module, used to input the real-time door-to-door receipt photos into the receipt photo recognition model for compliance identification, and generate corresponding receipt warning prompt information according to the identification result of the receipt photo recognition model.

[0012] Optionally, in a first implementation method of the second aspect of the present invention, the acquisition and division module includes: an acquisition unit, used to collect historical receipt image sample data; a cleaning unit, used to clean the historical receipt image sample data to obtain image sample cleaned data; and a division unit, used to divide the image sample cleaned data into a training set, a verification set and a test set according to a preset ratio.

[0013] Optionally, in a second implementation of the second aspect of the present invention, the augmented construction module includes: a transformation unit, used to use AutoAugment to perform mirroring, rotation, scaling, translation and brightness adjustment transformation processing on the training set to obtain a first training set; a cropping unit, used to use Cutout to crop the training set to obtain a second training set; a mixing unit, used to use mixup to mix the training set, the first training set and the second training set to obtain an augmented training set; and a construction unit, used to construct an AlexNet network model using Dropout regularization technology.

[0014] Optionally, in a third implementation of the second aspect of the present invention, the training and tuning module includes: a training unit, used to input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model; a first evaluation unit, used to use the verification set to evaluate the performance of the receipt photo recognition model to obtain an evaluation result; a judgment unit, used to judge whether the receipt photo recognition model is overfitting based on the evaluation result; and an adjustment unit, used to adjust the hyperparameters of the receipt photo recognition model if so.

[0015] Optionally, in a fourth implementation method of the second aspect of the present invention, the evaluation and receiving module includes: a second evaluation unit, used to comprehensively evaluate the final performance indicators of the receipt photo recognition model through the test set, and the final performance indicators include calculation accuracy, recall rate, F1-score and confusion matrix; a monitoring unit is created, used to create a listener, and use the listener to monitor the real-time door-to-door receipt photo information uploaded by the courier terminal; a receiving unit, used to receive the real-time door-to-door receipt photo information uploaded by the courier terminal when the listener monitors the real-time door-to-door receipt photo information; a parsing unit, used to parse the real-time door-to-door receipt photo information to obtain receipt photo parsing information; a preprocessing unit, used to preprocess the receipt photo parsing information to obtain real-time door-to-door receipt photos.

[0016] Optionally, in a fifth implementation of the second aspect of the present invention, the identification generation module includes: an input identification unit, used to input the real-time on-site receipt photo into the receipt photo recognition model for compliance identification; an output calling unit, used to output the recognition result of the receipt photo recognition model and call the early warning prompt statement library; a matching unit, used to match the prompt words in the early warning prompt statement library according to the mapping relationship between the recognition result and the early warning prompt statement library; a generation unit, used to generate receipt warning prompt information according to the prompt words, and the receipt warning prompt information includes compliance prompt information and re-upload prompt information.

[0017] Optionally, in a sixth implementation method of the second aspect of the present invention, it also includes: a determination module, used to determine a protocol method for communicating with the courier terminal; a conversion module, used to convert the receipt warning prompt information into a standard format corresponding to the protocol method; and a sending module, used to send the receipt warning prompt information to the courier terminal according to the protocol method, so that the courier terminal generates and displays a prompt page based on the receipt warning prompt information.

[0018] A third aspect of the present invention provides a door-to-door delivery photo recognition device, the door-to-door delivery photo recognition device comprising: a memory and at least one processor, the memory storing instructions; at least one of the processors calling the instructions in the memory to enable the door-to-door delivery photo recognition device to perform each step of the door-to-door delivery photo recognition method described above.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of any of the above-mentioned methods for identifying door-to-door delivery receipt photos.

[0020] In the technical solution of the present invention, data augmentation is performed on a training set by using Cutout and mixup to obtain an augmented training set, an AlexNet network model is constructed by using Dropout regularization technology, the augmented training set is input into the AlexNet network model for model training, a receipt photo recognition model is obtained, real-time door-to-door receipt photos are input into the receipt photo recognition model for compliance recognition, corresponding receipt warning prompt information is generated according to the recognition result of the receipt photo recognition model, the compliance of the door-to-door delivery receipt photos can be automatically verified, the degree of intelligence is improved, the quality and efficiency of express delivery services are improved, the burden and dependence of manual review are reduced, the review accuracy is improved, and the operating costs of enterprises are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1A first flow chart of a method for identifying a door-to-door delivery signature photo provided by an embodiment of the present invention;

[0022] Figure 2 A second flow chart of the method for identifying door-to-door delivery signature photos provided in an embodiment of the present invention;

[0023] Figure 3 A third flow chart of the method for identifying door-to-door delivery signature photos provided in an embodiment of the present invention;

[0024] Figure 4 A fourth flow chart of the method for identifying door-to-door delivery signature photos provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of the structure of a door-to-door delivery signature photo recognition device provided by an embodiment of the present invention;

[0026] Figure 6 Another structural schematic diagram of the door-to-door delivery signature photo recognition device provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a door-to-door delivery signature photo recognition device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention provides a door-to-door delivery signature photo recognition method, device, equipment and storage medium, which automatically verify the compliance of the door-to-door delivery signature photo, improve the degree of intelligence, improve the quality and efficiency of express delivery services, reduce the burden and dependence on manual review, improve the accuracy of review, and reduce the operating costs of enterprises.

[0029] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the method for identifying a door-to-door delivery receipt photo in an embodiment of the present invention includes:

[0031] 101. Collect historical receipt image sample data, and divide the historical receipt image sample data into a training set, a verification set, and a test set according to a preset ratio;

[0032] In this embodiment, historical receipt image sample data is collected, covering receipt images of various scenarios, including different weather, lighting conditions, shooting angles and backgrounds, etc., to improve the adaptability of the model, and the collected images are labeled to indicate which images are valid receipts and which are invalid receipts, and relevant information (such as express delivery number, time, etc.) is recorded. The historical receipt image sample data is divided into a training set, a verification set and a test set according to a preset ratio, which can be divided into 70% (training set), 15% (verification set) and 15% (test set).

[0033] 102. Use Cutout and mixup to augment the training set data to obtain the augmented training set, and use the Dropout regularization technology to build the AlexNet network model;

[0034] In this embodiment, Cutout increases the robustness of the model by randomly masking some areas in the images of the training set. Cutout is used to randomly select a rectangular area for each image and set it to zero. The size ratio of the masked area can be set, for example, 10%-20% of the area of ​​the image is masked. Mixup is a data augmentation method that generates new samples by linear interpolation. Two images and their corresponding labels are randomly selected from the training set, and the two images are linearly combined by a hyperparameter λ (usually between 0 and 1). The above process is repeated to generate multiple new training samples to enhance the diversity of the training set. AlexNet consists of 5 convolutional layers, 3 fully connected layers and a ReLU activation function. A Dropout layer is added before each fully connected layer. The commonly used drop probability is 0.5, that is, 50% of the neurons are randomly dropped during each training. Dropout is enabled during training so that the network can see different network structures in each iteration. Dropout is turned off in the testing and verification stages, and the outputs of all neurons are used to obtain more stable results.

[0035] 103. Input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and optimize the receipt photo recognition model using the validation set;

[0036] In this embodiment, the augmented training set is input into the AlexNet network model in batches for training. After the training is completed, the model weights obtained from the training are saved to obtain a receipt photo recognition model. The trained receipt photo recognition model is evaluated using a validation set, and indicators such as the model's accuracy, precision, and recall are calculated to understand the performance of the model on the validation set. According to the evaluation results of the validation set, the hyperparameters of the model are adjusted. For example, the learning rate, Dropout probability, batch size, etc. can be modified to improve the model performance. Methods such as grid search or random search are used to systematically find the best hyperparameter combination, and necessary model retraining and evaluation are performed until the model achieves satisfactory performance on the validation set.

[0037] 104. Evaluate the receipt photo recognition model through the test set and receive the real-time door-to-door receipt photos uploaded by the courier terminal;

[0038] In this embodiment, the test set is input into the trained receipt photo recognition model, and the output results of the model, including the predicted label and confidence score, are recorded. The performance indicators of the model on the augmented test set, such as accuracy, precision, recall and F1 score, are calculated, and the confusion matrix is ​​visualized to intuitively understand in which categories the model performs well or poorly. The misclassified samples are analyzed to find out the reasons why the model fails under specific conditions for subsequent optimization. It is ensured that the mobile terminal (such as a mobile phone or handheld device) used by the courier can support the photo taking and uploading functions, and the application is installed to enable it to interact with the receipt photo recognition model. When the customer signs for the package, the courier takes clear receipt photos through the terminal device, and uploads the taken receipt photos to the server in real time through the mobile network. During the upload process, necessary image compression and format conversion are performed to improve transmission efficiency.

[0039] 105. Input the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generate corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model;

[0040] In this embodiment, after receiving the real-time receipt photo uploaded by the courier, the image is first preprocessed to improve the accuracy of subsequent recognition, and the processed receipt photo is input into the receipt photo recognition model. The model infers the input receipt photo and outputs the prediction result, including the receipt status (such as "received", "not received"), possible abnormal situations (such as "signature does not match", "photo is blurred", etc.) and the corresponding confidence score. According to the output results of the model, it is judged whether the receipt photo meets the predetermined compliance standards, for example, verify whether the signature is clearly visible, whether the signatory is consistent with the expectation, and whether the photo contains relevant express information (such as express number, timestamp, etc.). The output results of the model are analyzed to extract key information, such as receipt status, recognition confidence, abnormal prompts, etc. For the identified abnormal situations, a receipt warning prompt information is generated.

[0041] In the embodiment of the present invention, the training set is augmented by using Cutout and mixup to obtain an augmented training set, the AlexNet network model is constructed by using the Dropout regularization technology, the augmented training set is input into the AlexNet network model for model training, and a receipt photo recognition model is obtained. The real-time door-to-door receipt photos are input into the receipt photo recognition model for compliance recognition, and corresponding receipt warning prompt information is generated according to the recognition result of the receipt photo recognition model. The compliance of the door-to-door delivery receipt photos can be automatically verified, the degree of intelligence is improved, the quality and efficiency of express delivery services are improved, the burden and dependence of manual review are reduced, the review accuracy is improved, and the operating costs of enterprises are reduced.

[0042] See also Figure 2 The second embodiment of the door-to-door delivery receipt photo recognition method in the embodiment of the present invention includes:

[0043] 201. Collect historical receipt image sample data;

[0044] In this embodiment, an API interface is used to extract historical receipt records and corresponding photos from the company's express delivery management system, and the receipt photos provided by the customer are used for supplementary collection. The collected historical receipt images are annotated to ensure that each sample contains necessary information, such as the receipt status, whether the signature is clearly visible, and the time and location of the image.

[0045] 202. Clean the historical receipt image sample data to obtain image sample cleansing data;

[0046] In this embodiment, the collected historical receipt image sample data is imported into the data processing system, and the basic information of the data set is checked, including the number of samples, file format, size, etc., to ensure that the data can be accessed and read normally. It is checked whether there are missing values ​​in the data set (such as missing images, signature information, timestamps, etc.), and the missing values ​​are filled using interpolation or mean filling methods. It is checked whether there are duplicate image samples in the data set, and duplicates are identified and deleted by calculating hash values ​​or comparing image features to ensure the uniqueness of the data. Images that are not related to the receipt subject are filtered out according to preset standards (such as scene inconsistency, irrelevant content, etc.), and the annotation information of the sample data is checked to ensure the accuracy and consistency of the annotations. It is ensured that all image files follow a unified format and naming rules to obtain image sample cleaned data.

[0047] 203. Divide the image sample cleaned data into a training set, a validation set, and a test set according to a preset ratio;

[0048] In this embodiment, the ratio of the training set, the validation set, and the test set is set according to the project requirements and the data scale. The ratios can be 70% (training set), 15% (validation set), and 15% (test set). Then, the image sample cleaned data is divided into the training set, the validation set, and the test set according to the preset ratio. Before the division, the cleaned image sample data is randomized to avoid the order of the data affecting the division result. For example, a random number generator can be used to disrupt the order of the samples.

[0049] 204. Use AutoAugment to perform mirroring, rotation, scaling, translation, and brightness adjustment transformation processing on the training set to obtain a first training set;

[0050] In this embodiment, AutoAugment is used to perform the following basic transformations on the images of the training set: horizontally flipping the image, randomly rotating the image by a certain angle, scaling the image proportionally, randomly translating the image in the horizontal or vertical direction, and randomly changing the brightness of the image. After applying the above enhancement strategies, the generated training set will contain images that have been mirrored, rotated, scaled, translated, and brightness adjusted, while maintaining the original state of the training set, thereby obtaining a first training set.

[0051] 205. Use Cutout to trim the training set to obtain a second training set;

[0052] In this embodiment, the size and number of the occluded areas are first set, and then Cutout is implemented by randomly selecting a rectangular area on the image of the training set and occluding it, thereby realizing the cropping of the training set while maintaining the original state of the training set. After applying Cutout, the generated second training set will contain the cropped image.

[0053] 206. Use mixup to mix the training set, the first training set, and the second training set to obtain an augmented training set;

[0054] In this embodiment, a hyperparameter is set to control the intensity of sample mixing, Mixup is applied to the training set, the first training set, and the second training set, data is loaded and mixed through DataLoader, and an augmented version is generated for each data set to obtain an augmented training set.

[0055] 207. Use Dropout regularization technology to build the AlexNet network model;

[0056] In this embodiment, AlexNet is a classic convolutional neural network, which is usually composed of 5 convolutional layers, a maximum pooling layer, 3 fully connected layers, a ReLU activation function and a Softmax output layer. When constructing AlexNet, a Dropout layer is added to an appropriate layer, for example, Dropout is added to a fully connected layer.

[0057] In the embodiment of the present invention, by collecting and cleaning historical receipt image sample data, it is ensured that the data used for model training is of high quality, and noise and irrelevant information are effectively removed. A variety of data enhancement techniques are used to significantly improve the diversity of the data set, making the model more robust when facing different scenarios and conditions, thereby improving the generalization ability of the model. The data set is divided into a training set, a validation set, and a test set in a reasonable proportion, so that the model training, commissioning, and evaluation processes are more scientific and rigorous, which helps to prevent overfitting and ensure the credibility of the test results. The Dropout regularization technology is used to construct the AlexNet network model, which effectively reduces the risk of overfitting and improves the learning ability and prediction accuracy of the model. By combining a variety of data enhancement and regularization methods, a comprehensive solution is formed, which can better adapt to complex practical application scenarios and improve the performance of the overall model. This series of steps can systematically improve the efficiency and effect of model training, and ultimately achieve higher recognition accuracy and reliability.

[0058] See also Figure 3 A third embodiment of the method for identifying a door-to-door delivery receipt photo in an embodiment of the present invention includes:

[0059] 301. Input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model;

[0060] In this embodiment, the augmented data set is loaded into DataLoader so that it can be read in batches during the training process. A loss function suitable for the classification task, such as cross entropy loss, is used, and an optimizer is selected. During the training process, the augmented training set data is passed into the AlexNet network model, and the loss and accuracy during the training and verification process are monitored. After the training is completed, the signed photo recognition model is obtained, and the signed photo recognition model is saved for subsequent use.

[0061] 302. Use the validation set to evaluate the performance of the receipt photo recognition model and obtain the evaluation results;

[0062] In this embodiment, the receipt photo recognition model is set to evaluation mode, the validation set is used to evaluate the performance of the receipt photo recognition model, the accuracy of the receipt photo recognition model on the validation set is calculated, and the evaluation result is obtained. The performance of the receipt photo recognition model is further analyzed based on the evaluation result. If the accuracy is high, it means that the model can be well generalized to the augmented data and has a certain robustness. If the accuracy is low, it may be necessary to further optimize the model, such as adjusting hyperparameters, adding more data augmentation methods, or fine-tuning the model. In order to better understand the performance of the model, a confusion matrix can be drawn to see which categories have better recognition effects and which categories have worse recognition effects.

[0063] 303. Determine whether the receipt photo recognition model is overfitting based on the evaluation results;

[0064] In this embodiment, high augmented training set accuracy and low validation set accuracy are signs of overfitting. The surface model performs well on the augmented training set, but performs poorly on the unseen validation set, which means that the model may have learned the noise or features in the training data rather than the true distribution of the data. The augmented training set and validation set accuracies are similar and both are very high, indicating that the model has good generalization ability. The low accuracies of the augmented training set and validation set may be due to underfitting of the model, indicating that the model is not capable of capturing potential patterns in the data. By plotting the accuracy (or loss) of the validation set as a function of the number of training rounds, the overfitting phenomenon can be more intuitively observed.

[0065] 304. If yes, adjust the hyperparameters of the receipt photo recognition model;

[0066] In this embodiment, if the receipt photo recognition model is overfitted, the receipt photo recognition model is adjusted after finding the optimal hyperparameters.

[0067] 305. The final performance indicators of the receipt photo recognition model are comprehensively evaluated through the test set. The final performance indicators include calculation accuracy, recall rate, F1-score and confusion matrix;

[0068] In this embodiment, the final performance indicators of the receipt photo recognition model are comprehensively evaluated through the test set to calculate the precision, recall rate, F1-score and confusion matrix. The precision reflects the proportion of correct classification of the model. The higher the value, the better the overall performance of the model. The recall rate is a measure of the model's recognition ability for positive samples. The higher the value, the fewer samples missed by the model. The F1-score comprehensively considers the precision and recall rate and is suitable for cases of class imbalance. The higher the value, the better the overall performance of the model. The confusion matrix visualizes the classification performance of the model to help understand which categories are confused.

[0069] 306. Create a listener, and use the listener to monitor the real-time door-to-door receipt photo information uploaded by the courier terminal;

[0070] In this embodiment, a suitable technology stack is selected to implement the listener, and the selected technology stack is used to write the listener, which is responsible for receiving the photo information uploaded by the courier terminal and ensuring that the courier's terminal can establish a connection with the listener.

[0071] 307. When the monitor monitors the real-time door-to-door receipt photo information, the real-time door-to-door receipt photo information uploaded by the courier terminal is received;

[0072] In this embodiment, when the courier's terminal uploads photos, the listener captures the upload behavior through a specific event mechanism and determines the structure of the uploaded data for rapid parsing and processing.

[0073] 308. Analyze the real-time door-to-door receipt photo information to obtain receipt photo analysis information;

[0074] In this embodiment, after receiving the real-time door-to-door receipt photo information, the real-time door-to-door receipt photo information can be decoded to obtain the receipt photo analysis information, and the receipt photo analysis information can be saved as a file or directly stored in a database.

[0075] 309. Pre-process the parsed information of the receipt photo to obtain a real-time on-site receipt photo;

[0076] In this embodiment, after obtaining the signature photo, the integrity of the signature photo analysis information is verified, and then some basic preprocessing steps are performed on the signature photo analysis information, such as resizing, denoising, enhancing contrast, etc., to obtain a real-time door-to-door signature photo, and the image is converted into a unified format to maintain consistency to obtain a real-time door-to-door signature photo.

[0077] In the embodiment of the present invention, a clear logical chain is formed between the steps, from model training to real-time monitoring, ensuring that each link is closely connected and improving the overall work efficiency. Through real-time evaluation and hyperparameter adjustment, the problem of model overfitting can be identified and solved in time to ensure the adaptability and accuracy of the model. A variety of performance indicators (such as accuracy, recall rate, etc.) are used in combination to provide multi-dimensional feedback for the effectiveness of the model to ensure the reliability and practicality of the model. The introduction of a listener mechanism enables the system to quickly respond to real-time data uploaded by couriers, adapt to actual business needs, and improve user experience. By augmenting the training set and verification set, the generalization ability of the model is enhanced, making it perform better in complex scenarios. The real-time uploaded receipt photos are effectively parsed and preprocessed to ensure data quality and provide an accurate basis for subsequent decision-making.

[0078] See also Figure 4 The fourth embodiment of the method for identifying a door-to-door delivery receipt photo in the embodiment of the present invention includes:

[0079] 401. Input the real-time door-to-door receipt photo into the receipt photo recognition model for compliance recognition;

[0080] In this embodiment, a trained receipt photo recognition model is loaded, and the prepared image is input into the receipt photo recognition model, and the receipt photo recognition model analyzes the photo to determine whether it meets the compliance standards.

[0081] 402. Output the recognition result of the receipt photo recognition model and call the early warning prompt statement library;

[0082] In this embodiment, the receipt photo recognition model outputs the recognition result, and the recognition result is recorded in a database or log, including the order ID, recognition status and confidence score. An early warning prompt statement library is prepared so that it can be called according to the recognition result. A function is created to return the corresponding prompt information based on the input recognition result and confidence level, that is, according to the recognition result (compliant or non-compliant) and confidence level, a suitable prompt word is selected for feedback.

[0083] 403. Matching prompt words in the warning prompt sentence library according to the mapping relationship between the recognition result and the warning prompt sentence library;

[0084] In this embodiment, first, it is necessary to clarify the possible states of the recognition results and their corresponding categories, which include compliance and non-compliance, determine a confidence threshold to judge the credibility of the recognition results, build a mapping relationship based on the recognition results and confidence, implement matching logic in the program, and call the corresponding prompt according to the current recognition results and confidence.

[0085] 404. Generate receipt warning prompt information according to the prompt, and the receipt warning prompt information includes compliance prompt information and re-upload prompt information;

[0086] In this embodiment, a receipt warning prompt message is generated based on the prompt message, and the receipt warning prompt message includes compliance prompt message and re-upload prompt message. The compliance prompt message is used to indicate that the recognition result is compliant, and the re-upload prompt message is used to indicate that the recognition result is not compliant or requires further operation.

[0087] 405. Determine a protocol method for communicating with the courier terminal;

[0088] In this embodiment, a protocol mode for communicating with the courier terminal is determined, and the protocol modes include HTTP / HTTPS, WebSocket, MQTT, FTP / SFTP, etc.

[0089] 406. Convert the receipt warning prompt information into a standard format corresponding to the agreement method;

[0090] In this embodiment, if HTTP / HTTPS or WebSocket protocol is selected, the JSON format is used because it is lightweight and easy to parse. If the MQTT protocol is used, a binary format such as Protocol Buffers is selected. The collected warning prompt information is mapped one by one to the selected data format to ensure that each information point has a corresponding field, and the signed warning prompt information is designed to be a data structure that conforms to the selected protocol.

[0091] 407. Sending the receipt warning prompt information to the courier terminal according to the agreement, so that the courier terminal generates and displays a prompt page based on the receipt warning prompt information;

[0092] In this embodiment, the receipt warning prompt information is sent to the courier terminal according to the protocol, and the courier terminal uses the mobile terminal development tool to generate the receipt warning prompt information into a prompt page.

[0093] In the embodiment of the present invention, the system can quickly identify the compliance of door-to-door signature photos, ensure timely discovery and handling of potential problems, and improve service quality. By combining the recognition results with the early warning prompt statement library, the system can automatically generate accurate prompt information, effectively guide the courier to take corresponding measures, determine the communication protocol method with the courier terminal, make information transmission more efficient, and be able to adapt to different terminal requirements. The prompt information is converted into a standard format, which improves the accuracy and consistency of information transmission and reduces the possibility of misunderstanding. By generating and displaying prompt pages, the courier's operating experience is enhanced to ensure that they can quickly understand and take necessary actions. The entire process realizes automation from input to output, reduces the need for human intervention, and improves work efficiency.

[0094] The above describes the door-to-door delivery receipt photo recognition method in the embodiment of the present invention. The following describes the door-to-door delivery receipt photo recognition device in the embodiment of the present invention. Figure 5 In one embodiment of the present invention, a device for identifying a door-to-door delivery receipt photo includes:

[0095] The collection and division module 501 is used to collect historical receipt image sample data and divide the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio;

[0096] An augmentation construction module 502 is used to perform data augmentation on the training set using Cutout and mixup to obtain an augmented training set, and to construct an AlexNet network model using a Dropout regularization technique;

[0097] The training and tuning module 503 is used to input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and to tune the receipt photo recognition model using a validation set;

[0098] An evaluation receiving module 504 is used to evaluate the receipt photo recognition model through a test set and receive real-time door-to-door receipt photos uploaded by the courier terminal;

[0099] The identification generation module 505 is used to input the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generate corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model.

[0100] In this embodiment, the training set is augmented by using Cutout and mixup to obtain an augmented training set, the Dropout regularization technology is used to build an AlexNet network model, the augmented training set is input into the AlexNet network model for model training, and a receipt photo recognition model is obtained. The real-time door-to-door receipt photos are input into the receipt photo recognition model for compliance recognition, and corresponding receipt warning prompt information is generated according to the recognition results of the receipt photo recognition model. The compliance of the door-to-door delivery receipt photos can be automatically verified, the degree of intelligence is improved, the quality and efficiency of express delivery services are improved, the burden and dependence of manual review are reduced, the review accuracy is improved, and the operating costs of the enterprise are reduced.

[0101] See also Figure 6 Another embodiment of the door-to-door delivery receipt photo recognition device in the embodiment of the present invention includes:

[0102] The collection and division module 501 is used to collect historical receipt image sample data and divide the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio;

[0103] An augmentation construction module 502 is used to perform data augmentation on the training set using Cutout and mixup to obtain an augmented training set, and to construct an AlexNet network model using a Dropout regularization technique;

[0104] The training and tuning module 503 is used to input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and to tune the receipt photo recognition model using a validation set;

[0105] An evaluation receiving module 504 is used to evaluate the receipt photo recognition model through a test set and receive real-time door-to-door receipt photos uploaded by the courier terminal;

[0106] Identification generation module 505, used for inputting the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generating corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model;

[0107] In this embodiment, the acquisition and division module 501 includes: an acquisition unit 5011, which is used to acquire historical receipt image sample data; a cleaning unit 5012, which is used to clean the historical receipt image sample data to obtain image sample cleaned data; and a division unit 5013, which is used to divide the image sample cleaned data into a training set, a verification set and a test set according to a preset ratio.

[0108] In this embodiment, the augmented construction module 502 includes: a transformation unit 5021, which is used to use AutoAugment to perform mirroring, rotation, scaling, translation and brightness adjustment transformation processing on the training set to obtain a first training set; a cropping unit 5022, which is used to use Cutout to crop the training set to obtain a second training set; a mixing unit 5023, which is used to use mixup to mix the training set, the first training set and the second training set to obtain an augmented training set; a construction unit 5024, which is used to construct an AlexNet network model using Dropout regularization technology.

[0109] In this embodiment, the training and tuning module 503 includes: a training unit 5031, which is used to input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model; a first evaluation unit 5032, which is used to use a verification set to evaluate the performance of the receipt photo recognition model to obtain an evaluation result; a judgment unit 5033, which is used to judge whether the receipt photo recognition model is overfitting based on the evaluation result; and an adjustment unit 5034, which is used to adjust the hyperparameters of the receipt photo recognition model if so.

[0110] In this embodiment, the evaluation receiving module 504 includes: a second evaluation unit 5041, which is used to comprehensively evaluate the final performance indicators of the receipt photo recognition model through a test set, and the final performance indicators include calculation accuracy, recall rate, F1-score and confusion matrix; a creation monitoring unit 5042, which is used to create a listener, and use the listener to monitor the real-time door-to-door receipt photo information uploaded by the courier terminal; a receiving unit 5043, which is used to receive the real-time door-to-door receipt photo information uploaded by the courier terminal when the listener monitors the real-time door-to-door receipt photo information; a parsing unit 5044, which is used to parse the real-time door-to-door receipt photo information to obtain the receipt photo parsing information; a preprocessing unit 5045, which is used to preprocess the receipt photo parsing information to obtain the real-time door-to-door receipt photo.

[0111] In this embodiment, the identification generation module 505 includes: an input identification unit 5051, which is used to input the real-time on-site receipt photo into the receipt photo recognition model for compliance identification; an output call unit 5052, which is used to output the recognition result of the receipt photo recognition model and call the early warning prompt statement library; a matching unit 5053, which is used to match the prompt words in the early warning prompt statement library according to the mapping relationship between the recognition result and the early warning prompt statement library; a generation unit 5054, which is used to generate receipt warning prompt information according to the prompt words, and the receipt warning prompt information includes compliance prompt information and re-upload prompt information.

[0112] In this embodiment, it also includes: a determination module 506, which is used to determine the protocol method for communicating with the courier terminal; a conversion module 507, which is used to convert the receipt warning prompt information into a standard format corresponding to the protocol method; a sending module 508, which is used to send the receipt warning prompt information to the courier terminal according to the protocol method, so that the courier terminal generates and displays a prompt page based on the receipt warning prompt information.

[0113] above Figure 5 and Figure 6 The door-to-door delivery receipt photo recognition device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The door-to-door delivery receipt photo recognition device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0114] Figure 7 6 is a schematic diagram of the structure of a door-to-door delivery photo recognition device provided by an embodiment of the present invention. The door-to-door delivery photo recognition device 600 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 610 (for example, one or more processors) and a memory 620, and one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. Among them, the memory 620 and the storage medium 630 can be temporary storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the door-to-door delivery photo recognition device 600. Furthermore, the processor 610 may be configured to communicate with the storage medium 630, and execute a series of instruction operations in the storage medium 630 on the door-to-door delivery photo recognition device 600 to implement the steps of the door-to-door delivery photo recognition method provided by the above-mentioned method embodiments.

[0115] The door-to-door delivery photo recognition device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input and output interfaces 660, and / or one or more operating systems 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 7 The structure of the door-to-door delivery photo recognition device shown does not constitute a limitation on the door-to-door delivery photo recognition device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0116] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the door-to-door delivery signature photo recognition method.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0119] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying door-to-door delivery signature photos, characterized in that: include: Collect historical receipt image sample data, and divide the historical receipt image sample data into a training set, a verification set, and a test set according to a preset ratio; Using Cutout and Mixup to augment the training set to obtain an augmented training set, and using Dropout regularization technology to build an AlexNet network model; Input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and optimize the receipt photo recognition model using the verification set; The receipt photo recognition model is evaluated by using the test set, and real-time door-to-door receipt photos uploaded by the courier terminal are received; The real-time on-site receipt photo is input into the receipt photo recognition model for compliance recognition, and corresponding receipt warning prompt information is generated according to the recognition result of the receipt photo recognition model.

2. The door-to-door delivery receipt photo recognition method according to claim 1, characterized in that: The collecting of historical receipt image sample data and dividing the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio include: Collect historical receipt image sample data; Cleaning the historical receipt image sample data to obtain image sample cleaned data; The image sample cleaning data is divided into a training set, a validation set and a test set according to a preset ratio.

3. The door-to-door delivery receipt photo recognition method according to claim 1, characterized in that: The method uses Cutout and mixup to augment the training set to obtain an augmented training set, and uses the Dropout regularization technology to construct the AlexNet network model, including: Using AutoAugment to perform mirroring, rotation, scaling, translation, and brightness adjustment transformation processing on the training set to obtain a first training set; Use Cutout to trim the training set to obtain a second training set; Using mixup to mix the training set, the first training set, and the second training set to obtain an augmented training set; The AlexNet network model is constructed using Dropout regularization technology.

4. The method for identifying door-to-door delivery receipt photos according to claim 1, characterized in that: The step of inputting the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and optimizing the receipt photo recognition model through the verification set includes: Input the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model; Using the verification set to evaluate the performance of the receipt photo recognition model, obtaining an evaluation result; Determining whether the receipt photo recognition model is overfitting according to the evaluation result; If so, adjust the hyperparameters of the receipt photo recognition model.

5. The door-to-door delivery receipt photo recognition method according to claim 1, characterized in that: The step of evaluating the receipt photo recognition model by using the test set and receiving the real-time door-to-door receipt photo uploaded by the courier terminal includes: Comprehensively evaluate the final performance indicators of the receipt photo recognition model through the test set, and the final performance indicators include calculation accuracy, recall rate, F1-score and confusion matrix; Create a listener, and use the listener to monitor the real-time door-to-door receipt photo information uploaded by the courier terminal; When the monitor monitors the real-time door-to-door receipt photo information, the monitor receives the real-time door-to-door receipt photo information uploaded by the courier terminal; Parse the real-time on-site receipt photo information to obtain receipt photo parsing information; The parsed information of the receipt photo is preprocessed to obtain a real-time on-site receipt photo.

6. The method for identifying door-to-door delivery receipt photos according to claim 1, characterized in that: The step of inputting the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generating corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model, includes: Inputting the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition; Output the recognition result of the receipt photo recognition model and call the early warning prompt statement library; Matching prompt words in the warning prompt sentence library according to a mapping relationship between the recognition result and the warning prompt sentence library; A receipt warning prompt message is generated according to the prompt message, and the receipt warning prompt message includes a prompt message of meeting the requirements and a prompt message of re-uploading.

7. The door-to-door delivery receipt photo recognition method according to claim 1, characterized in that: After inputting the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generating corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model, the method further includes: Determining a protocol mode for communicating with the courier terminal; Converting the receipt warning prompt information into a standard format corresponding to the protocol mode; The receipt warning prompt information is sent to the courier terminal according to the protocol method, so that the courier terminal generates and displays a prompt page based on the receipt warning prompt information.

8. A door-to-door delivery signature photo recognition device, characterized in that: include: An acquisition and division module is used to acquire historical receipt image sample data and divide the historical receipt image sample data into a training set, a verification set and a test set according to a preset ratio; An augmentation construction module, used to perform data augmentation on the training set using Cutout and mixup to obtain an augmented training set, and to construct an AlexNet network model using a Dropout regularization technique; A training and tuning module, used for inputting the augmented training set into the AlexNet network model for model training to obtain a receipt photo recognition model, and tuning the receipt photo recognition model through the verification set; An evaluation receiving module, used to evaluate the receipt photo recognition model using the test set, and receive real-time door-to-door receipt photos uploaded by the courier terminal; The identification generation module is used to input the real-time on-site receipt photo into the receipt photo recognition model for compliance recognition, and generate corresponding receipt warning prompt information according to the recognition result of the receipt photo recognition model.

9. A door-to-door delivery signature photo recognition device, characterized in that: The door-to-door delivery signature photo recognition device comprises: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors calls the instructions in the memory to enable the door-to-door delivery photo recognition device to perform each step of the door-to-door delivery photo recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the door-to-door delivery signature photo recognition method as described in any one of claims 1-7 are implemented.

Images