AR intelligent glasses-based express parcel signing auxiliary method and system

Abstract

The invention discloses an express parcel signing auxiliary method and system based on AR intelligent glasses, and the method comprises the following steps: 1, association relationship binding: building a binding relationship among a three-segment code, a business employee number, an intelligent glasses equipment number and a mobile phone application on an express waybill, and 2, scanning and recognizing the waybill, an express waybill is automatically identified when entering the visual field of the camera, and an identification result is automatically filled into a corresponding information box; a third step of voice recording and signing; a fourth step of photographing and checking a signing position, and uploading the signing position to an application server for checking after successful signing; and a fifth step of data synchronization and flow closed loop. Synchronizing the operation data to the bound mobile phone application in real time; oCR recognition, voice interaction and automatic process control are integrated through the intelligent glasses, the express signing efficiency and compliance are remarkably improved, a salesman does not need to operate a mobile phone, equipment binding and data real-time synchronization ensure that information affiliation is correct, and an efficient and verifiable integrated solution is provided.

Description

Technical Field

[0001] The present invention relates to the field of express logistics technology, and in particular to an express package signing assistance method and system based on AR smart glasses. Background Art

[0002] The signing of express parcels is the last link in the entire process of express parcel circulation. In the current signing process, the salesperson needs to rely on smartphones as the core operating tool to complete the full-link signing action. Specifically, the salesperson must first use the mobile phone camera to scan the parcel bill to obtain the waybill information and upload the signing status. To ensure the compliance of the signing, the salesperson must also take an additional photo containing the parcel, signing location and environmental information. After the system verifies that it meets the specifications, it is uploaded to the cloud. Then, the signing method is entered through the mobile phone application (such as "signing at home", "signing at the collection point", etc.), and the signing result is triggered to be uploaded to the background system, and finally the signing closed loop is completed. During this process, the salesperson needs to frequently operate the mobile phone, switch between scanning, inputting, taking pictures and other functional modules, and must strictly follow the standardized operating procedures to meet the signing verification requirements;

[0003] The signing operation is very dependent on mobile phones, which also brings many negative impacts on the efficiency and work experience of the salesperson, which is specifically reflected in the following aspects. First, it occupies the hands of the salesperson. The current signing process requires operating the mobile phone, package and transportation tool at the same time. The salesperson is forced to switch devices frequently during transportation or movement, resulting in operational fragmentation. In particular, frequent switching of devices when getting on and off the vehicle, and carrying packages significantly reduces the signing efficiency. Second, multi-tasking operations on mobile phones reduce efficiency. Salespeople need to handle a large number of packages at the same time during peak delivery periods, and frequently switch between the functions of mobile phone applications. The ongoing signing process is easily interrupted, resulting in reduced efficiency. Third, the human-computer interaction of mobile phone devices is not good. The existing mobile phone signing system relies on touch screen and keyboard input. Salespeople need to lower their heads to operate, which distracts their attention from the surrounding environment and increases the risk of collision or express package drops. Although the voice interaction function has been partially applied, the salesperson still needs to constantly check the mobile phone screen when operating. Therefore, the present invention proposes an express package signing auxiliary method and system based on AR smart glasses to solve the problems existing in the prior art. Summary of the Invention

[0004] In response to the above problems, the purpose of the present invention is to propose an express package signing assistance method and system based on AR smart glasses. The express package signing assistance method and system based on AR smart glasses integrate OCR recognition, voice interaction and automated process control through AR smart glasses, significantly improving the efficiency and compliance of express delivery: the salesperson does not need to operate the mobile phone, the AR interface superimposes the express order number, receiving address and other key information in real time, and automatically enters the signatory information in combination with voice commands, realizing "hands-free" smooth operation; the system forcibly triggers the photo of the signing location and reviews the compliance of the photo (package, environmental characteristics, etc.) through AI algorithm, and simultaneously records the operation log and dynamic picture to form a traceable evidence chain; at the same time, the deep learning algorithm ensures the accuracy of OCR recognition, and the real-time synchronization of device binding and data ensures the correct ownership of information, which not only reduces manual misjudgment and process interruption, but also meets the "address-based delivery" regulatory requirements, providing an efficient, safe and verifiable integrated solution for express delivery terminal signing.

[0005] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: a method and system for assisting express package signing based on AR smart glasses, comprising the following steps:

[0006] Step 1: Bind the relationship by establishing a binding relationship between the three-segment code on the express delivery bill, the business employee number, the smart glasses device number, and the mobile phone application;

[0007] Step 2: Scan and identify the delivery bill. After the salesperson logs in to the app and enters the receipt interface, the smart glasses camera automatically turns on and displays a real-time preview. When the delivery bill enters the camera's field of view, it is automatically recognized using OCR technology, and the recognition result is automatically filled in the corresponding information box.

[0008] Step 3: Voice input for signing. After the OCR recognition in step 2 is completed, the system automatically activates the voice input module, prompting the salesperson to speak the signatory information. The voice command is parsed by NLP technology, and keywords are extracted for entry. The keywords extracted from the voice command are automatically filled in the "Signatory" information box. After the information box is filled, the screen displays the prompt "Signatory Successful" and confirms it through voice broadcast;

[0009] Step 4: Take a photo of the signing location for review. After the signing is successful in step 3, the photo of the signing location is uploaded to the application server for review, and the user is automatically redirected to the scanning interface of the next express delivery bill. After the review is successful, the smart glasses screen and speaker will prompt "The signing location photo was successfully taken." If the review does not meet the requirements, the user will continue to stay on the signing location photo shooting interface until the user chooses to return or the review is successful.

[0010] Step 5: Data synchronization and process closed loop, operation data is synchronized to the bound mobile application in real time to ensure seamless connection with the background system.

[0011] A further improvement is that the information automatically recognized by the OCR technology in step 2 includes the express delivery number and the delivery address information. When the preview screen is displayed, the screen overlays the display information including the express delivery number, the delivery address, the recipient, and the historical information entry of the previous express delivery number.

[0012] An express package signing assistance system based on AR smart glasses includes a smart glasses body, a device binding authentication module, a visual processing module, a voice interaction processing module, a signing location photo review module, a process control synchronization module and a compliance assurance traceability module. The device binding authentication module is used to establish a binding relationship between the smart glasses device number, the business employee number, the mobile application account and the express delivery bill to ensure the accuracy of the operation data. The visual processing module is used to control the smart glasses camera to capture the express delivery bill and the environment in real time, automatically identify the key information such as the express delivery bill number and the delivery address through OCR technology, and superimpose the operation interface on the AR screen. The management module is used to control the signing process through voice commands, parse the salesperson's voice input, extract keywords and fill them into the corresponding information fields. The signing location photo review module is used to control the smart glasses camera to take signing photos including the package, environment and location, and upload them to the cloud for compliance review. If the review fails, it will prompt to retake the photos. The process control synchronization module is used to realize real-time synchronization of data between the smart glasses and mobile applications and cloud servers, and reduce manual intervention by automatically controlling the jump of the signing process. The compliance assurance traceability module is used to enforce the requirement to take photos of the signing location, and monitor the entire link of the shooting process, providing complete operation logs and impact evidence.

[0013] A further improvement is that the device binding authentication module includes a device information database unit and an identity authentication unit. The device information database unit is used to store the association between smart glasses, mobile phones, and salesmen, and the identity authentication unit is used to complete login through face recognition or work number verification.

[0014] Further improvements are: the visual processing module includes a camera control unit, an OCR recognition unit and an AR display unit, the camera control unit is used to dynamically adjust the camera focus and framing range, the OCR recognition unit is used to extract the face sheet text information based on a deep learning algorithm, and the AR display unit is used to implement a rendering operation interface on the transparent lens.

[0015] A further improvement is that the voice interaction processing module includes a voice recognition unit, an NLP processing unit and a voice feedback unit. The voice recognition unit is used to convert voice instructions into text, the NLP processing unit is used to parse semantics and generate structured instructions, and the voice feedback unit is used to broadcast operation prompts through the smart glasses speaker.

[0016] Further improvements are as follows: the signing location photo review module includes an image acquisition unit, an AI review unit and an exception recording unit. The image acquisition unit is used to automatically capture the signing photos that meet the preset composition requirements. The AI ​​review unit is used to verify whether the photos contain the package, environmental characteristics and compliance requirements through the target detection algorithm. The exception recording unit is used to mark orders that have not been photographed or have failed the review, and associate the camera images before and after the shooting as traceability evidence.

[0017] Further improvements are: the process control synchronization module includes a process control unit, a data synchronization unit and a data cache unit. The process control unit is used to automatically switch to the next link according to the current operation status. The data synchronization unit is used to synchronize the data on the smart glasses end to the mobile phone application and the cloud database. The data cache unit is used to store recent receipt records and supports quick retrieval of the previous order information through the AR interface.

[0018] Further improvements are: the compliance assurance traceability module includes an operation log database unit, a video storage unit and a traceability query unit. The operation log database unit records the operation time, device status, voice instructions and review results of each order. The video storage unit is used to save dynamic images before and after the photo of the signing location is taken. The traceability query unit is used to facilitate administrators to retrieve associated camera videos and operation records through the order number.

[0019] The beneficial effects of the present invention are as follows: the present invention integrates OCR recognition, voice interaction and automated process control through AR smart glasses, significantly improving the efficiency and compliance of express delivery: the salesperson does not need to operate the mobile phone, the AR interface superimposes the express delivery number, delivery address and other key information in real time, and automatically enters the signatory information in combination with voice commands, realizing "hands-free" smooth operation; the system forces the triggering of taking pictures at the signing location and reviews the compliance of the photos (package, environmental characteristics, etc.) through AI algorithms, and simultaneously records operation logs and dynamic pictures to form a traceable chain of evidence; at the same time, deep learning algorithms ensure the accuracy of OCR recognition, and real-time synchronization of device binding and data ensures that the information is correctly attributed, which not only reduces manual misjudgment and process interruption, but also meets the "address-based delivery" regulatory requirements, providing an efficient, safe and verifiable integrated solution for express delivery terminal signing. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the steps of the present invention;

[0021] Figure 2 This is a system architecture diagram of the present invention;

[0022] Figure 3 This is a diagram of the device binding authentication module architecture of the present invention;

[0023] Figure 4This is a diagram of the visual processing module architecture of the present invention;

[0024] Figure 5 This is a diagram of the voice interaction processing module architecture of the present invention;

[0025] Figure 6 This is the architecture diagram of the receipt location photo review module of the present invention;

[0026] Figure 7 This is a diagram of the process control synchronization module architecture of the present invention;

[0027] Figure 8 This is the architecture diagram of the compliance assurance and tracing module of the present invention. DETAILED DESCRIPTION

[0028] In order to deepen the understanding of the present invention, the present invention will be further described in detail below with reference to the examples. The examples are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0029] Example 1

[0030] according to Figure 1 As shown, this embodiment provides an express package signing assistance system based on AR smart glasses, including the following steps:

[0031] Step 1: Bind the relationship by establishing a binding relationship between the three-segment code on the express delivery bill, the business employee number, the smart glasses device number, and the mobile phone application;

[0032] By associating express parcels with salespersons through three-segment codes, the accuracy of data attribution is ensured, effectively improving the convenience of salesperson operations.

[0033] Step 2: Scan and identify the delivery bill. After the salesperson logs in to the app and enters the receipt interface, the smart glasses camera automatically turns on and displays a real-time preview. When the delivery bill enters the camera's field of view, it is automatically recognized using OCR technology, and the recognition result is automatically filled in the corresponding information box.

[0034] The information automatically recognized by OCR technology includes the express delivery number and delivery address information. When the preview screen is displayed, the screen overlays the display information including the express delivery number, delivery address, recipient, and the historical information entry of the previous express delivery number.

[0035] Step 3: Voice input for signing. After the OCR recognition in step 2 is completed, the system automatically activates the voice input module, prompting the salesperson to speak the signatory information. The voice command is parsed by NLP technology, and keywords are extracted for entry. The keywords extracted from the voice command are automatically filled in the "Signatory" information box. After the information box is filled, the screen displays the prompt "Signatory Successful" and confirms it through voice broadcast;

[0036] Enter the consignee's information through voice commands to avoid sales staff typing or clicking on the mobile phone screen, improve operational fluency, use NLP technology to parse natural language instructions, extract standardized signing methods, and reduce human errors.

[0037] Step 4: Take a photo of the signing location for review. After the signing is successful in step 3, the photo of the signing location is uploaded to the application server for review, and the user is automatically redirected to the scanning interface of the next express delivery bill. After the review is successful, the smart glasses screen and speaker will prompt "The signing location photo was successfully taken." If the review does not meet the requirements, the user will continue to stay on the signing location photo shooting interface until the user chooses to return or the review is successful.

[0038] By automatically jumping to the photo-taking interface, we ensure that each order is required to complete the photo taking of the signing location to meet the policy's regulatory requirements for "delivery by address". At the same time, we record orders that have not been photographed or failed the review, and link the dynamic images before and after the shooting to provide a chain of evidence for disputes over false signing.

[0039] Step 5: Data synchronization and process closed loop: Operation data is synchronized to the bound mobile application in real time to ensure seamless connection with the backend system;

[0040] Synchronize smart glasses operation data to mobile phones and the cloud in real time to ensure that the backend system, salespersons and user-side information are consistent, adapting to high-intensity delivery scenarios.

[0041] Example 2

[0042] according to Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 As shown, this embodiment provides an express package signing assistance system based on AR smart glasses, including a smart glasses body, a device binding authentication module, a visual processing module, a voice interaction processing module, a signing location photo review module, a process control synchronization module, and a compliance assurance and tracing module;

[0043] The device binding authentication module is used to establish the binding relationship between the smart glasses device number, business employee number, mobile application account and express delivery label to ensure the accuracy of the attribution of operation data. It includes a device information database unit and an identity authentication unit. The device information database unit is used to store the association between smart glasses, mobile phones and salesmen. The identity authentication unit is used to complete login through face recognition or work number verification.

[0044] The visual processing module is used to control the smart glasses camera to capture the express delivery bill and environmental images in real time, automatically identify key information such as the express delivery bill number and the delivery address through OCR technology, and superimpose the operation interface on the AR screen. It includes a camera control unit, an OCR recognition unit and an AR display unit. The camera control unit is used to dynamically adjust the camera focus and framing range. The OCR recognition unit is used to extract the delivery bill text information based on a deep learning algorithm. The AR display unit is used to render the operation interface on the transparent lens.

[0045] The voice interaction processing module is used to control the signing process through voice commands, parse the salesperson's voice input, extract keywords and fill them into the corresponding information fields. It includes a voice recognition unit, an NLP processing unit and a voice feedback unit. The voice recognition unit is used to convert voice commands into text, the NLP processing unit is used to parse semantics and generate structured instructions, and the voice feedback unit is used to broadcast operation prompts through the smart glasses speaker.

[0046] The receipt location photo review module is used to control the smart glasses camera to take receipt photos including the package, environment and location, and upload them to the cloud for compliance review. It prompts to retake the photo if the review fails. It includes an image acquisition unit, an AI review unit and an exception recording unit. The image acquisition unit is used to automatically capture receipt photos that meet the preset composition requirements. The AI ​​review unit is used to verify whether the photos contain the package, environmental characteristics and compliance requirements through the target detection algorithm. The exception recording unit is used to mark orders that have not been photographed or have failed the review, and associate the camera images before and after shooting as traceability evidence.

[0047] The process control synchronization module is used to achieve real-time data synchronization between the smart glasses, mobile applications, and cloud servers. It reduces manual intervention by automatically controlling the signing process jump. It includes a process control unit, a data synchronization unit, and a data cache unit. The process control unit is used to automatically switch to the next link according to the current operation status. The data synchronization unit is used to synchronize data from the smart glasses to the mobile application and cloud database. The data cache unit is used to store recent signing records and supports quick retrieval of the previous order information through the AR interface.

[0048] The compliance assurance and traceability module is used to enforce the requirement to take photos of the signing location and monitor the entire process, providing complete operation logs and impact evidence. It includes an operation log database unit, a video storage unit, and a traceability query unit. The operation log database unit records the operation time, device status, voice commands, and review results of each order. The video storage unit is used to save dynamic images before and after the signing location photo is taken. The traceability query unit is used to facilitate administrators to retrieve associated camera videos and operation records through the order number.

[0049] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for assisting express package receipt based on AR smart glasses, characterized in that The following steps are involved: Step 1: Bind the relationship by establishing a binding relationship between the three-segment code on the express delivery bill, the business employee number, the smart glasses device number, and the mobile phone application; Step 2: Scan and identify the delivery bill. After the salesperson logs in to the app and enters the receipt interface, the smart glasses camera automatically turns on and displays a real-time preview. When the delivery bill enters the camera's field of view, it is automatically recognized using OCR technology, and the recognition result is automatically filled in the corresponding information box. Step 3: Voice input for signing. After the OCR recognition in step 2 is completed, the system automatically activates the voice input module, prompting the salesperson to speak the signatory's information. The system then uses NLP technology to analyze the voice command, extract keywords for input, and automatically fill the extracted keywords into the "Signatory" information box. After the information box is filled, the screen displays "Signed for Success" and confirms it with a voice announcement. Step 4: Take a photo of the signing location for review. After successfully signing in step 3, the photo of the signing location is uploaded to the application server for review, and the user is automatically redirected to the scanning interface of the next express delivery bill. After a successful review, the smart glasses screen and speaker will prompt "Signature location photo taken successfully." If the review does not meet the requirements, the user will remain on the signing location photo shooting interface until the user chooses to return or the review is successful. Step 5: Data synchronization and process closed loop, operation data is synchronized to the bound mobile application in real time to ensure seamless connection with the background system.

2. The method for assisting express package receipt based on AR smart glasses according to claim 1, characterized in that: The information automatically recognized by the OCR technology in step 2 includes the express delivery number and the delivery address information. When the preview screen is displayed, the screen overlays the display information including the express delivery number, the delivery address, the recipient, and the historical information entry of the previous express delivery number.

3. An express package signing assistance system based on AR smart glasses, characterized by: It includes a smart glasses body, a device binding authentication module, a visual processing module, a voice interaction processing module, a signing location photo review module, a process control synchronization module and a compliance assurance traceability module. The device binding authentication module is used to establish a binding relationship between the smart glasses device number, the business employee number, the mobile application account and the express delivery bill to ensure the accuracy of the operation data. The visual processing module is used to control the smart glasses camera to capture the express delivery bill and the environment in real time, automatically identify the express delivery bill number, the delivery address and other key information through OCR technology, and superimpose the operation interface on the AR screen. The voice interaction processing module is used to use voice commands Control the receipt process, parse the salesperson's voice input, extract keywords and fill them into the corresponding information fields. The receipt location photo review module is used to control the smart glasses camera to take receipt photos including the package, environment and location, and upload them to the cloud for compliance review. If the review fails, it will prompt you to retake the photos. The process control synchronization module is used to achieve real-time synchronization of data between the smart glasses and the mobile application and cloud service end, and reduce manual intervention by automatically controlling the jump of the receipt process. The compliance assurance traceability module is used to enforce the requirement to take photos of the receipt location and monitor the entire link of the shooting process, providing complete operation logs and impact evidence.

4. The express package receipt assistance system based on AR smart glasses according to claim 3 is characterized by: The device binding authentication module includes a device information database unit and an identity authentication unit. The device information database unit is used to store the association relationship between smart glasses, mobile phones, and salesmen. The identity authentication unit is used to complete login through face recognition or work number verification.

5. The express package receipt assistance system based on AR smart glasses according to claim 3 is characterized by: The visual processing module includes a camera control unit, an OCR recognition unit and an AR display unit. The camera control unit is used to dynamically adjust the camera focus and framing range. The OCR recognition unit is used to extract the face sheet text information based on a deep learning algorithm. The AR display unit is used to implement a rendering operation interface on the transparent lens.

6. The express package receipt assistance system based on AR smart glasses according to claim 3 is characterized by: The voice interaction processing module includes a voice recognition unit, an NLP processing unit and a voice feedback unit. The voice recognition unit is used to convert voice instructions into text, the NLP processing unit is used to parse semantics and generate structured instructions, and the voice feedback unit is used to broadcast operation prompts through the smart glasses speaker.

7. The express package receipt assistance system based on AR smart glasses according to claim 3 is characterized by: The receipt location photo review module includes an image acquisition unit, an AI review unit and an exception recording unit. The image acquisition unit is used to automatically capture receipt photos that meet preset composition requirements. The AI ​​review unit is used to verify whether the photos contain packages, environmental characteristics and compliance requirements through a target detection algorithm. The exception recording unit is used to mark orders that have not been photographed or have failed the review, and associate the camera images before and after the shooting as traceability evidence.

8. The express package receipt assistance system based on AR smart glasses according to claim 3 is characterized by: The process control synchronization module includes a process control unit, a data synchronization unit and a data cache unit. The process control unit is used to automatically switch to the next link according to the current operation status. The data synchronization unit is used to synchronize the data on the smart glasses end to the mobile phone application and the cloud database. The data cache unit is used to store recent receipt records and supports quick retrieval of the previous order information through the AR interface.

9. The express package receipt assistance system based on AR smart glasses according to claim 3, characterized in that: The compliance assurance and traceability module includes an operation log database unit, a video storage unit, and a traceability query unit. The operation log database unit records the operation time, device status, voice instructions, and review results of each order. The video storage unit is used to save dynamic images before and after the photo of the receipt location is taken. The traceability query unit is used to facilitate administrators to retrieve associated camera videos and operation records through the order number.

Images