Image recognition-based online car-hailing order locking method

Abstract

This application relates to a ride-hailing order locking method based on image recognition, comprising: upon entering an order verification session, first prompting the user to upload an order information image; receiving user input and determining whether it contains an image to be identified; inputting the image into a classification module based on a visual language model to distinguish between order information images and payment information images; validating the order information image, and if it fails, providing reasons based on the number of uploads and guiding the user to re-upload or transfer to human assistance; if the verification passes, extracting key information such as order number, license plate number, ride time, and origin and destination, retrieving the order according to the information combination type and selecting the corresponding query strategy, locking the unique order, generating details, and verifying it once; if the order is not locked, guiding the user to re-upload, upload payment information images, or transfer to human assistance based on the number of uploads; if there is no image, classifying the problem based on the historical dialogue and providing guidance or traffic diversion.

Description

Technical Field

[0001] This application relates to the field of ride-hailing order processing technology, and in particular to a ride-hailing order locking method based on image recognition. Background Technology

[0002] As the ride-hailing business continues to expand, the number of user inquiries regarding order inquiries, fare disputes, trip irregularities, and invoice issuance has increased significantly in customer service / AI customer service scenarios. To quickly locate the specific order inquired about by users, existing technologies typically employ multi-round verification methods based on text / voice: for example, users verbally or manually input information such as the license plate number, pick-up and drop-off locations, pick-up time, and order number, which the system or human customer service then uses for retrieval and matching. However, due to the multi-source and unstructured nature of ride-hailing order elements, and the possibility of users using abbreviations, slips of the tongue, typos, and inconsistent formats, the above methods reveal the following shortcomings in practical applications: On the one hand, multiple rounds of inquiries and repeated confirmations lead to lengthy conversation processes, requiring users to supplement information repeatedly, resulting in a heavy interaction burden and significantly reducing processing efficiency in high-concurrency scenarios. On the other hand, order matching often relies on a single field or a small number of fields for retrieval. Any recognition deviation or input error in any key field (such as missing / miswritten license plate number, insufficient time precision, or inconsistent origin and destination descriptions) may result in no order being found or multiple orders being matched, making it difficult to quickly identify a unique order. In addition, existing solutions lack a unified image classification, validity verification, and fallback guidance mechanism for handling user-uploaded images. Users may upload screenshots that are not from the order page, payment page, or do not contain key fields, resulting in invalid interactions, repeated uploads, and frequent transfers to human agents, affecting user experience and service quality. Summary of the Invention

[0003] To overcome, to some extent, the problems in related technologies regarding ride-hailing order verification scenarios, such as cumbersome interactions due to reliance on multiple rounds of text / voice communication, easy loss or errors in key order information, resulting in low order retrieval and matching efficiency and difficulty in locking unique orders, this application provides a ride-hailing order locking method based on image recognition.

[0004] The proposed solution is as follows: A method for locking ride-hailing orders based on image recognition, comprising: S1. Enter the order verification session according to the user's request, and determine whether the current session is the first inquiry; if it is the first inquiry, send guidance information to the user to upload the order information image and provide a sample image; S2. Receive user input and determine whether the user input contains the image to be recognized; if it contains the image to be recognized, proceed to S3; if it does not contain the image to be recognized, proceed to S9. S3. Input the image to be identified into the image classification and recognition module, and identify the image category based on the visual language model; when it is identified as an order information image, proceed to S4; when it is identified as a payment information image, proceed to S7. S4. Perform validity validation on the order information image; if the validation passes, proceed to S5; if the validation fails, output the reason for the failure based on the number of times the order information image has been uploaded and guide the user to re-upload, or output a prompt and end the session or transfer to manual processing when the preset number of uploads is reached. S5. Extract key order information from the order information image, including at least one or more of the following: order number, license plate number, pick-up time, pick-up location, and drop-off location; determine the information combination type based on the extracted key order information, and call the order query strategy corresponding to the information combination type to perform order retrieval; if a unique order is found, generate order details and confirm with the user; if no unique order is found, proceed to S6. S6. When a unique order is not locked, a fallback procedure is performed based on the number of times the order information image has been uploaded: if the order is not locked for the first time, the user is guided to re-upload the order information image; if the order is not locked for the second time, the user is guided to upload the payment information image; if the preset threshold for the number of attempts is reached, the user is prompted that no order can be found and the process is transferred to a human for manual processing. S7. When the image category is a payment information image, determine whether the preconditions for processing payment information images are met based on the number of times order information images have been uploaded. If the conditions are met, proceed to S8. If the conditions are not met, output a prompt and guide the upload of order information images or end the session when the preset number of uploads is reached. S8. Extract the transaction order number and / or merchant order number from the payment information image, and query the order based on the transaction order number and / or merchant order number; if a unique order is found, generate the order details and confirm with the user; if no unique order is found, output a prompt and transfer to manual processing. S9. Use generative models combined with historical chat records to classify user questions, and perform corresponding guidance, supplementary information collection, or transfer to manual processing based on the classification results.

[0005] Preferably, the validity verification of the order information image includes at least the following: Verify that the image is from an order page on this platform, contains a license plate number, contains the rental time, and contains at least one of the following: origin and destination. When the verification fails, output the specific reason for the failure and provide guidance on supplementing / replacing the uploaded content; The prerequisite for processing the payment information image in S7 is that the number of times the order information image has been uploaded is greater than or equal to 2. If this condition is not met, the user will be prompted that the payment information image does not meet the current processing requirements, and the user will be guided to upload the order information image first.

[0006] Preferably, the preset number of times threshold is 3 times; When the number of consecutive uploads of order information images does not meet the requirements and reaches the threshold, a prompt message indicating that multiple uploads do not meet the requirements will be output, and the user will be advised to find the corresponding order again before entering the session. The user will also be asked to actively end the session or transfer the case to a human for processing.

[0007] Preferably, the information combination type includes: the order information image contains an order number that conforms to the order numbering rules of this platform; The corresponding order query strategy is as follows: use the order number as the search key to query the order. If the order is found, proceed to confirmation and verification. If the order is not found, proceed to step S6.

[0008] Preferably, the information combination type includes: the order information image contains the license plate number and the vehicle usage time accurate to the second or minute; The corresponding order query strategy is as follows: A query time range is generated based on the vehicle usage time; when the vehicle usage time is accurate to the second, the start time and end time of the time range are the same; when the vehicle usage time is accurate to the minute, the start time is 00 seconds appended to the minute and the end time is 59 seconds appended to the minute. Query orders based on license plate number and the time range; If a unique order is found, proceed to a confirmation and verification step. If multiple orders are found, when an order entry with a circled / highlighted mark is detected in the image, further filtering is performed based on the field of the circled / highlighted mark to obtain a unique order; otherwise, it is considered that the unique order has not been locked and proceeds to step S6.

[0009] Preferably, the information combination type includes: the order information image is an order list page and multiple candidate order information can be extracted; the candidate order information includes at least one or more of the following: license plate number, vehicle usage time, origin and destination; The corresponding order query strategy is as follows: When a circled / highlighted item is detected in an image, the vehicle usage time of the circled / highlighted item is used first to generate a query time range, and the query is performed based on the license plate number and the time range. Then, the start and end points of the circled / highlighted item are used to filter for a unique order. If no selected / highlighted item is detected, and the image also contains the vehicle usage date or time from the order details page, the candidate order information is first filtered based on the vehicle usage date or time to form a smaller candidate set. Generate query time ranges for each item in the filtered candidate set or the original candidate set, perform license plate number queries and time range queries, and filter unique orders based on the start and end points of the corresponding items.

[0010] Preferably, the information combination type includes: the order information image contains the ride time, origin and destination accurate to the second or minute; The corresponding order query strategy is as follows: Generate a query time range based on the vehicle usage time, and query orders by origin, destination and time range; If a unique order is found, proceed to a confirmation and verification step. If no unique order is found, the candidate order information extracted from the order list page is filtered a second time by the usage time, origin and destination. If a unique candidate is found and the usage time of the candidate is accurate to the second, a second query is performed with the time range accurate to the second to lock the unique order. Otherwise, proceed to step S6.

[0011] Preferably, the information combination type includes: The information combination types include: the order details page contains the origin and destination, and the order list page contains the car rental time accurate to the second or minute; The corresponding order query strategy is as follows: When the candidate order information extracted from the list page is unique, the vehicle usage time of the candidate order is used to generate a query time range, and the order is queried by origin, destination and time range; When the candidate order information is not unique, if a selected / highlighted item is detected, the start and end points of the selected / highlighted item and the usage time are used to generate a query time range for querying; Otherwise, if the details page contains the ride time, it is used to filter candidate order information; or if the details page contains the origin and destination, it is used to filter candidate order information. After filtering out a unique candidate, the order is then queried by origin, destination and time range. If no unique order is locked, proceed to step S6.

[0012] Preferably, when the key information of the order does not meet any preset information combination type, the order query strategy is to enter step S6 to perform guided retransmission, guided upload of payment information image, or transfer to manual processing.

[0013] Preferably, the categories of user issues include at least: confirming / verifying orders, denying / verifying orders, inability to provide order images, and other issues; Specifically, when the user's question is to confirm the order, the user is guided to enter a specific question and exit the order confirmation process; When a user's question is about denying order verification or other issues, the system will guide them to re-upload the order information image or transfer the user to a human agent based on the number of times the order information image has been uploaded and the number of times the payment information image has been uploaded. When a user's problem is that they cannot provide an order image, the decision will be made to transfer the case to a human operator or to continue guiding them to upload an order information image, depending on whether an image upload has already occurred at least once. When the user's problem is another issue, after completing or skipping the order lock, the intent recognition is performed and the problem handling strategy is generated by combining historical chat records and locked order information, and the corresponding handling instructions are output or the process is triggered to transfer to human processing.

[0014] The technical solution provided in this application may include the following beneficial effects: This application constructs a closed-loop process in the order verification session, encompassing "initial inquiry-guided upload, image classification and recognition, order image validity verification, key information extraction, selection of query strategy based on information combination type, locking a unique order and verifying it once, and fallback routing for failed attempts." This enables the system to automatically obtain elements such as order number, license plate number, usage time, and origin and destination directly from the order / payment screenshot and complete accurate retrieval, significantly reducing the time consumption and human input errors associated with traditional multi-round text / voice verification. Simultaneously, by leveraging compliance verification and a progressive guidance mechanism based on upload count thresholds, and a mechanism for transferring to human intervention, the probability of invalid interactions and repeated communication is reduced, improving the efficiency, accuracy, and stability of order locking and session processing, thereby enhancing customer service efficiency and user experience.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] Figure 1 This is a flowchart illustrating a ride-hailing order locking method based on image recognition, provided in one embodiment of this application. Detailed Implementation

[0018] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0019] Figure 1 This is a flowchart illustrating an image recognition-based ride-hailing order locking method according to an embodiment of this application. (Refer to...) Figure 1 A method for locking ride-hailing orders based on image recognition, comprising: S1. Enter the order verification session according to the user's request, and determine whether the current session is the first inquiry; if it is the first inquiry, send guidance information to the user to upload the order information image and provide a sample image; S2. Receive user input and determine whether the user input contains the image to be recognized; if it contains the image to be recognized, proceed to S3; if it does not contain the image to be recognized, proceed to S9. S3. Input the image to be identified into the image classification and recognition module, and identify the image category based on the visual language model; if it is identified as an order information image, proceed to S4; if it is identified as a payment information image, proceed to S7. S4. Perform validity validation on the order information image; if the validation passes, proceed to S5; if the validation fails, output the reason for the failure based on the number of times the order information image has been uploaded and guide the user to re-upload, or output a prompt and end the session or transfer to manual processing when the preset number of uploads is reached. S5. Extract key order information from the order information image, including at least one or more of the following: order number, license plate number, pick-up time, pick-up location, and drop-off location; determine the information combination type based on the extracted key order information, and call the order query strategy corresponding to the information combination type to perform order retrieval; if a unique order is found, generate order details and confirm with the user; if no unique order is found, proceed to S6. S6. When a unique order is not locked, a fallback procedure is performed based on the number of times the order information image has been uploaded: if the order is not locked for the first time, the user is guided to re-upload the order information image; if the order is not locked for the second time, the user is guided to upload the payment information image; if the preset threshold for the number of attempts is reached, the user is prompted that no order can be found and the process is transferred to a human for manual processing. S7. When the image category is payment information image, determine whether the preconditions for processing payment information image are met based on the number of times the order information image has been uploaded. If the condition is met, proceed to S8. If the condition is not met, output a prompt and guide the upload of the order information image or end the session when the preset number of uploads is reached. S8. Extract the transaction order number and / or merchant order number from the payment information image, and query the order based on the transaction order number and / or merchant order number; if a unique order is found, generate the order details and confirm with the user; if no unique order is found, output a prompt and transfer to manual processing. S9. Use generative models combined with historical chat records to classify user questions, and perform corresponding guidance, supplementary information collection, or transfer to manual processing based on the classification results.

[0020] An order verification session refers to the interactive process between the user and the customer service system to resolve issues related to ride-hailing orders. This session can be conducted in various formats, including text, voice, or images.

[0021] User input refers to the content entered by the user based on the guidance information.

[0022] Order information images refer to images uploaded by users that contain details of ride-hailing orders or order lists. These images typically include key information such as order number, license plate number, pick-up time, pick-up location, and drop-off location.

[0023] Payment information images refer to images uploaded by users that contain payment receipts or transaction records. These images typically include information such as transaction order numbers or merchant order numbers.

[0024] The image classification and recognition module is a functional unit used to receive images to be recognized and classify them. This module can distinguish between different types of images, such as order information images or payment information images.

[0025] A visual language model (VLM) is an artificial intelligence model capable of processing both image and text information simultaneously. This model combines visual features and linguistic semantics to achieve deep understanding and recognition of image content, such as identifying text, layout, and entities within an image. In practical applications, the VLM model can be used.

[0026] Key order information refers to important data extracted from order information images that is used to uniquely identify or assist in order retrieval. This information includes at least one or more of the following: order number, license plate number, pick-up time, and drop-off location.

[0027] Information combination type refers to the preset rules for classifying order query conditions based on different combinations of key order information extracted from order information images. Different information combination types correspond to different order query strategies.

[0028] An order query strategy refers to a specific algorithm or process used to retrieve matching orders from a backend database based on a particular combination of information. This strategy aims to improve the accuracy and efficiency of order locking by effectively utilizing extracted key information.

[0029] Guided re-upload refers to sending a prompt message to the user's device to instruct them to re-upload the image. For example, when a user needs to re-upload an order information image, the message "Please re-upload the order information image" can be sent to the user's device to prompt them to do so.

[0030] Generative models are artificial intelligence models that can understand user intent and generate corresponding responses or perform specific actions. These models combine historical chat logs to categorize user questions and provide suggestions such as guidance, information gathering, or referral to human intervention.

[0031] This application provides a method for locking ride-hailing orders based on image recognition, the specific implementation process of which may include the following steps: Preliminary steps: Connect to the user's phone call and perform an order inquiry based on the user's order inquiry request initiated over the phone. If no order is found, send an SMS to the user's device, which contains a mini-program URL. This mini-program can be opened to access online customer service, where the user can initiate an order verification session with online customer service.

[0032] In the order verification session, steps S1-S9 are executed. In S1, based on the user's request, the system enters the order verification session. At this point, the system determines whether this is the user's first time initiating the session. If it is, the system sends guidance information to the user, prompting them to upload an image of their order information, and can provide a sample image to aid understanding. For example, the system can send a text message stating, "Please upload a screenshot of your order so we can quickly verify the order information," along with a clear sample order screenshot. If it is not the first time, the system continues processing based on the historical session status.

[0033] In S2, the system receives user input. Then, the system determines whether the user input contains the image to be recognized. This determination can be made by detecting the presence of image data in the input stream. If the image to be recognized is detected, the process proceeds to S3. If the image to be recognized is not detected, for example, if the user inputs plain text, the process proceeds to S9.

[0034] In step S3, the image to be identified is input into the image classification and recognition module. This module uses a visual language model to classify the image. For example, the visual language model can analyze the visual features of the image and the text information it contains to determine whether the image is an order information image or a payment information image. If the recognition result is an order information image, the process proceeds to step S4. If the recognition result is a payment information image, the process proceeds to step S7.

[0035] In S4, the identified order information images undergo validity verification. This verification may include checks on image format, clarity, and content completeness. For example, the system can check if the image is in a common image format (such as JPG or PNG), if the image content is blurry, or if the image contains any identifiable text information. If the verification passes, the process proceeds to S5. If the verification fails, the system outputs the reason for the failure based on the number of times the order information image has been uploaded and guides the user to re-upload. For example, the system may prompt, "The image is not clear; please re-upload a clear screenshot of the order." If the number of uploads reaches a preset threshold, the system outputs a prompt message and can choose to end the session or transfer the session to human customer service. This preset threshold can be configured according to actual operational needs, for example, set to 3 times.

[0036] In S5, key order information is extracted from the valid order information image. This key information includes at least one or more of the following: order number, license plate number, pick-up time, and pick-up and drop-off locations. For example, the system can use Optical Character Recognition (OCR) technology to recognize text in the image and combine it with Natural Language Processing (NLP) technology to structurally extract these key fields from the recognized text. Based on the extracted key order information, the system determines its corresponding information combination type. For example, if only the order number is extracted, it corresponds to one information combination type; if the license plate number and pick-up time are extracted, it corresponds to another information combination type. Subsequently, the system calls the order query strategy corresponding to the information combination type to perform order retrieval. If a unique matching order is found in the background database, the system generates the order details and confirms them with the user. If no unique order is found, such as multiple orders or no orders are found, the process proceeds to S6.

[0037] In S6, when a unique order cannot be locked, the system performs a fallback based on the number of times the order information image has been uploaded. For example, if a unique order cannot be locked the first time, the system can guide the user to re-upload the order information image to obtain clearer or more complete order information. If a unique order cannot be locked the second time, the system can guide the user to upload a payment information image to attempt to search for the order through the payment information. If the number of uploads reaches a preset threshold, the system will notify the user that no order could be found and transfer the session to human customer service. This preset threshold may be the same as or different from the threshold in S4.

[0038] In S7, when the image category is identified as a payment information image, the system determines whether the precondition for processing the payment information image is met based on the number of times the order information image has been uploaded. This precondition can be set according to business logic, for example, requiring the user to have attempted to upload an order information image at least once before uploading the payment information image. If the precondition is met, the processing flow proceeds to S8. If the precondition is not met, the system outputs a prompt message and guides the user to upload an order information image. If the number of uploads reaches a preset threshold, the system can choose to terminate the session.

[0039] In S8, the transaction order number and / or merchant order number are extracted from the payment information image. This extraction process can also utilize OCR technology to recognize text in the image and identify the transaction order number or merchant order number. Subsequently, the system queries the backend order database based on the extracted transaction order number and / or merchant order number. If a unique matching order is found, the system generates the order details and confirms them with the user. If no unique order is found, the system outputs a prompt message and transfers the session to human customer service for processing.

[0040] In S9, when user input does not contain the image to be recognized, the system uses a generative model combined with historical chat history to categorize the user's question. For example, the generative model can analyze the user's input text and previous conversation context to determine whether the user wants to inquire about order details, modify an order, complain about a driver, or has other issues. Based on the categorization results, the system performs corresponding guidance, collects supplementary information, or transfers the issue to human assistance. For example, if the user's question is categorized as "inquiring about order details" but the order has not yet been locked, the system may guide the user to provide more information; if the question is complex or cannot be handled automatically, it will be transferred to human customer service.

[0041] This application automates the processing of user-uploaded order or payment information images by introducing image recognition and visual language models, effectively solving the problems of low information collection efficiency and poor accuracy in traditional ride-hailing order verification. Users no longer need to manually enter cumbersome information; they can quickly lock their orders simply by uploading images, significantly optimizing the user experience. Simultaneously, the multi-stage verification and fallback processing mechanism improves the success rate of order locking and reduces the workload of customer service personnel.

[0042] Example 2 It should be noted that the validity verification of order information images includes at least the following: Verify that the image is from an order page on this platform, contains a license plate number, contains the rental time, and contains at least one of the following: origin and destination. When the verification fails, output the specific reason for the failure and provide guidance on supplementing / replacing the uploaded content; The prerequisite for processing payment information images in S7 is that the number of order information images uploaded is greater than or equal to 2. If this condition is not met, the user will be prompted that the payment information image does not meet the current processing requirements, and the user will be guided to upload the order information image first.

[0043] In some of the above implementations, after the system recognizes that the image uploaded by the user is an order information image, it attempts to extract key information from it to locate the order. However, the order information image uploaded by the user may not be the order page of this platform, or the image content may be incomplete or unclear, making it impossible to effectively extract key order information, thereby affecting the accuracy and efficiency of order retrieval, and may even reduce the user experience due to repeated attempts with invalid images.

[0044] To address this, this application further proposes a validity check for order information images. This check aims to ensure that the order information images uploaded by users meet the requirements for subsequent extraction of key order information and order retrieval. Specifically, this check can include several aspects. For example, it can check whether the image is from the platform's order page. This can be achieved through image recognition technology, such as training a deep learning model to identify unique visual elements of the platform's order page, such as the platform logo, specific UI layout, font style, or color theme. The system compares the image to be checked with a preset order page template for the platform, or determines its source through feature extraction and matching. Simultaneously, it can also check whether the image contains a license plate number, which is typically achieved through Optical Character Recognition (OCR) technology. The system first performs text region detection on the image, then performs OCR recognition on the detected text, and uses regular expressions or a pre-trained named entity recognition model to determine whether a string matching the license plate number format exists. The license plate number is one of the important identifiers for locking a specific order. Furthermore, it can also check whether the image contains the usage time; similarly, date and time information in the image is identified through OCR technology. The system recognizes common date and time formats (such as "year-month-day hour:minute:second" and "year-month-day hour:minute") and determines if valid ride-hailing time information exists. Ride-hailing time is crucial for narrowing down order search results. Furthermore, it can verify if the image contains at least one of the origin and destination. Combining OCR and Natural Language Processing (NLP) technologies, the system identifies text in the image and extracts possible location information. This can be accomplished using a pre-defined location dictionary, geocoding services, or named entity recognition models. The origin and destination are key geographical information for further precisely pinpointing the order.

[0045] If any of the above validity checks fails, the system will generate a corresponding prompt based on the specific reason for the failure, and provide guidance on supplementing or replacing the uploaded content. For example, if the platform logo is not recognized in the image, the system will prompt, "The image you uploaded does not appear to be from this platform's order page. Please upload a screenshot of your order from this platform." If the license plate number is not detected, the system will prompt, "No license plate number was detected in the image. Please ensure the license plate number is clearly visible." The system will also provide clear guidance, such as suggesting that the user re-upload a clearer image, upload an image containing specific information, or providing sample images for reference to help the user understand and provide a compliant order information image.

[0046] Furthermore, in step S7 above, when the image category is payment information image, the system will determine whether the precondition for processing payment information images is met based on the number of times order information images have been uploaded. This precondition sets a logical priority: only after at least two failed attempts to lock the order using order information images will the user be allowed to upload payment information images as an auxiliary means. The system maintains a counter to record the number of times the user uploads order information images. Before entering step S7 to process payment information images, the system checks the value of this counter; the precondition is met only when the number of order information image uploads is greater than or equal to 2. If the precondition is not met, the system will prompt the user that the payment information image does not meet the current processing requirements and guide the user to upload order information images first. For example, the system can prompt, "To help you check your order more accurately, please upload a screenshot of the order details page first. If you have tried uploading the order details page multiple times and still cannot lock the order, we will guide you to upload payment information images." This helps to standardize the user operation process and ensure that the system attempts to lock the order according to the preset priority.

[0047] By introducing validity checks on order information images, the system can perform multi-dimensional checks upon receiving such images to ensure the correctness of the image source and the completeness of key information. This effectively avoids complex subsequent processing of invalid or incomplete images, significantly improving the efficiency of order information extraction and retrieval. When validation fails, the system outputs specific reasons for the failure and provides clear upload guidance, greatly improving user experience and reducing the likelihood of users repeatedly uploading invalid images due to a lack of understanding of the requirements. This guides users to provide compliant order information more quickly. Furthermore, by setting preconditions for processing payment information images, the system can prioritize users using order information images—a more direct and efficient order locking method—only using payment information images as a supplementary means after multiple failed attempts with order information images. This phased and strategic processing flow not only optimizes resource utilization but also provides users with a clearer and more reasonable order locking path, improving the overall success rate of order locking methods and user satisfaction.

[0048] Example 3 It should be noted that the preset threshold for the number of attempts is 3. When the number of consecutive uploads of order information images that do not meet the requirements reaches the threshold, a prompt message will be output indicating that multiple uploads have failed to meet the requirements. The user will be advised to find the corresponding order again and re-enter the session. The user will also be asked to actively end the session or transfer the case to a human for processing.

[0049] Specifically, this application sets the preset attempt threshold to 3 times. This preset attempt threshold refers to the maximum number of times the system allows a user to attempt an order in a specific scenario, such as when the uploaded order information image does not meet the requirements or when a unique order cannot be locked. Setting this threshold aims to balance the flexibility of user operation with the efficiency of system processing, giving users a certain margin for error while preventing them from getting bogged down in an infinite number of invalid attempts, thereby optimizing the user experience and saving system resources. This threshold can be a configurable system parameter, for example, set in the system's backend management interface or defined through a configuration file. In this embodiment, the threshold is specifically set to 3 times, meaning that a user can attempt to upload order information images or perform order queries a maximum of 3 times.

[0050] When the number of consecutively uploaded order information images that do not meet the requirements reaches the threshold, the system will output a message indicating multiple failed uploads and suggest that the user find the corresponding order again before re-entering the conversation. The system will then either actively end the conversation or transfer the issue to manual processing. This step is the final handling strategy adopted by the system when a user repeatedly attempts to upload order information images but fails to meet the requirements. Its purpose is to clearly inform the user of the current problem, provide suggestions for the next steps, and effectively terminate ineffective automated processes, preventing the user from getting stuck. Specifically, the system can send a pre-set text message to the user through the chat interface, such as: "You have uploaded multiple failed order information images. To help you more accurately, we suggest you first verify the order information, find the correct order page screenshot, and then re-initiate the conversation." The message contains clear guidance, directing the user to complete the information preparation work externally, rather than continuing to try within the current conversation. Meanwhile, the system can send a session termination command, closing the current chat window or redirecting the user to the main interface, thereby terminating the current automated service process; alternatively, the system can package the context information of the current session (including user questions, uploaded images, number of attempts, etc.) and route it to the human customer service queue, where human customer service can intervene based on this information to provide users with more personalized assistance.

[0051] Through the above technical solution, in the image recognition-based ride-hailing order locking method, when the order information images uploaded by the user are continuously non-compliant, the system can effectively avoid users wasting too much time on invalid attempts by setting a preset threshold and taking appropriate action, thus improving the efficiency of automated processing. By timely outputting clear prompts, guiding users to prepare information again, or transferring them to human service, the system ensures that user problems are ultimately resolved, significantly improving user satisfaction and the system's intelligence level, and avoiding resource waste and negative user experience caused by repeated invalid operations.

[0052] Example 4 This embodiment describes the various information combination types: The first type of information combination includes: order information images containing order numbers that conform to the order numbering rules of this platform; The corresponding order query strategy is as follows: use the order number as the search key to query the order. If the order is found, proceed to confirmation and verification. If the order is not found, proceed to step S6.

[0053] The second type of information combination includes: order information images containing license plate numbers and the exact time of vehicle use, accurate to the second or minute; The corresponding order query strategy is as follows: The query time range is generated based on the vehicle usage time; when the vehicle usage time is accurate to the second, the start time and end time of the time range are the same; when the vehicle usage time is accurate to the minute, the start time is 00 seconds appended to the hour and minute, and the end time is 59 seconds appended to the hour and minute. Search for orders based on license plate number and time range; If a unique order is found, proceed to a confirmation and verification step. If multiple orders are found, when an order entry with a circled / highlighted field is detected in the image, further filtering is performed based on the field of the circled / highlighted entry to obtain a unique order; otherwise, it is considered that the unique order has not been locked and proceeds to S6.

[0054] The third type of information combination includes: the order information image is an order list page and multiple candidate order information can be extracted; the candidate order information includes at least one or more of the following: license plate number, usage time, origin and destination; The corresponding order query strategy is as follows: When a circled / highlighted item is detected in an image, the vehicle usage time of the circled / highlighted item is used first to generate a query time range, and the query is performed based on the license plate number and time range. Then, the start and end points of the circled / highlighted item are used to filter for unique orders. If no selected / highlighted items are detected, and the image also contains the vehicle usage date or time from the order details page, the candidate order information will be filtered by the vehicle usage date or time to form a smaller candidate set. Generate query time ranges for each item in the filtered candidate set or the original candidate set, perform license plate number queries and time range queries, and filter unique orders based on the start and end points of the corresponding items.

[0055] The fourth type of information combination includes: order information images containing the exact time of the ride, origin, and destination, accurate to the second or minute; The corresponding order query strategy is as follows: Generate a query time range based on the vehicle usage time, and query orders by origin, destination and time range; If a unique order is found, proceed to a confirmation and verification step. If no unique order is found, the candidate order information extracted from the order list page is filtered a second time by the usage time, origin and destination. If a unique candidate is found and the usage time of the candidate is accurate to the second, a second query is performed with the time range accurate to the second to lock the unique order. Otherwise, proceed to S6.

[0056] The fifth type of information combination includes: Information combination types include: the order details page includes the origin and destination, and the order list page includes the rental time accurate to the second or minute; The corresponding order query strategy is as follows: When the candidate order information extracted from the list page is unique, the vehicle usage time of the candidate order is used to generate a query time range, and the order is queried by origin, destination and time range; When the candidate order information is not unique, if a selected / highlighted item is detected, the start and end points of the selected / highlighted item and the usage time are used to generate a query time range for querying; Otherwise, if the details page contains the ride time, use it to filter candidate order information; or if the details page contains the origin and destination, use it to filter candidate order information. After filtering to find a unique candidate, search for the order by origin, destination, and time range. If no unique order is found, proceed to S6.

[0057] Furthermore, when the key information of an order does not meet any of the preset information combination types, the order query strategy is to enter S6 to perform guided retransmission, guided upload of payment information images, or transfer to manual processing.

[0058] Specifically, when the key order information does not meet any preset information combination type, it means that after the system extracts key order information (such as order number, license plate number, ride time, pick-up location, and drop-off location) from the order information image, it attempts to match this extracted information with a variety of predefined information combination types. This condition is triggered when the extracted key information cannot fully match or be categorized into any preset combination pattern. For example, if only the license plate number is identified in the image, but the ride time is not, and all preset combination types require both the license plate number and the ride time to exist simultaneously, it will be determined that no preset information combination type is met. At this time, the order query strategy will no longer attempt to execute specific order query logic, but will directly jump to step S6. Step S6 is a preset fallback mechanism designed to handle various situations where a unique order cannot be successfully locked. By directly entering S6, the system can uniformly and strategically guide users to perform subsequent operations, avoiding a deadlock due to the inability to match the query strategy. After entering S6, the system will perform guided retransmission, guided upload of payment information images, or transfer to manual processing. The system's "Guided Retransmission" feature prompts users to re-upload order information images, potentially providing specific guidance such as "Please upload a clearer screenshot of the order" or "Please ensure the image contains complete order information." This is typically used when the system fails to identify a unique order on the first attempt, giving users a chance to correct their mistakes. "Guided Payment Information Image Upload" means that if multiple attempts to upload order information images fail, the system will guide the user to upload payment information images. This is because payment information (such as transaction order number and / or merchant order number) is another important basis for order identification and can serve as a backup plan after order information image identification fails. "Transfer to Human Processing" means that if the automated processing attempts, including guided retransmission and guided payment information image upload, fail to identify the order after reaching a preset threshold, the system will notify the user that the order could not be found and transfer the issue to human customer service. This ensures that complex or abnormal situations can be handled manually, improving the problem resolution rate.

[0059] Through the above technical solution, when the key order information extracted from the order information image cannot match any preset information combination type, the system no longer blindly tries inapplicable query strategies, but directly enters the preset fallback processing flow S6. This processing mechanism ensures that even when the information recognition is incomplete or does not meet expectations, the system can provide clear and step-by-step guidance, such as prompting the user to re-upload the order information image, guiding them to upload a payment information image as a backup plan, or transferring the process to manual processing after multiple attempts. This effectively avoids process interruptions or user confusion caused by information mismatch, improves the robustness of the order locking method and user experience, and ensures effective response and processing of user requests in various complex scenarios.

[0060] Example 5 In this embodiment, the classification of user issues includes at least: confirming / verifying orders, denying / verifying orders, inability to provide order images, and other issues; Specifically, when the user's question is to confirm the order, the user is guided to enter a specific question and exit the order confirmation process; When a user's question is about denying order verification or other issues, the system will guide them to re-upload the order information image or transfer the user to a human agent based on the number of times the order information image has been uploaded and the number of times the payment information image has been uploaded. When a user's problem is that they cannot provide an order image, the decision will be made to transfer the case to a human operator or to continue guiding them to upload an order information image, depending on whether an image upload has already occurred at least once. When the user's problem is another issue, after completing or skipping the order lock, the intent recognition is performed and the problem handling strategy is generated by combining historical chat records and locked order information, and the corresponding handling instructions are output or the process is triggered to transfer to human processing.

[0061] Specifically, when the user's question is to confirm or verify the order, the user is guided to enter the specific question and exit the intelligent agent process for confirming the order, so that subsequent problem processing is executed based on the confirmed order information; When a user asks to verify order information, the system first checks the number of times the order information image has been uploaded. If the number of times the order information image has been uploaded is greater than or equal to 2, the system further checks the number of times the payment information image has been uploaded. If the number of times the payment information image has been uploaded is greater than or equal to 1, the system prompts that the order cannot be found even after providing attachments and transfers the user to a human for processing. Otherwise, the user is guided to re-upload the payment information image. If the number of times the order information image has been uploaded is less than 2, the user is guided to re-upload the order information image. When a user's problem is that they cannot provide an order image, the system checks the number of times the order information image has been uploaded: if the number of times the order information image has been uploaded is greater than or equal to 1, the process is transferred to a human operator; if the number of times the order information image has been uploaded is 0, the user is prompted to upload an order information image to assist with the query, and the session is terminated when the preset threshold has been reached. When the user's question is "other issues", first check the number of times the order information image has been uploaded: if the number of times the order information image has been uploaded is greater than or equal to 2, then check the number of times the payment information image has been uploaded. If the number of times the payment information image has been uploaded is greater than or equal to 1, then prompt the user that the consultation order still cannot be found after providing attachments and transfer the case to human assistance; otherwise, guide the user to re-upload the payment information image; if the number of times the order information image has been uploaded is less than 2, then guide the user to re-upload the order information image.

[0062] Through the above technical solution, this application provides a refined classification and differentiated processing strategy for various specific intentions that users may have during the order verification process in the image recognition-based ride-hailing order locking method. When a user's problem is clearly classified as "confirming order verification," "denying order verification," "unable to provide order image," or "other problems," the system can perform targeted guidance, retransmission, transfer to human processing, or provide specific handling instructions based on different classification results. For example, for confirmed orders, the system does not repeat the confirmation but directly guides the user to input specific questions, improving dialogue efficiency; for denial or generalized problems, it intelligently determines whether to guide retransmission or transfer to human processing based on the number of image uploads, avoiding unnecessary repetitive operations. In particular, after the order is locked, the system can combine historical chat records and locked order information to perform deeper intent recognition for "other problems," thereby generating personalized processing strategies and outputting accurate handling instructions, or triggering transfer to human processing when necessary. This sophisticated and intelligent interaction mechanism significantly enhances the user experience, reduces user confusion and waiting time in complex scenarios, effectively reduces the frequency of human customer service intervention, and improves the automation efficiency and accuracy of the entire order verification process.

[0063] The following example will provide a more detailed explanation of the above technical solution: User A had questions about the fare after completing a ride-hailing trip and wanted to verify the order through the platform's smart assistant.

[0064] S1. Enter the order verification session and be guided to upload: User A accesses the smart assistant chat interface through the platform application. The system determines that this is User A's first order verification request. To efficiently obtain order information, the system sends User A a guiding message: "Hello! Please upload a screenshot of your order information so that we can quickly verify it. You can refer to the following example image." This avoids the need for users to verbally or manually input large amounts of order information in the traditional customer service model, simplifying the initial interaction steps and reducing the complexity and potential errors of information collection.

[0065] S2. Receive user input and determine if it contains an image: Following the instructions, User A took a screenshot of their ride-hailing order details page on their phone and uploaded it to the chat interface. The system received User A's input and recognized that it contained the image to be identified.

[0066] S3. Image Classification and Recognition: The system inputs the picture to be recognized uploaded by User A into the picture classification and recognition module. This module analyzes and recognizes the picture content based on a vision-language model. The model recognizes that the picture is an "order information picture" because it contains typical order detail elements such as license plate number, pick-up time, pick-up location, drop-off location, etc. This step uses image recognition technology to directly process visual information, avoiding the transcription and parsing errors that may occur in traditional speech recognition (ASR) or natural language processing (NLP) when dealing with accents, noise, or unstructured text, and improving the accuracy of information recognition.

[0067] S4. Validity check of the order information picture: The system performs a validity check on the recognized order information picture. The check process includes: Check if the picture is the order page of this platform: The system confirms that the picture actually comes from this online car-hailing platform by recognizing specific UI elements and brand logos in the picture.

[0068] Check if it contains the license plate number: The system detects whether there is text information in the picture that conforms to the license plate number format.

[0069] Check if it contains the pick-up time: The system recognizes whether there is clear date and time information in the picture.

[0070] Check if it contains at least one of the starting point and the ending point: The system detects whether there is pick-up location or drop-off location information in the picture.

[0071] Assume that the picture uploaded by User A is clear and contains all necessary information, and the check passes. If the picture uploaded by User A for the first time is blurred, the system will output the specific reason for non-pass, such as "The picture you uploaded is blurred. Please re-upload a clear order screenshot", and give guidance on re-uploading. If User A uploads pictures that do not meet the requirements three times consecutively (reaching the preset threshold of 3 times), the system will output "You have uploaded pictures that do not meet the requirements multiple times. It is recommended that you re-find the corresponding order and then enter the session. This session will automatically end or be transferred to manual processing" to avoid an infinite loop and improve processing efficiency.

[0072] S5. Extract key order information and perform order retrieval: After the check passes, the system accurately extracts the key order information from the order information picture, including order number (such as "ORD123456789"), license plate number (such as "Beijing A12345"), pick-up time (such as "2023-10-26 14:35:22"), pick-up location (such as "Location A") and drop-off location (such as "Location B").

[0073] Based on the key order information extracted, the system determines the information combination type. For example, if the license plate number and the pick-up time accurate to the second are included in the picture, the system will identify the corresponding combination type.

[0074] The system calls the order query strategy corresponding to this information combination type to perform order retrieval.

[0075] Strategy example (information combination type: including license plate number and pick-up time accurate to the second or minute): Based on the extracted pick-up time "2023-10-26 14:35:22", the system generates a query time range, and both the start time and the end time are "2023-10-26 14:35:22".

[0076] The system queries the order based on the license plate number "Beijing A12345" and this time range.

[0077] Suppose the system retrieves a unique order through this query strategy. The system then generates the order details and conducts a confirmation and verification with User A: "We have locked an order for you: Order number [ORD123456789], license plate number [Beijing A12345], pick-up time [2023-10-26 14:35:22], pick-up location [Location A], drop-off location [Location B]. Is this the order?" This multi-field combination query strategy is significantly superior to the traditional single-field-dependent query method, improving the accuracy and success rate of order matching.

[0078] Strategy example (information combination type: the order information picture is the order list page and multiple candidate order information can be extracted): If the screenshot uploaded by User A is of the order list page and there is an order entry circled or highlighted by the user in the picture, the system will preferentially use the pick-up time, license plate number, starting point, and ending point information of this circled / highlighted entry to generate a query time range and conduct a query to quickly lock the unique order that the user is concerned about.

[0079] If no circled / highlighted entry is detected, but the picture contains the pick-up date or pick-up time of the order details page, the system will first filter the candidate orders with this information to form a smaller candidate set, and then conduct a query for each item in the filtered candidate set one by one. This ability to intelligently process the information on the list page further improves the order locking efficiency in complex scenarios.

[0080] Strategy example (information combination type: the order details page contains the starting point and the ending point, and the order list page contains the pick-up time accurate to the second or minute): If the system extracts information from both the order details page and the order list page at the same time, and the candidate order information extracted from the list page is unique, the system will use the pick-up time of this candidate order and the starting point and ending point of the details page for combined query.

[0081] If the candidate order information is not unique, and selected / highlighted items are detected, the start and end points of the selected / highlighted items and the usage time are used to generate a query time range for querying.

[0082] Otherwise, if the details page includes the ride time or origin and destination, use it to filter candidate order information. After filtering to the unique candidate, search for the order by origin, destination, and time range.

[0083] S6. Handling of situations where a unique order is not locked: If the system fails to find a unique order (e.g., due to insufficient image information or the existence of multiple similar orders), the system will perform a fallback based on the number of times the order information images have been uploaded: First attempt failed to lock: The system prompted user A to re-upload the order information image, prompting "Unable to lock the unique order. Please try re-uploading a clearer order screenshot or one containing more information." Second failure to lock: If user A is still unable to lock the unique order after uploading again, the system will guide user A to upload a payment information image, prompting "Still unable to lock the unique order. You can try uploading a screenshot of the payment information, such as WeChat / Alipay payment voucher." Reaching a preset threshold (e.g., 3 attempts): If the system fails to locate the unique order after multiple attempts, it will display the message, "Sorry, your order could not be found. You have been transferred to a human customer service representative." This multi-stage fallback mechanism ensures that when automated processing fails to resolve the issue, the user can be promptly transferred to human assistance, improving the consistency of the user experience.

[0084] S7. Preconditions for processing payment information images: Suppose user A uploads a payment information image under the guidance of system S6. After system S3 recognizes it as a payment information image, it will determine whether the precondition for processing the payment information image is met based on the number of times the order information image has been uploaded. By default, this precondition is that the number of order information image uploads is greater than or equal to 2. Since user A has already uploaded order information images twice, this condition is met. If user A directly uploads a payment information image without having uploaded an order information image before, the system will prompt "The payment information image does not meet the current processing requirements; please upload the order information image first," ensuring the rationality of the processing flow.

[0085] S8. Payment Information Image Processing and Order Inquiry: When the preconditions are met, the system extracts the transaction order number and / or merchant order number from the payment information image. For example, it extracts the Alipay transaction number "20231026XXXXXXXX". The system then queries the ride-hailing platform's backend database based on this transaction order number. If a unique order is found, the system generates order details and verifies them with user A. If no unique order is found, the system outputs a prompt and transfers the case to manual processing. This provides users with another way to secure their orders, increasing the success rate of order verification.

[0086] S9. Classification and handling of user issues when there is no image input: Suppose user A does not upload an image at the start of the session, but instead directly enters "I can't find my order". The system determines in S2 that the image to be identified is not included, and then proceeds to S9.

[0087] The system uses a generative model combined with historical chat records to classify user questions.

[0088] The user's question is "Confirming and verifying the order": If, after confirming and verifying the order in S5 or S8, user A replies "Yes, this is the order", the system will guide user A to describe the specific problem (e.g., "Do you have any specific questions about this order?"), and exit the order confirmation process to enter the problem-solving stage.

[0089] For user issues such as "denying order verification" or other problems: If user A replies "not this order", the system will guide the user to re-upload the order information image or transfer the user to a human for processing, based on the number of times the order information image has been uploaded and the number of times the payment information image has been uploaded.

[0090] The user's problem is "unable to provide order image": If user A says "I cannot provide order image", the system will decide whether to transfer the case to manual processing or continue to guide the user to upload order information image, depending on whether at least one image upload has already occurred.

[0091] For user issues categorized as "Other Issues": If user A's issue is "I want to complain about the driver," after completing or skipping the order lock, the system will perform intent recognition and, combined with historical chat records and locked order information, generate an issue handling strategy, outputting corresponding handling instructions or triggering a transfer to human assistance. This ability to intelligently classify and handle user issues other than image-related ones further enhances the comprehensiveness of the intelligent assistant and the user experience, reducing reliance on human customer service.

[0092] Through the above specific examples, this technical solution uses image recognition technology to achieve automated and high-precision extraction of order information. Combined with various intelligent query strategies and a multi-stage fallback mechanism, it effectively solves the problems of low information collection efficiency, poor accuracy, cumbersome user interaction, and reliance on single fields in traditional methods, significantly improving the efficiency of ride-hailing order verification and user experience.

[0093] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0094] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0095] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0096] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0097] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0098] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0099] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0101] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An image recognition-based online car-hailing order locking method, characterized in that, include: S1. Enter the order verification session based on the user's request, and determine whether the current session is the first inquiry; If this is the first question, then send guidance information to the user to upload an image of the order information and provide a sample image; S2. Receive user input and determine whether the user input contains the image to be recognized; if it contains the image to be recognized, proceed to S3; if it does not contain the image to be recognized, proceed to S9. S3. Input the image to be identified into the image classification and recognition module, and identify the image category based on the visual language model; If the image is identified as an order information image, proceed to S4; if the image is identified as a payment information image, proceed to S7. S4. Perform validity validation on the order information image; proceed to S5 if the validation passes. When the verification fails, output the reason for the failure based on the number of times the order information image has been uploaded and guide the user to upload it again, or output a prompt and end the session or transfer the request to manual processing when the preset number of uploads is reached. S5. Extract key order information from the order information image, including at least one or more of the following: order number, license plate number, vehicle usage time, pick-up location, and drop-off location; determine the information combination type based on the extracted key order information, and call the order query strategy corresponding to the information combination type to perform order retrieval; if a unique order is found, generate order details and confirm with the user. If no unique order is found, proceed to step S6; S6. When a unique order is not locked, a fallback procedure is performed based on the number of times the order information image has been uploaded: if the order is not locked for the first time, the user is guided to re-upload the order information image; if the order is not locked for the second time, the user is guided to upload the payment information image; if the preset threshold for the number of attempts is reached, the user is prompted that no order can be found and the process is transferred to a human for manual processing. S7. When the image category is a payment information image, determine whether the preconditions for processing payment information images are met based on the number of times order information images have been uploaded. If the conditions are met, proceed to S8. If the conditions are not met, output a prompt and guide the upload of order information images or end the session when the preset number of uploads is reached. S8. Extract the transaction order number and / or merchant order number from the payment information image, and query the order based on the transaction order number and / or merchant order number; When a unique order is found, order details are generated and confirmed with the user. If no unique order is found, a prompt is displayed and the process is transferred to a human operator. S9. Use generative models combined with historical chat records to classify user questions, and perform corresponding guidance, supplementary information collection or transfer to manual processing based on the classification results; The information combination types include: the order information image is an order list page and multiple candidate order information can be extracted; the candidate order information includes at least one or more of the following: license plate number, vehicle usage time, origin and destination; The corresponding order query strategy is as follows: When a circled / highlighted item is detected in an image, the vehicle usage time of the circled / highlighted item is used first to generate a query time range, and the query is performed based on the license plate number and the time range. Then, the start and end points of the circled / highlighted item are used to filter for a unique order. If no selected / highlighted item is detected, and the image also contains the vehicle usage date or time from the order details page, the candidate order information is first filtered based on the vehicle usage date or time to form a smaller candidate set. Generate query time ranges for each item in the filtered candidate set or the original candidate set, perform license plate number query and time range query, and filter unique orders based on the start and end points of the corresponding items; The information combination types also include: the order details page includes the origin and destination, and the order list page includes the car rental time accurate to the second or minute; The corresponding order query strategy is as follows: When the candidate order information extracted from the list page is unique, the vehicle usage time of the candidate order is used to generate a query time range, and the order is queried by origin, destination and time range; When the candidate order information is not unique, if a selected / highlighted item is detected, the start and end points of the selected / highlighted item and the usage time are used to generate a query time range for querying; Otherwise, if the details page contains the ride time, it is used to filter candidate order information; or if the details page contains the origin and destination, it is used to filter candidate order information. After filtering out a unique candidate, the order is then queried by origin, destination and time range. If no unique order is locked, proceed to step S6.

2. The method of claim 1, wherein, The validity verification of order information images should at least include: Verify that the image is from an order page on this platform, contains a license plate number, contains the rental time, and contains at least one of the following: origin and destination. When the verification fails, output the specific reason for the failure and provide guidance on supplementing / replacing the uploaded content; The prerequisite for processing the payment information image in S7 is that the number of times the order information image has been uploaded is greater than or equal to 2. If this condition is not met, the user will be prompted that the payment information image does not meet the current processing requirements, and the user will be guided to upload the order information image first.

3. The method of claim 1, wherein, The preset threshold number of times is 3 times; When the number of consecutive uploads of order information images does not meet the requirements and reaches the threshold, a prompt message indicating that multiple uploads do not meet the requirements will be output, and the user will be advised to find the corresponding order again before entering the session. The user will also be asked to actively end the session or transfer the case to a human for processing.

4. The method of claim 1, wherein, The information combination types include: order information images containing order numbers that conform to the order numbering rules of this platform; The corresponding order query strategy is as follows: use the order number as the search key to query the order. If the order is found, proceed to confirmation and verification. If the order is not found, proceed to step S6.

5. The method of claim 1, wherein, The information combination types include: order information images containing license plate numbers and vehicle usage time accurate to the second or minute; The corresponding order query strategy is as follows: A query time range is generated based on the vehicle usage time; when the vehicle usage time is accurate to the second, the start time and end time of the time range are the same; when the vehicle usage time is accurate to the minute, the start time is 00 seconds appended after the hour and minute, and the end time is 59 seconds appended after the hour and minute. Query orders based on license plate number and the time range; If a unique order is found, proceed to a confirmation and verification step. If multiple orders are found, when an order entry with a circled / highlighted mark is detected in the image, further filtering is performed based on the field of the circled / highlighted mark to obtain a unique order; otherwise, it is considered that the unique order has not been locked and proceeds to step S6.

6. The method of claim 1, wherein, The information combination types include: order information images containing the ride time, origin, and destination accurate to the second or minute; The corresponding order query strategy is as follows: Generate a query time range based on the vehicle usage time, and query orders by origin, destination and time range; If a unique order is found, proceed to a confirmation and verification step. If no unique order is found, the candidate order information extracted from the order list page is filtered a second time by the usage time, origin and destination. If a unique candidate is found and the usage time is accurate to the second, a second query is performed with the time range accurate to the second to lock the unique order. Otherwise, proceed to step S6.

7. The method of claim 1, wherein, When the key information of the order does not meet any of the preset information combination types, the order query strategy is to enter step S6 to perform guided retransmission, guided upload of payment information images, or transfer to manual processing.

8. The method according to claim 1, characterized in that, The categories of user issues should at least include: confirming / verifying orders, denying / verifying orders, inability to provide order photos, and other issues; Specifically, when the user's question is to confirm the order, the user is guided to enter a specific question and exit the order confirmation process; When a user's question is about denying order verification or other issues, the system will guide them to re-upload the order information image or transfer the user to a human agent based on the number of times the order information image has been uploaded and the number of times the payment information image has been uploaded. When a user's problem is that they cannot provide an order image, the decision will be made to transfer the case to a human operator or to continue guiding them to upload an order information image, depending on whether an image upload has already occurred at least once. When the user's problem is another issue, after completing or skipping the order lock, the intent recognition is performed and the problem handling strategy is generated by combining historical chat records and locked order information, and the corresponding handling instructions are output or the process is triggered to transfer to human processing.

Images