Remote ordering method and system based on real-time video interaction and computer readable storage medium

Abstract

The invention discloses a remote ordering method and system based on real-time video interaction and a computer readable storage medium, and belongs to the technical field of electronic commerce. According to the method, a platform server (30) coordinates establishment of a real-time audio and video communication session between a customer terminal (10) and a merchant terminal (20). In a video call, a customer can check commodities displayed by a merchant in real time, and an instruction is sent out in a voice mode and the like. An integrated interface of the merchant terminal (20) allows for the addition of a preset item or the entry of non-preset temporary item attributes (name, price, etc.) to the electronic order through a dedicated module during the duration of the video call. The platform server (30) manages order data, including structured storage of temporary commodities, and synchronizes an order state to both terminals in real time. According to the invention, the problems of non-transparent information, low interaction efficiency and incapability of flexibly processing non-standard products in the prior art are solved, and the transparency, efficiency and user experience of remote shopping are improved through seamless integration of video interaction and order operation.

Description

1. TECHNICAL FIELD

[0001] The present application relates to the technical field of network communication and electronic commerce, and specifically relates to an order generation method, system and related terminal equipment for a remote shopping scenario, which integrates real-time video communication, dynamic product display and selection, and flexible processing of preset and temporary (non-preset) products. 2. BACKGROUND

[0002] Existing online ordering platforms (such as food delivery platforms and fresh food e-commerce platforms) mainly rely on static text and image information to display products. This mode has significant technical defects, limiting user experience and transaction efficiency:

[0003] • Information lag and opacity: static information cannot reflect the key status of products in real time, such as the freshness of fresh products, the daily supply situation of seasonal products, or the real-time inventory style of specific products. Users are like "blind selection" and it is difficult to make accurate judgments, especially for non-standard products and experience-sensitive products.

[0004] • Lack of processing capacity for non-standard products: existing systems are usually built around a preset product database, and lack effective presentation, interaction, and pricing mechanisms for bulk food priced by weight, or products that require on-site selection of colors or personalized customization (such as special taste requirements).

[0005] • Interaction and operation process is fragmented: although some platforms provide text or voice communication channels, these communication methods are separate from the order modification and confirmation process. Users need to switch between communication interfaces and order interfaces, which is cumbersome and cannot achieve a smooth experience of "what you see is what you get".

[0006] • Integration barrier of underlying technology: existing technologies have not effectively solved the data interaction and state synchronization problem between real-time communication technology and order processing system, lack of order data model designed specifically for accommodating temporary and unstructured product information, and lack of integrated user interface technology solution that can seamlessly select products and modify orders while maintaining video communication.

[0007] Therefore, there is an urgent need in the art for a technical solution that seamlessly combines real-time visual verification, efficient communication, and flexible order operation (especially for handling non-preset products) to overcome the shortcomings of existing technologies. 3. SUMMARY

[0008] (1) Technical problem to be solved:

[0009] The application aims to provide a remote ordering technical solution based on real-time video interaction, to solve the technical problems of existing technology, such as non-transparent commodity information, low interaction efficiency, insufficient support for non-preset commodities, and the separation of interaction and operation process, and to realize an efficient, reliable and flexible remote shopping experience that supports preset and temporary commodity ordering.

[0010] (2) Technical solution:

[0011] Aspect one: a remote ordering method based on real-time video interaction, characterized by the following steps:

[0012] Step A (video session establishment): the customer terminal (10) sends a video call establishment request containing the target merchant identifier to the platform server (30) through the application program interface (API); the platform server (30) verifies the request and pushes the call invitation to the merchant terminal (20) of the target merchant.

[0013] Step B (channel negotiation): after the merchant terminal (20) confirms acceptance, the platform server (30) coordinates the establishment of a bidirectional audio and video stream transmission channel (50) based on the real-time transmission protocol (preferably using RTP / RTCP or WebRTC technology) between the customer terminal (10) and the merchant terminal (20), realizing real-time video call.

[0014] Step C (real-time interaction and instruction acquisition): during the video call, the customer terminal (10) receives and displays real-time video streams from the merchant terminal (20), and the operator of the merchant terminal (20) displays the on-site goods according to the customer's instructions or independently; at the same time, the merchant terminal (20) or the platform server (30) receives the commodity selection instructions sent by the customer through voice, text message or video screen marking, etc.

[0015] Step D (adding goods to order): the operator of the merchant terminal (20) responds to the commodity selection instruction, and in the state of maintaining video call, executes at least one of the following commodity adding actions through the integrated user interface (UI) configured by it to add commodity information to an electronic order data associated with the current video call session: Figure 4

[0016] D1 (add preset commodity): trigger the query of the platform server (30) background preset commodity database (35) or the call of local cache, select the preset commodity item corresponding to the customer's instruction, and add the preset commodity information (such as commodity ID, specification, unit price) to the electronic order data.

[0017] D2 (add temporary commodity): activate a temporary commodity information input module (36, Figure 5 ​), receiving operator input for defining attribute information of a non-preset item (e.g. item name (5a), price (5b) or pricing rules, optional quantity / weight or description (5c)); the system generates a unique identifier (TID) for the temporary item, and adds temporary item data containing the unique identifier and the attribute information to the electronic order data.

[0018] • Step E (Order Status Synchronization): Changes in the electronic order data (e.g. addition of item, update of total price) are processed by the order management core module (34) of the platform server (30), and are synchronized in real-time or near real-time to the interfaces (e.g. area 4b) of the customer terminal (10) and the merchant terminal (20) for display through a low-latency communication mechanism (e.g. WebSocket).

[0019] • Step F (Order Confirmation and Subsequent Processes): Before or after the video call ends, the electronic order is finally determined according to the preset process (e.g. customer confirmation at terminal (10)), and the platform server (30) triggers subsequent payment and order fulfillment processes (e.g. notifies the background warehouse or delivery system).

[0020] Aspect Two: A merchant terminal (20) for performing the merchant-side operations in the above method, characterized in that it comprises:

[0021] • at least one processor (21);

[0022] • a memory (22) storing instructions executable by the processor;

[0023] • a communication interface (23) for establishing network connections with the platform server (30) and the customer terminal (10);

[0024] • audio / video input / output devices (24) (e.g. camera, microphone, speaker);

[0025] • a display screen (25);

[0026] • the instructions, when executed by the processor, cause the merchant terminal (20) to perform the following functions:

[0027] ■ receive a video call invitation through the communication interface (23) and participate in establishing a real-time audio / video communication session;

[0028] ■ perform real-time audio / video interaction through the audio / video devices (24), and transmit the live picture captured by the camera after encoding;

[0029] ■ present an integrated user interface (UI) on the display screen (25) Figure 4), the interface at least contains: real-time video display area (4a), order summary display area (4b), and commodity addition operation control area (4c), allowing order operation during video call duration;

[0030] ■in response to the addition of preset commodity instruction received through the operation control, calling the preset commodity data and generating an addition request;

[0031] ■in response to the addition of temporary commodity instruction received through the operation control, activating the temporary commodity input module ( Figure 5 ) to receive attribute input, and generating an addition request containing temporary commodity information;

[0032] ■send the commodity addition request to the platform server (30) through the communication interface (23), and receive and display order status updates.

[0033] Aspect three: a remote ordering system based on real-time video interaction, characterized in that it comprises:

[0034] ● the merchant terminal (20) as described in aspect two;

[0035] ● the customer terminal (10) is configured with the functions of real-time video call with the merchant terminal (20), viewing orders, issuing commodity selection instructions and confirming orders;

[0036] ● the platform server (30) is configured with:

[0037] ■user account management module (31) and merchant account management module (32);

[0038] ■video call signaling processing and media stream negotiation / relay management module (33);

[0039] ■order management core module (34) for: creating and maintaining electronic order instances for video ordering sessions; receiving addition commodity requests (containing preset commodity IDs or temporary commodity detailed data) from the merchant terminal (20); verifying and persistently storing electronic order data;

[0040] ■preset commodity database (35);

[0041] ■temporary commodity data storage / management module (36) for structured processing and storage of temporary commodity information and its association with specific orders (for example, using independent data table or using JSON / JSONB field in order details table);

[0042] ■order status synchronization module for pushing order status updates to the customer terminal (10) and the merchant terminal (20);

[0043] ■ Payment interface and back-end execution system interface module.

[0044] Aspect 4: A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the steps of the method as described in aspect 1.

[0045] (3) Beneficial effects:

[0046] This invention offers significant technical advantages through the deep integration of real-time video interaction and order processing:

[0047] ●Enhancing Information Transparency and Trust: Real-time video streaming allows customers to view the actual condition of products (especially fresh produce, non-standardized products, and customized items) as if they were physically present, achieving "what you see is what you get," significantly reducing information asymmetry and boosting confidence in purchasing decisions. Information latency is expected to be reduced from minutes to sub-seconds.

[0048] ● Improved Communication and Operational Efficiency: Real-time audio and video interaction is more intuitive and efficient than traditional text, images, or pure voice methods, enabling quick and accurate understanding of customer needs and reducing misunderstandings. The integrated user interface supports simultaneous video communication and order processing, eliminating the need for interface switching. Simulation tests show that it can improve order placement and processing efficiency by approximately 30%-60%.

[0049] ●Enhancing transaction flexibility and expanding sales opportunities: Through a structured temporary goods handling mechanism, merchants can easily handle goods outside the preset menu, such as selling daily specials, near-expiry specials, and handling customized customer requests. They can also set prices in real time based on the situation on-site (e.g., weighing) or adopt specific pricing rules, providing merchants with an effective technical means to promote and sell immediacy and special goods online.

[0050] ●Optimize the user shopping experience: It provides a novel and highly interactive remote shopping method that meets consumers' needs for personalized services and an intuitive shopping process.

[0051] ● Provides reliable technical implementation and standardized data management: Adopting standardized real-time communication protocols (such as WebRTC) ensures call stability and cross-platform compatibility. The structured data processing and storage mechanism for temporary goods, integrated into the standardized order management process, facilitates subsequent statistical analysis, traceability, and system maintenance, representing an effective technical improvement to the existing order system. 4. Description of the attached drawings:

[0052] ● Figure 1 The system network topology diagram of an embodiment of the present invention is shown, illustrating the basic connection relationship and main data flow (video stream, signaling stream, order data stream) between the customer terminal (10), the merchant terminal (20), and the platform server (30).

[0053] ● Figure 2 A detailed flowchart of one embodiment of the method of the present invention is shown, including the main steps (AF) from initiating a video call to order completion and its branch logic (such as processing preset products and temporary products).

[0054] ● Figure 3 A functional block diagram of an embodiment of the merchant terminal (20) of the present invention is shown, including hardware components (processor (21), memory (22), communication interface (23), audio and video device (24), display screen (25)) and key software modules (such as video communication module (26), order management module (27), and product input module (28)).

[0055] ● Figure 4 A schematic diagram of the operation interface of an embodiment of the merchant terminal (20) of the present invention is shown, including interface partitions: (4a) real-time video display area; (4b) current order product list and total price area; (4c) operation button area, including buttons such as "add product", "mute", "flip camera", "end call", and "confirm order".

[0056] ● Figure 5 The diagram illustrates a portion or tab of a sub-interface or pop-up window triggered by the "Add Product" operation in an embodiment of the present invention, used for adding temporary products, including a product name input box (5a), a price input box (5b) (or a unit price and quantity / weight input box), an optional description input box (5c), and a confirm add button (5d).

[0057] ● Figure 6 A functional block diagram of an embodiment of the platform server (30) of the present invention is shown, including key modules such as user / merchant management (31, 32), video session management (33), order processing core (34), preset product library (35), and temporary product data storage / management (36) and their interaction relationships. 5. Detailed Implementation:

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0059] Example 1: Remote video ordering for a seafood restaurant based on WebRTC

[0060] Reference Figures 1 to 6 In this embodiment, Alice, a customer terminal (10) user (e.g., a smartphone with a specific application installed), wants to purchase seafood from Bob's seafood restaurant that day.

[0061] ●Process Initiation and Connection Establishment: Alice selects Bob's restaurant in the APP and initiates a "video order" request. This request is sent to the platform server (30) via HTTPS protocol in the form of a RESTful API. The request may contain authentication information such as JWT. After verification, the platform server (30) pushes the call invitation to Bob's restaurant's merchant terminal (20) (e.g., a tablet or smart POS machine) via WebSocket. After Bob accepts the request, the platform server (30) coordinates the two parties through a signaling server (which may use SIP or custom signaling) to establish an end-to-end encrypted audio and video stream channel using WebRTC's ICE protocol (STUN / TURN) (using H.264 / VP8 video encoding and Opus audio encoding).

[0062] ●Video Interaction and Product Selection: Alice sees a seafood tank displayed by Bob through the camera on the merchant terminal (20) on her phone screen. Alice gives a voice instruction to buy a Boston lobster.

[0063] ● Temporary Product Addition: Bob retrieves the selected lobsters, weighs them (e.g., 1.2kg), and then displays them on the integrated interface of the merchant terminal (20). Figure 4 In area (4c), click the "Add Temporary Item" button. A system pop-up will appear as follows: Figure 5 The input interface is shown. Bob enters the name "Boston Lobster - Freshly Caught" (5a), selects the pricing method as "unit price", enters the unit price "300 yuan / kg", and enters the weight "1.2kg" (5b). The system can automatically calculate the price or Bob can confirm the total price and click the confirm button (5d). The merchant terminal (20) packages this temporary product information (including the automatically generated unique temporary product ID, name, pricing method, unit price, weight, total price, etc.) into a JSON object format and sends it to the platform server (30) through HTTPS API (e.g., POST / orders / {sessionId} / items / temp).

[0064] ● Order Update and Synchronization: Upon receiving the request, the order processing core module (34) of the platform server (30) verifies the validity of the data and adds an item to the order instance associated with this video session. This temporary product information is stored in a structured manner (e.g., in a JSONB type field of the order details table, or associated with a separate temporary product table). The platform server (30) then pushes the updated order overview (including the product list and total price) to the customer terminal (10) and the merchant terminal (20) via WebSocket, and the two boundaries (e.g., regions (4b)) are refreshed and displayed in real time.

[0065] ● Adding a Preset Product: Next, Alice ordered a "Steamed Scallops with Garlic and Vermicelli" dish from the menu. Bob clicked "Add Preset Product" on the interface (4c) and selected the dish from the pop-up product list. The merchant terminal (20) sent the ID and quantity of the preset product (e.g., after obtaining details via GET / products / {id}, POST / orders / {sessionId} / items / preset) to the platform server (30). The platform server (30) queried the preset product library (35) to obtain the product details, updated the order instance, and synchronized the updated order information to both terminals again.

[0066] ● Order Confirmation and Completion: Finally, both parties confirm that the order is correct. Alice clicks the "Confirm Order" button on the customer terminal (10). The platform server (30) marks the order status as pending payment and may call the payment gateway interface. After successful payment, the order status is updated and the merchant's back-end system (such as the kitchen printing system) is notified to start food preparation.

[0067] Example 2 (Technical Variations and Extensions):

[0068] In other embodiments of the present invention, the following changes may exist:

[0069] ●Video Initiator: Merchants can proactively initiate video calls to customers who are already online, for the purpose of promoting special products or providing after-sales service.

[0070] ● Temporary Goods Definition Extension: The price of temporary goods does not have to be a fixed value. Instead, a pricing rule (such as "current price" or "floating according to market price") is selected. The final price is not determined when the goods are added to the order. It is updated and confirmed after the goods are prepared (such as after weighing).

[0071] ●Inventory linkage: For preset products, the merchant terminal (20) can call the inventory query interface provided by the platform server (30) in real time when adding them, and give a prompt if the inventory is insufficient. For temporary products (such as weighed seafood), the virtual inventory (if any) in the background can be deducted after successful addition.

[0072] ●Enhanced Interaction: In addition to voice, customers can also indicate selected products by circling or marking on the video screen. The merchant terminal (20) can use image recognition technology to help understand the customer's intentions. WebRTC's DataChannel can be used to transmit such labeled data.

[0073] ● Diverse terminal types: Merchant terminals (20) can be dedicated smart POS devices that integrate cameras, touch screens, printers, and POS payment functions. Customer terminals (10) can be web browsers, mini-programs, or native apps.

[0074] ● Diverse technical implementations: The front-end can use frameworks such as React, Vue, and Angular; the back-end can use languages ​​such as Java, Python, and Node.js, and can be deployed as a microservice architecture to improve scalability and reliability. Database options include PostgreSQL (using JSONB), MySQL, and MongoDB.

[0075] ● Network adaptability: It can implement a video bitrate adaptive algorithm to dynamically adjust video quality according to network bandwidth and ensure smooth calls.

[0076] ●High Availability and Fault Tolerance: The system design must consider abnormal situations such as video call interruptions, network partitions, and server failures. Mechanisms such as state machine management, data redundancy, and transaction compensation should be used to ensure state recovery and data consistency. Order data updates can employ optimistic locking or distributed transaction solutions.

Claims

1. A remote order placement method based on real-time video interaction, characterized in that, Includes the following steps: ●A real-time audio and video communication session based on the real-time transmission protocol is established between the customer terminal (10) and the merchant terminal (20) through the coordination of the platform server (30); ●The customer terminal (10) receives and displays the real-time video stream transmitted by the merchant terminal (20); ●The merchant terminal (20) receives a product selection instruction from the customer terminal (10) based on the real-time audio and video communication session; ● In response to the product selection instruction, the merchant terminal (20), during the duration of the real-time audio and video communication session, performs at least one of the following operations through its configured user interface to add product information to the electronic order data associated with the real-time audio and video communication session: ■ Add preset products: Based on the instruction, call the preset product database (35) of the platform server (30) or the local cache to obtain preset product information and add it to the electronic order data; ■ Add Temporary Product: Receive attribute information input for defining at least one temporary product, which is not predefined in the preset product database (35) of the platform server (30), and the attribute information includes at least product identification information and price information or pricing rules; generate a unique identifier for the temporary product, and add the temporary product data containing the unique identifier and the attribute information to the electronic order data; ●The platform server (30) updates the electronic order data according to the addition operation of the merchant terminal (20), and synchronizes the update information of the order status during the session to at least one of the customer terminal (10) and the merchant terminal (20).

2. The method according to claim 1, characterized in that, The step (b) of receiving attribute information input for defining temporary goods includes receiving text, numbers or option selections input by the operator through the graphical user interface (FIG. 5) configured on the merchant terminal (20) to determine the name (5a), price (5b) or pricing rules of the temporary goods, and optional quantity, weight or description (5c).

3. The method according to claim 1 or 2, characterized in that, The attribute information of the temporary goods is stored in a structured manner in the electronic order data, for example, using JSON format data.

4. The method according to claim 1, characterized in that, It also includes the step of receiving an order confirmation instruction from the customer terminal (10) or the merchant terminal (20) during or after the real-time audio and video communication session, and triggering subsequent payment processing or order fulfillment process based on the instruction.

5. The method according to claim 1, characterized in that, The customer terminal (10) sends product selection instructions in at least one of the following ways: voice instructions, text messages, or annotations on the displayed real-time video stream.

6. The method according to claim 1, characterized in that, The real-time audio and video communication sessions are established and managed using WebRTC technology.

7. A merchant terminal (20), characterized in that, include: ● Processor (21); ● Memory (22) stores instructions that are executed by the processor (21); ●Communication interface (23); ● Audio and video input / output devices (24); ● Display screen (25); ● When the instruction is executed by the processor (21), the merchant terminal (20) performs the following functions: ■ Establish a connection with the platform server (30) and customer terminal (10) through the communication interface (23) and participate in real-time audio and video communication sessions; ■ Audio and video interaction and video capture are performed through the audio and video input / output device (24); ■ A user interface (Figure 4) is displayed on the display screen (25), the user interface being configured to: ◆ Displays a video stream (4a) from the customer terminal (10) or captured by its own camera; ◆Receive user input to select adding preset product information to the electronic order data associated with the session; ◆ Provide an interactive control (4c, Figure 5) for inputting at least one temporary product attribute information, and use the acquired attribute information to generate an add request; ◆ Displays the current status of the electronic order data (4b); ■ The request generated by the product addition operation is sent to the platform server (30) through the communication interface (23).

8. The merchant terminal (20) according to claim 7, characterized in that, The user interface (Figure 4) integrates the video display area (4a), the order summary display area (4b), and the operation control area (4c) into the same screen view.

9. A remote ordering system based on real-time video interaction, characterized in that, include: ●The merchant terminal (20) as described in claim 7 or 8; ●The customer terminal (10) is configured to conduct real-time audio and video communication sessions with the merchant terminal (20), send product selection instructions, receive order status updates, and confirm orders; ● Platform server (30), configured as follows: ■ Coordinate the establishment and management of the real-time audio and video communication sessions; ■ Create and maintain electronic order data for the session; ■ Receive and process the add preset product request and add temporary product request from the merchant terminal (20), wherein the add temporary product request includes the structured attribute information of the temporary product; ■ Includes a preset product database (35) and a temporary product data storage mechanism (36) for storing and managing the temporary product attribute information and its association with specific orders; ■ Persistently store the electronic order data in the database; ■ Synchronize the order status update to the customer terminal (10) and / or the merchant terminal (20); ■ Handling final order confirmation and subsequent processes.

10. The system according to claim 9, characterized in that, The temporary commodity data storage mechanism (36) of the platform server (30) is implemented in at least one of the following ways: using JSON or JSONB type fields in the order details table of a relational database; or using a separate NoSQL database table.

11. A computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 6.

Images