Remote interaction method, apparatus, device, and program product

Abstract

The application discloses a remote interaction method, device, equipment and program product, aiming at solving the problem that the existing touch interaction scheme based on video call excessively depends on terminal touch screen hardware capability, resulting in that a large number of non-touch screen terminals cannot be intuitively remotely operated. The scheme is applied to an intermediate node for establishing a video call between a target terminal and a customer service system, and after the target terminal and the customer service system establish a video call supporting touch interaction, a key signal of a dial pad from the target terminal is received; then, based on the key signal, a corresponding touch interaction instruction is determined; and then, the touch interaction instruction is sent to the customer service system, so that the customer service system executes the touch interaction instruction in the video call.

Description

Technical Field

[0001] This application relates to the field of remote interaction technology, and in particular to a remote interaction method, apparatus, device, and program product. Background Technology

[0002] In remote interactive applications based on communication networks, especially in customer service video call scenarios, remote interaction typically refers to the ability of users to perform real-time operations and provide feedback to a remote service system via terminal devices. In recent years, with the popularization of video communication technology, some customer service systems have supported touch interaction during video calls. For example, users can directly click or swipe on the touchscreen terminal, and the system can respond in real time and jump to the corresponding business interface, thus achieving a intuitive "what you see is what you get" operating experience.

[0003] However, current video call solutions that support touch interaction typically rely on the terminal having a touchscreen capability. For a large number of communication terminals that lack touch functionality (such as non-touchscreen mobile phones with only a physical dial pad), users cannot directly click or swipe on the screen, thus preventing them from enjoying the convenience of touch interaction and limiting the coverage of video customer service and the consistency of user experience. Therefore, how to enable non-touchscreen terminals to achieve touchscreen-like interactive operations in video calls has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application proposes a remote interaction method, apparatus, device, and program product, aiming to solve the problem that existing touch interaction solutions based on video calls rely excessively on the terminal's touchscreen hardware capabilities, resulting in a large number of non-touchscreen terminals being unable to perform intuitive remote operations. Accordingly, the technical solution of this application is as follows: In a first aspect, embodiments of this application provide a remote interaction method applied to an intermediate node for establishing a video call between a target terminal and a customer service system, the method comprising: After a video call is established between the target terminal and the customer service system, key signals from the dial pad of the target terminal are received; wherein, the video call supports touch interaction; Based on the button signals, the corresponding touch interaction commands are determined; Send a touch interaction command to the customer service system so that the customer service system executes the touch interaction command during the video call.

[0005] Secondly, embodiments of this application provide a remote interaction device used as an intermediate node for establishing video calls between a target terminal and a customer service system. The device includes: A key signal receiving module is used to receive key signals from the dial pad of the target terminal after a video call is established between the target terminal and the customer service system; wherein the video call supports touch interaction. A touch interaction command generation module is used to determine the corresponding touch interaction command based on the button signal; The touch interaction command transmission module sends touch interaction commands to the customer service system so that the customer service system executes the touch interaction commands during the video call.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a computer program product, the computer program product including a computer-readable storage medium storing a computer program operable to cause a computer to perform the method described in the first aspect.

[0008] This application embodiment is applied to an intermediate node for establishing a video call between a target terminal and a customer service system. After the video call is established between the target terminal and the customer service system, it can receive key signals from the target terminal's dial pad, parse the key signals into corresponding touch interaction commands, and then send the touch interaction commands to the customer service system to drive execution in the video call. This solution maps general dial pad input to standard touch commands, enabling terminals without touch screen hardware to simulate clicks, swipes, and other interactive operations using buttons. Thus, without modifying the terminal hardware, it achieves the technical effect of remote touch control of non-touch screen terminals in video calls that support touch interaction. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the first type of remote interaction method according to an embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the application architecture of the remote interaction method according to an embodiment of this application.

[0012] Figure 3 This is a schematic diagram of a second type of remote interaction method according to an embodiment of this application.

[0013] Figure 4This is a schematic diagram of the business interface used in the remote interaction method of this application embodiment.

[0014] Figure 5 This is a schematic diagram illustrating the remote interaction method for determining the sliding trajectory according to an embodiment of this application.

[0015] Figure 6 This is a schematic diagram of the remote interaction method for sliding the business interface according to an embodiment of this application.

[0016] Figure 7 This is a schematic diagram of the structure of a remote interaction method apparatus according to an embodiment of this application.

[0017] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] As mentioned earlier, current video call solutions supporting touch interaction typically rely on the terminal having touchscreen capabilities. However, for many non-touchscreen communication terminals equipped only with physical dial pads, users cannot directly click or swipe on the screen. This not only prevents them from enjoying the convenience of touch interaction but also limits the user coverage and consistency of video customer service. Therefore, how to enable non-touchscreen terminals to achieve touchscreen-like interactive operations during video calls has become an urgent technical problem to be solved.

[0020] In view of this, this application proposes a remote interaction method, apparatus, device, and program product, aiming to solve the problem that existing solutions cannot provide intuitive remote interaction for non-touchscreen terminals due to excessive reliance on terminal touchscreen hardware. The technical solutions provided by various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0021] One embodiment of this application provides a remote interaction method applied to an intermediate node in establishing a video call between a target terminal and a customer service system. In this embodiment, the target terminal refers to a communication terminal without a touchscreen, equipped only with a physical dial pad and other input devices; the customer service system refers to a business processing system deployed on the server side that supports video calls and touch interaction responses; and the intermediate node refers to a gateway or server device located in the communication network, responsible for signaling and media stream connection and processing. In this application scenario, a user initiates a video call to the customer service system through the target terminal. After the call is established, the intermediate node receives key signals from the terminal's dial pad, parses and converts them into corresponding touch interaction instructions, and forwards them to the customer service system. This enables remote interaction support for non-touchscreen terminals, allowing users to simulate touch interaction through key operations to complete functions such as business inquiries and processing. Figure 1 This is a flowchart illustrating the remote interaction method, including: S101, after establishing a video call between the target terminal and the customer service system, receives key signals from the dial pad of the target terminal; wherein, the video call supports touch interaction.

[0022] The significance of this step is to provide an alternative interactive input channel for terminals that lack touchscreen operation capabilities. Since the target terminal is usually a non-touchscreen terminal, it cannot directly collect the coordinate information of screen clicks or swipes. However, the dial pad, as its physical input component, can generate clear and transmittable key signals. Therefore, in an established video call session that supports touch interaction, by receiving and parsing such key signals, the user's key operations can be mapped into subsequent executable touch interaction commands, thereby simulating touchscreen-style remote interaction.

[0023] In practice, the reception of key signals is usually accomplished through the call signaling layer: after a video call is established, the target terminal sends the key characters generated by the user pressing the dial pad to the signaling gateway of the intermediate node through the communication network signaling channel; the signaling gateway parses the signaling message, identifies the key values ​​carried in it, extracts them as key signals and forwards them to the subsequent interactive command parsing module to trigger the subsequent instruction conversion and interactive execution process.

[0024] S102 determines the corresponding touch interaction command based on the button signal.

[0025] The significance of this step is to transform the simple physical key inputs from the dial pad into semantic operation commands that the customer service system can understand and execute, conforming to the touchscreen interaction logic. Since the key signal itself only represents a pressed key value (such as the number "1"), it does not directly carry interactive intentions such as "click here" or "swipe in which direction." Therefore, the mapping and parsing mechanism defined in this step must be used to convert these signals into structured touch interaction instructions, thereby achieving accurate simulation of touch interaction on non-touchscreen terminals at the business level.

[0026] Specifically, this embodiment can implement the following two types of touch interaction commands: The first method is a click-based touch command. To simulate a click, interactive interface elements must first be extracted from the video stream of the video call. Specifically, the video stream is frame-by-frame processed to obtain keyframe images containing static business layouts. Subsequently, based on optical character recognition (OCR) technology, all interface areas containing text with specific interactive functions (such as "Apply," "Query," and "Return") within these keyframe images are identified, and these areas are designated as interactive areas. For example, after identifying a rectangular area with the text "Business Query," it is defined as an interactive area. Next, each identified interactive area is associated with a unique dial pad key identifier (such as numbers 1 to 9), and this identifier is overlaid on the corresponding area in the video stream as a numerical label, thus intuitively informing the user of the mapping relationship between keys and functions. When a single key signal (such as "1") is received, the corresponding target interactive area (i.e., the "Business Query" area) is matched according to the established association, and a click-based touch command is generated for that area. This instruction essentially drives the system to perform a simulated click operation at the center coordinates of the area. Its technical effect is that it enables users to directly and accurately trigger specific business functions with a single button press, eliminating the cumbersome process of traditional voice or hierarchical button navigation and achieving efficient "one-click access" interaction.

[0027] The second type is the swipe touch command. When at least two key signals are received consecutively within a preset time interval (e.g., 1500 milliseconds), these key signals are determined to constitute a swipe operation combination. Based on the receiving order of these key signals and their physical position on the dial pad, a simulated swipe trajectory can be determined. For example, receiving keys "2" and "3" consecutively corresponds to a horizontal swipe to the right based on the dial pad layout; receiving keys "7" and "4" consecutively corresponds to a vertical swipe upwards. The system then generates a swipe touch command corresponding to this swipe trajectory to drive the interface to scroll in the corresponding direction. To improve the stability of the swipe operation and user experience, and to prevent interface jitter and system resource waste caused by rapid or accidental touches, the key signal sequence can be smoothed before determining the swipe operation combination. The specific process is as follows: The sequence of key signals arriving in chronological order is treated as a discrete-time domain signal. First, a Discrete Fourier Transform is performed on it to analyze its frequency components. Then, a low-pass filter (such as a Butterworth filter with a cutoff frequency of 1.5Hz) is used to filter the signal, suppressing high-frequency key inputs that may be due to misoperation or excessively rapid key presses. Finally, an Inverse Fourier Transform is performed on the filtered signal to reconstruct a new, smoothed, and frequency-reduced key signal sequence, which is then used as the basis for subsequent sliding analysis. The technical advantage of this process is that it effectively filters out noise and jitter during operation, making the generated sliding commands smoother and more consistent with the user's true intentions, while reducing the system computation and communication overhead caused by frequent interface refreshes.

[0028] S103, send a touch interaction command to the customer service system so that the customer service system can execute the touch interaction command during the video call.

[0029] The significance of this step lies in completing the closed loop of remote interactive operations, ultimately translating the user intent represented by the front-end button signals into actual actions and visual feedback on the customer service system interface, thereby enabling the user to effectively control the remote business system. Specifically, whether it's a click touch command generated by a single button mapping (e.g., a command in the format "click, (Xn,Yn)", representing a click at screen coordinates (Xn,Yn)) or a swipe touch command generated by a combination of buttons (e.g., a command in the format "swipe, [LB-to-RT,26.57°]", representing a swipe from the lower left to the upper right at an angle of 26.57°), both will be sent to the corresponding business module of the customer service system via a predetermined communication protocol (such as HTTP). After receiving and executing these commands, the customer service system will trigger business logic processing and interface state updates, such as page navigation or interface scrolling, thereby responding to the user's interaction requests.

[0030] It should be noted that this embodiment is a loop scheme that dynamically and continuously follows the video call. That is, it continuously receives and parses key signals and generates touch interaction commands to feed back to the customer service system. At the same time, the customer service system generates an updated business interface due to the execution of commands, which will form a new video stream. This updated video stream will be processed again through frame extraction, optical character recognition and digital annotation to generate a video screen with the latest key association labels, and then sent back to the target terminal via the media gateway. This allows the user to continuously see the interface state after the interaction and perform the next round of operation based on the new labels, thus forming a complete and sustainable remote interaction closed loop.

[0031] In summary, the method of this embodiment is applied to the intermediate node for establishing a video call between the target terminal and the customer service system. After the video call is established between the target terminal and the customer service system, it can receive key signals from the dial pad of the target terminal, parse the key signals into corresponding touch interaction commands, and then send the touch interaction commands to the customer service system to drive execution in the video call. This solution maps the universal dial pad input to standard touch commands, enabling terminals without touch screen hardware to simulate clicks, swipes, and other interactive operations using keys. Thus, without modifying the terminal hardware, it achieves the technical effect of remote touch control of non-touch screen terminals in video calls that support touch interaction.

[0032] The specific application of the method in this embodiment will be described in detail below.

[0033] Figure 2 This is a schematic diagram of the architecture applied to the intermediate node in this embodiment. Specifically, this intermediate node is a non-touchscreen terminal remote interaction device. Its overall architecture consists of a call signaling layer, an interaction control layer, and a call media layer, which work together to achieve remote interaction between the non-touchscreen terminal and the customer service system. Wherein: The call signaling layer comprises two core modules: a signaling gateway and a call control center. The signaling gateway is responsible for receiving call signaling from the Android terminal (i.e., the target terminal) and key characters generated by the user's operation of the dial pad. It forwards the call signaling to the call control center for session routing and management, and sends the extracted key characters to the interaction control layer. The call control center is responsible for connecting the call to the corresponding signaling services and business modules of the call center customer service system.

[0034] The interaction control layer is the core functional layer, which includes an interaction command parsing module and an interaction driver module. The interaction command parsing module receives key characters from the signaling gateway and associated information (i.e., the mapping relationship between key identifiers and interactive areas) sent by the content labeling module of the call media layer, and generates specific touch interaction commands accordingly. The interaction driver module converts the command into a format executable by the customer service system's business modules (such as an HTTP request) and sends it to drive the business system to perform the corresponding interaction.

[0035] The call media layer is responsible for processing and enhancing the video stream, including a video frame extraction module, a content annotation module, and a media gateway. The video frame extraction module extracts keyframes from the business video stream pushed by the customer service system's video media service. The content annotation module identifies interactive areas in the keyframes based on optical character recognition technology and associates them with button icons, generating annotation information which is sent to the interaction command parsing module (for establishing mappings) and the media gateway (for generating visual annotations). The media gateway finally forwards the annotated video stream to the user terminal for display. The entire architecture, through three-layer collaboration, completes a closed loop from receiving user buttons, parsing interaction intent, driving system response, to updating the user interface, thus providing complete remote touch interaction capabilities for terminals without touchscreen hardware.

[0036] in, Figure 3 This is a flowchart illustrating the remote interaction method for non-touchscreen terminals provided in this embodiment, which includes the following steps: Steps 1 to 3: The user initiates a video call using the dialing function on a non-touchscreen terminal connected to the communication network (Step 1). The call signaling travels through the communication network to the signaling gateway at the intermediate node (Step 2). The signaling gateway forwards the call request to the call control center. The call control center distributes the call to the corresponding business channel under the call center customer service system based on the call number. Once the business channel answers, the video call between the user and the customer service system is officially established (Step 3).

[0037] Step 4: After the call is established, the customer service system's business channels will push the corresponding business video to the user. The video content usually includes an interactive interface with function entry points such as "Business Processing" and "Business Inquiry".

[0038] Step 5: The pushed business video stream is not sent directly to the user, but is first forwarded to the video frame extraction module of the intermediate node. This module extracts key business video frames (i.e., specific frames) from the video stream and filters out frames that are unchanged or have only minor changes to reduce the computational cost of subsequent processing.

[0039] Step 6: The content annotation module receives the extracted business content image. Based on optical character recognition (OCR) technology, this module identifies and records the interactive areas of the image in real time. For example, it identifies a rectangular area containing the text "Main Business" and its coordinates. After recognition, the content annotation module associates and adds a numerical label (1-9) to each interactive text recognition area. For example, the "Main Business" area is labeled with the number "1," and its label coordinates are usually located at the center point of the top edge of the recognition rectangle. This process filters out text such as "Business Inquiry," which is only used for introduction and is not annotated.

[0040] Step 7: The content annotation module merges the recognition results (including text content, associated key-triggered numbers, and area coordinates) into a related data string according to a predetermined format, such as "Main Business: 1, , , , And send it to the interactive command parsing module.

[0041] Step 8: The content annotation module sends the images with added numerical annotations to the media gateway.

[0042] Step 9: The media gateway converts the image into a video stream suitable for transmission over the communication network, and forwards this labeled video stream to the user's non-touchscreen terminal for display, so that the user can intuitively understand the correspondence between the dial pad keys (1~9) and the interactive services on the screen.

[0043] When a user performs a key press based on the visible annotations, the process is differentiated into the following paths according to the interaction type (click or swipe): For click operations, the process can be combined with the appendix. Figure 4 The business interface shown is illustrated as an example: Steps 10 and 11: When a user views a labeled video interface on a non-touchscreen terminal, for example, seeing... Figure 3 The service query interface shown has a "1" marked next to the "Promoted Services" button. If a user intends to subscribe to a promoted service, they press the corresponding "1" key on the dial pad. The dial pad key numbers on the user terminal are sent to the signaling gateway via the communication network's signaling channel (step 10). The signaling gateway parses the signaling, identifies it as key data rather than call control signaling, extracts the key value, and forwards it to the interactive command parsing module (step 11).

[0044] Step 12: After receiving a single keystroke of the number "1", the interaction command parsing module determines it as a click interaction. Based on the associated data received in Step 7, the module queries the target interactive area that matches the keystroke. Figure 4 The "Featured Services" button area is shown, and the coordinates of this area are calculated to simulate a click. The calculation method typically involves taking the coordinates of the center point of the rectangular area, i.e. , ;in The coordinates of the top left corner of the "Key Business" text area are: Set its bottom right corner coordinates. Then, the module is generated in the format "Click," The click interaction command.

[0045] Step 13: The interaction-driven module receives the click interaction instruction and sends the corresponding action command, such as click, to the business module of the customer service system through a specific communication protocol (such as HTTP protocol). This drives it to perform simulated click operations.

[0046] Step 14: After the customer service system executes the click operation, the current business interface is redirected, for example from... Figure 3 The main business query interface shown will redirect to the "Featured Businesses" details page (this details page may contain, for example,...). Figure 3 The lower section shows new interactive options such as "Business Processing" and "Return to Parent Level". After the interface redirects, the customer service system updates the business video content. This updated video will then re-enter the loop that started in step 5, undergoing a new round of frame extraction, recognition, and annotation to provide new key mapping relationships for subsequent user operations.

[0047] For sliding operations, the process can be combined with the attached... Figure 5 The following is an example illustrating the layout of the dial pad and the mapping of the sliding directions: Steps 15, 16, and 17: When a user needs to scroll through the service interface (such as viewing a long list), they can press multiple buttons on the dial pad consecutively within a short period of time. For example, ... Figure 4 As shown, if the user wants to swipe the interface to the right, they can quickly press buttons "2" and "3" in sequence (corresponding to the horizontal swipe example); if they want to swipe upwards, they can quickly press buttons "7" and "4" in sequence (corresponding to the vertical swipe example). These consecutive button presses are sent to the signaling gateway in sequence (steps 15 and 16). The signaling gateway parses and forwards the consecutive button press sequence to the interactive command parsing module (step 17). In this embodiment, the core mechanism for determining whether consecutive button presses constitute a swipe combination is based on time interval: when the interval between the current button press and the previous button press is less than or equal to 1500 milliseconds, they are considered to belong to the same swipe operation combination.

[0048] Step 18: After receiving a combination of swipe operations, the interactive command parsing module determines the direction and distance of each sub-swipe based on the receiving order of each button and their physical position on the dial pad, thereby generating the corresponding swipe interaction command. For example... Figure 4 As shown, the combination "2->3" represents a horizontal swipe of one unit distance to the right; the combination "7->4" represents a vertical swipe of one unit distance upwards; and the combination "7->2" represents a swipe from the bottom left to the top right at an angle of... The diagonal swipe is the first type of swipe. For more complex continuous swipes, such as the combination "1->7->5->6->0", it can be resolved into a series of vertical, diagonal, and horizontal swipes. Furthermore, to optimize the interactive experience and reduce the system load caused by frequent operations, the fast original key sequence can be smoothed and its frequency reduced before generating the final swipe command: the timing key signals are treated as discrete time-domain sequences. Through Discrete Fourier Transform Convert to the frequency domain to analyze its frequency components, where Index variables representing frequency domain sequences (values ​​ranging from 0 to...) ), The imaginary unit is used; then, a low-pass filter (such as a Butterworth filter, whose amplitude-frequency response is...) is used. ,in The signal angular frequency, The set cutoff angular frequency, (Filter order) to remove high-frequency jitter components; finally, perform an inverse Fourier transform on the filtered signal. The key timing signal is restored to a smoothed state, and then the sliding trajectory is analyzed and commands are generated based on this stable signal. The format may be "slide, [direction, angle, distance]".

[0049] Step 19: After receiving the sliding interaction command, the interaction-driven module converts it into a standardized command format that the customer service system's business module can recognize and execute. For example, the command "slide, [up, vertical, unit distance]" is converted into a slide(up, 1) request sent via the HTTP protocol and sent to the corresponding interface of the customer service system for execution.

[0050] Step 20: After the customer service system performs a swipe operation, the business interface scrolls in the corresponding direction, thus displaying a new content area. This effect can be combined with the attached... Figure 6 Here's a detailed explanation: After a single upward swipe, the interface transitions from the left to the right. The "Secondary Service 2" (labeled 3) at the bottom of the original interface scrolls up to the top of the screen, its label updated to "1". The original "Recharge & Payment" area moves upwards, and the labels for "Package 1 Processing" and "Package 2 Processing" are updated from 4 and 5 to 2 and 3 respectively. Simultaneously, the "Complaints" area, which was not fully displayed in the original interface, enters the visible range, with "Service 1 Complaint" and "Service 2 Complaint" becoming new interactive items, labeled 4 and 5 respectively. The customer service system then updates the output video content based on this new interface layout. This updated video stream will then re-enter the loop starting in step 5, triggering a new round of frame extraction, optical character recognition, and numerical labeling. This provides updated key mappings for subsequent user interactions, achieving a continuous closed loop of swipe browsing and interactive guidance.

[0051] In addition, corresponding to Figure 1 The method shown in this embodiment, in another embodiment, also provides a remote interaction device, which is similarly applied to an intermediate node for establishing a video call between a target terminal and a customer service system. Wherein, Figure 7 This is a structural diagram of the remote interaction device 700, including: The key signal receiving module 710 is used to receive key signals from the dial pad of the target terminal after a video call is established between the target terminal and the customer service system; wherein the video call supports touch interaction. The touch interaction command generation module 720 is used to determine the corresponding touch interaction command based on the button signal; The touch interaction instruction transmission module 730 sends touch interaction instructions to the customer service system so that the customer service system executes the touch interaction instructions during the video call.

[0052] In this embodiment, after a video call is established between the target terminal and the customer service system, the device receives key signals from the target terminal's dial pad, parses these key signals into corresponding touch interaction commands, and then sends the touch interaction commands to the customer service system to drive execution during the video call. This solution maps universal dial pad input to standard touch commands, enabling terminals without touchscreen hardware to simulate clicks, swipes, and other interactive operations using buttons. Thus, without modifying the terminal hardware, it achieves the technical effect of remote touch control of non-touchscreen terminals in video calls that support touch interaction.

[0053] Optionally, before receiving key signals from the dial pad of the target terminal, the touch interaction command generation module 730 is further configured to: determine at least one interactive area from the video stream of the video call, and associate each interactive area with a key identifier of the dial pad; and mark the corresponding associated key identifier on the interactive area corresponding to the video stream.

[0054] Optionally, the touch interaction command generation module 730 determines the corresponding touch interaction command based on the button signal, including: when a single button signal is received, determining a target interactive area that matches the button signal based on the association between the interactive area and the button identifier; and generating a click touch command for the target interactive area.

[0055] Optionally, the touch interaction command generation module 730 determines at least one interactive area from the video stream of the video call, including: performing frame extraction processing on the video stream of the video call to obtain key frame images; identifying at least one interface area with text having specific interactive functions in the key frame images based on optical character recognition technology; and determining the identified interface area as an interactive area.

[0056] Optionally, the touch interaction command generation module 730 determines the corresponding touch interaction command based on the button signals, including: when at least two button signals are received consecutively within a preset time interval, determining that all the button signals constitute a sliding operation combination; determining the sliding trajectory according to the receiving order of each button signal in the sliding operation combination and the corresponding button position; and generating a sliding touch command corresponding to the sliding trajectory.

[0057] Optionally, before determining that all the key signals constitute a sliding operation combination, the touch interaction instruction generation module 730 is further configured to: perform a discrete Fourier transform on all the received key signals to obtain the corresponding discrete time-domain sequence signal; smooth the discrete time-domain sequence signal by low-pass filtering; and perform an inverse Fourier transform on the smoothed discrete time-domain sequence signal to replace all the key signals before the discrete Fourier transform with all the key signals obtained by the inverse Fourier transform.

[0058] Optionally, the target terminal is a non-touchscreen terminal, and the dial pad is a physical button dial pad.

[0059] It should be noted that the remote interaction device of the charging pile in this embodiment can be used as... Figure 1 The execution body of the method shown is therefore able to achieve... Figure 1 The steps and functions of the method shown are illustrated.

[0060] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to it. Figure 8 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0061] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0062] Memory is used to store computer programs. Specifically, a computer program may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides the computer program to the processor.

[0063] Specifically, the processor reads the corresponding computer program from non-volatile memory into memory and then runs it, forming the above-mentioned logical structure. Figure 7 The remote interaction device shown. Correspondingly, the processor executes the program stored in the memory, and specifically performs the following operations: After a video call is established between the target terminal and the customer service system, key signals from the dial pad of the target terminal are received; wherein, the video call supports touch interaction.

[0064] Based on the button signal, the corresponding touch interaction command is determined.

[0065] Send a touch interaction command to the customer service system so that the customer service system executes the touch interaction command during the video call.

[0066] The above is as described in this instruction manual. Figure 1The remote interaction method disclosed in the illustrated embodiments can be applied to a processor and implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the processor's hardware or by instructions in software form. The processor mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0067] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0068] Furthermore, embodiments of this application also propose a computer program product, including a computer-readable storage medium storing one or more computer programs, the one or more computer programs including instructions.

[0069] When the aforementioned instructions are executed by a portable electronic device that includes multiple applications, they enable the portable electronic device to perform... Figure 1 The steps in the method shown include: After a video call is established between the target terminal and the customer service system, key signals from the dial pad of the target terminal are received; wherein, the video call supports touch interaction.

[0070] Based on the button signal, the corresponding touch interaction command is determined.

[0071] Send a touch interaction command to the customer service system so that the customer service system executes the touch interaction command during the video call.

[0072] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0074] The above are merely embodiments of this specification and are not intended to limit the scope of this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification. Furthermore, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this document.

Claims

1. A remote interaction method, applied to an intermediate node for establishing a video call between a target terminal and a customer service system, characterized in that, The method includes: After a video call is established between the target terminal and the customer service system, key signals from the dial pad of the target terminal are received; wherein, the video call supports touch interaction; Based on the button signals, the corresponding touch interaction commands are determined; Send a touch interaction command to the customer service system so that the customer service system executes the touch interaction command during the video call.

2. The method according to claim 1, characterized in that, Before receiving key signals from the dial pad of the target terminal, the method further includes: Determine at least one interactive area from the video stream of the video call, and associate each interactive area with a key label of the dial pad; Mark the corresponding button icons on the interactive area corresponding to the video stream.

3. The method according to claim 2, characterized in that, Based on the button signal, the corresponding touch interaction command is determined, including: When a single button signal is received, a target interactive area matching the button signal is determined based on the association between the interactive area and the button identifier. Generate a click touch command for the target interactive area.

4. The method according to claim 2, characterized in that, Determining at least one interactive area from the video stream of the video call includes: The video stream of the video call is subjected to frame extraction processing to obtain key frame images; Based on optical character recognition technology, at least one interface area with text having specific interactive functions in the keyframe image is identified; The identified interface area is determined as an interactive area.

5. The method according to claim 1, characterized in that, Based on the button signal, the corresponding touch interaction command is determined, including: When at least two of the button signals are received consecutively within a preset time interval, it is determined that all the button signals constitute a sliding operation combination. The sliding trajectory is determined based on the receiving order of each button signal in the sliding operation combination and the corresponding button position; Generate a sliding touch command corresponding to the sliding trajectory.

6. The method according to claim 5, characterized in that, Before determining that all the key signals constitute a sliding operation combination, the method includes: Perform a Discrete Fourier Transform on all received key signals to obtain the corresponding discrete time-domain sequence signals; The discrete time-domain sequence signal is smoothed by low-pass filtering; An inverse Fourier transform is performed on the smoothed discrete-time sequence signal to replace all the key signals before the discrete Fourier transform with all the key signals obtained by the inverse Fourier transform.

7. The method according to any one of claims 1 to 6, characterized in that, The target terminal is a non-touchscreen terminal, and the dial pad is a physical button dial pad.

8. A remote interactive device, used as an intermediate node to establish a video call between a target terminal and a customer service system, characterized in that, The device includes: A key signal receiving module is used to receive key signals from the dial pad of the target terminal after a video call is established between the target terminal and the customer service system; wherein the video call supports touch interaction. A touch interaction command generation module is used to determine the corresponding touch interaction command based on the button signal; The touch interaction command transmission module sends touch interaction commands to the customer service system so that the customer service system executes the touch interaction commands during the video call.

9. An electronic device, comprising: processor; And a memory arranged to store computer-executable instructions, characterized in that, when executed, the executable instructions cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer program product, the computer program product comprising a computer-readable storage medium storing a computer program, characterized in that, The computer program is operable to cause the computer to perform the method as described in any one of claims 1 to 7.

Images