Method and apparatus for interaction between intelligent wearable device and mobile terminal, and device and medium

Abstract

A method and apparatus for interaction between an intelligent wearable device and a mobile terminal, and a device and a medium. In the method, by means of collecting corner point coordinates of a target corner point on a mobile terminal, and determining a target extended-screen component on which the current user wants to operate, accurate identification and selection of an extended-screen component are realized; and by means of collecting the current gesture action of the current user, generating a corresponding component operation instruction, and executing a target operation on the target extended-screen component, the interaction efficiency between the intelligent wearable device and the mobile terminal is improved.

Description

Interaction method, device, equipment and medium between smart wearable device and mobile terminal

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 27, 2023, with application number 2023112621134, and invention name “Method, device, equipment and medium for interaction between smart wearable devices and mobile terminals”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of smart wearable technology, and in particular to a method, apparatus, computer equipment, and storage medium for interaction between a smart wearable device and a mobile terminal. Background Art

[0003] Extended Screen is a quick app engine running on smart large-screen devices, providing a new multi-screen interactive model. It can be launched and interacted with by mobile client applications and mini-programs in a similar way to screen mirroring, or directly operated and interacted with using the large-screen remote control. Extended Screen mirrors the mobile terminal application interface on the display interface of the smart wearable device, facilitating the rapid upload of mobile terminal services and content to the large screen, significantly reducing the development, promotion, and maintenance costs of large-screen applications.

[0004] Smart wearable devices can interact with mobile terminals on an extended screen based on augmented reality technology, mirroring the mobile terminal's desktop applications on the smart wearable device's display interface. While smart wearable devices currently display mobile terminal desktop applications on an extended screen, they cannot directly operate the extended screen components within the display interface. This results in the interaction between smart wearable devices and mobile terminals failing to meet user needs, resulting in a poor user experience with the interactive functions between smart wearable devices and mobile terminals.

[0005] Therefore, how to improve the user experience of the interactive functions between smart wearable devices and mobile terminals has become a technical problem that needs to be solved urgently.

[0006] Summary of the Invention

[0007] The present application provides a method, apparatus, device and storage medium for interaction between a smart wearable device and a mobile terminal, aiming to improve the user experience of the interaction function between the smart wearable device and the mobile terminal.

[0008] In a first aspect, the present application provides a method for interaction between a smart wearable device and a mobile terminal, comprising:

[0009] When the smart wearable device is in an extended screen interaction state with the mobile terminal, detecting the current posture of the mobile terminal;

[0010] When detecting that the current posture of the mobile terminal matches a preset interactive operation action, collecting the corner point coordinates of the target corner point on the mobile terminal;

[0011] Determining a target extended screen component based on an extended screen component area corresponding to the corner point coordinates;

[0012] When the current gesture action of the current user matches the preset gesture operation action, the target operation is performed on the target extended screen component based on the component operation instruction corresponding to the gesture operation action.

[0013] In a second aspect, the present application further provides an interactive device between a smart wearable device and a mobile terminal, comprising:

[0014] A posture detection module is used to detect the current posture of the mobile terminal when the smart wearable device is in an extended screen interaction state with the mobile terminal;

[0015] A corner point coordinate acquisition module, configured to acquire the corner point coordinates of a target corner point on the mobile terminal when detecting that the current posture of the mobile terminal matches a preset interactive operation action;

[0016] A component determination module, configured to determine a target extended screen component based on an extended screen component area corresponding to the corner point coordinates;

[0017] The component operation module is used to perform a target operation on the target extended screen component based on the component operation instruction corresponding to the gesture operation when the current gesture action of the current user matches the preset gesture operation action.

[0018] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method for interaction between the smart wearable device and the mobile terminal as described above are implemented.

[0019] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the method for interaction between the smart wearable device and the mobile terminal as described above are implemented.

[0020] The present application provides a method, apparatus, device, and storage medium for interaction between a smart wearable device and a mobile terminal. The method comprises the following steps: when the smart wearable device and the mobile terminal are in an extended screen interaction state, the smart wearable device detects the current posture of the mobile terminal and matches the current posture of the mobile terminal with a preset interactive operation action; and when the current posture matches the preset interactive operation action, the operation recognition process of the extended screen component is triggered, thereby improving the accuracy of component operation and avoiding false triggering of operation. By collecting the corner point coordinates of the target corner point on the mobile terminal, the target extended screen component that the current user wants to operate is determined based on the extended screen component area where the corner point coordinates are located, thereby achieving accurate identification and selection of the extended screen component; by comparing the current user's current gesture action with the preset gesture operation action, the gesture operation action that matches the current gesture action is determined, and then a component operation instruction corresponding to the gesture operation action is generated. According to the component operation instruction, the target operation is performed on the target extended screen component, so that the current user can directly operate the extended screen component on the display interface of the smart wearable device, thereby improving the interaction efficiency between the smart wearable device and the mobile terminal and enhancing the user's user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] FIG1 is a flow chart of a first embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application;

[0023] FIG2 is a flow chart of a second embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application;

[0024] FIG3 is a schematic diagram of a detailed flow chart of the first step of a second embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application;

[0025] FIG4 is a flow chart of a third embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application;

[0026] FIG5 is a schematic diagram showing the distribution of extended screen components in an extended screen mode activated state according to an embodiment of the present application;

[0027] FIG6 is a schematic diagram showing a relative coordinate relationship distribution between an extended screen component area and a mobile terminal according to an embodiment of the present application;

[0028] FIG7 is a schematic diagram showing a relative size relationship distribution between second size information of an extended screen assembly and first size information of a mobile terminal provided by an embodiment of the present application;

[0029] FIG8 is a schematic diagram showing another relative size relationship distribution between the second size information of an extended screen assembly and the first size information of a mobile terminal provided by an embodiment of the present application;

[0030] FIG9 is a structural diagram of a first embodiment of an interactive device between a smart wearable device and a mobile terminal provided by the present application;

[0031] FIG10 is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0032] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0035] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0036] Please refer to Figure 1, which is a flow chart of a first embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application.

[0037] As shown in FIG1 , the method for interaction between the smart wearable device and the mobile terminal includes steps S101 to S104 .

[0038] Step S101: When the smart wearable device and the mobile terminal are in an extended screen interaction state, the smart wearable device detects the current posture of the mobile terminal;

[0039] In one embodiment, a smart wearable device refers to a wearable eyewear device that has an independent operating system and can implement various functions through software installation, such as AR glasses (Augmented Reality goggles), VR glasses (Virtual Reality), AR helmets, etc.

[0040] In one embodiment, a mobile terminal or mobile communication terminal refers to a computer device that can be used on the move, including a mobile phone, a notebook, a tablet computer, etc.

[0041] In one embodiment, the smart wearable device can detect the current posture of the mobile terminal using an image acquisition device. The image acquisition device refers to a device component capable of image acquisition and image processing, such as a camera, infrared sensor, laser scanner, etc. The image acquisition device can be carried by the smart wearable device itself or be a standalone device installed on the smart wearable device.

[0042] In one embodiment, an image acquisition device is communicatively connected to a smart wearable device to capture image or video data of the mobile device and a user's hand, and to identify the posture, coordinates, and movements of the mobile device and the user's hand based on the captured image and video data. Upon recognizing that a mobile terminal has moved into the image acquisition device's capture image, the image acquisition device can identify the mobile terminal's outline information and corner coordinates, and can also track and identify information such as the mobile terminal's movement trajectory.

[0043] Step S102: when it is detected that the current posture of the mobile terminal matches a preset interactive operation action, collecting the corner point coordinates of the target corner point on the mobile terminal;

[0044] In one embodiment, the display interface of the smart wearable device can be used as a two-dimensional coordinate system, the upper left corner of the display interface can be used as the origin of the coordinate system, the two sides of the display interface can be used as the coordinate axes of the two-dimensional coordinate system, and the scale of the coordinate axis can be set according to the length and width of the display interface. In this way, the outline coordinates and corner point coordinates of the mobile terminal can be tracked in the display interface of the smart wearable device.

[0045] In one embodiment, the image acquisition device of the smart wearable device can track and calculate the outline and corner coordinates of the mobile terminal in real time. If it is detected that the current posture of the mobile terminal matches an interactive operation action, that is, the similarity between the current posture and the interactive operation action is greater than or equal to a similarity threshold (e.g., 90%), the image acquisition device will capture the current corner coordinates of the mobile terminal. Interactive operation actions may include double-clicking, remaining stationary for a preset period of time, sliding, etc.

[0046] In one embodiment, the corner coordinates of the mobile terminal may be edge lines representing sides of the mobile terminal based on the outline coordinates of the mobile terminal, wherein the intersection of two intersecting edge lines may be used as the corner coordinates of the mobile terminal.

[0047] In one embodiment, the four corners of the mobile terminal are generally designed to be arc-shaped. The intersection coordinates of the various edge lines of the mobile terminal can be calculated based on the contour coordinates of the mobile terminal. The four intersection points correspond to the four arc corners of the mobile terminal respectively. The arc point on each arc corner that is closest to the corresponding intersection coordinates can be used as the corner point.

[0048] In one embodiment, a mobile terminal generally has four sides, that is, the coordinates of the four corner points of the mobile terminal can be tracked. In order to improve the recognition accuracy of the extended screen component, one of the four corner points can be pre-set as the target corner point, such as the upper left corner of the mobile terminal.

[0049] In one embodiment, the smart wearable device can use an image acquisition device to identify the outline of the mobile terminal and activate the mobile terminal's extended screen function, thereby enabling extended screen interaction between the smart wearable device and the mobile terminal. In the extended screen interaction state, the smart wearable device uses the image acquisition device to capture the coordinates of the mobile terminal's corner points, such as the coordinates of the upper right corner.

[0050] Step S103: determining a target extended screen component based on the extended screen component area corresponding to the corner point coordinates;

[0051] In one embodiment, upon detecting that the current posture of the mobile terminal matches a preset interactive operation, the coordinates of the target corner points of the mobile terminal are collected, and the extended screen component area where the corner coordinates of the target corner points are located is determined as the target area. The target extended screen component corresponding to this area is the extended screen component on which the user currently wants to perform an operation. The extended screen component is a high-performance cross-terminal development framework for smart large-screen devices and a fast application engine running on smart wearable devices. It helps developers quickly build application services for smart wearable devices and provides multi-screen interaction capabilities.

[0052] In one embodiment, the extended screen component can be a mapping of the mobile terminal application interface to the smart wearable device display interface, facilitating the quick display of mobile terminal services and content on the smart wearable device display interface. The extended screen component includes the mobile terminal application component, which is used to display the mobile terminal application display content, such as videos played by video software, real-time bullet comments, and reading pages of novel software.

[0053] In one embodiment, the extended screen component also includes a functional module component of the smart wearable device, which facilitates users to query and call the functions and services of the smart wearable device, such as product descriptions, a list of connectable devices, a network list, etc.

[0054] Furthermore, based on the corner point coordinates, a current interaction action of the current user operating the mobile terminal is identified; when the current interaction action meets the trigger condition corresponding to the preset interaction operation action, the current interaction action is determined to be the interaction operation action.

[0055] In one embodiment, the interactive operation action can be identified based on the coordinates of the preset corners of the mobile terminal. For example, the current user operates the mobile terminal to perform a double-click action, that is, the coordinate value of the target corner point of the mobile terminal points to the same coordinate position within a preset time; or when the image acquisition device recognizes that there is an obvious tapping action at the corner of the mobile terminal, the landing point coordinates of the target corner point in the tapping action are collected. If the landing point coordinates are located in a certain extended screen component area, the extended screen component corresponding to the extended screen component area is determined to be the target extended screen component; or when it is identified that the time that the corner point coordinates of the target corner point stay in a certain extended screen component area reaches a preset threshold, the extended screen component corresponding to the extended screen component area can be determined to be the target extended screen component that the current user wants to view or perform a certain operation.

[0056] In one embodiment, if it is detected that the current user operates the mobile terminal to perform an interactive operation, but the coordinate area corresponding to the corner point coordinates does not belong to the extended screen component area, it can be determined that the interactive operation is invalid and no operation instruction is generated.

[0057] In one embodiment, if it is detected that the current user operates the mobile terminal to perform an interactive operation, but the coordinate area corresponding to the corner point coordinates does not belong to the extended screen component area, the distance between the corner point coordinates and the midpoint position of each extended screen component area can also be calculated, and the extended screen component area corresponding to the shortest distance is determined to be the landing area of ​​the corner point coordinates, thereby determining that the extended screen component corresponding to the landing area is the target extended screen component.

[0058] In one embodiment, the extended screen component area is the display area of ​​each extended screen component in the display interface of the smart wearable device. Using the display interface as a two-dimensional coordinate system, the coordinates of the four corner points of the extended screen component area can be determined, and the side length of the extended screen component area can be calculated based on the four corner point coordinates. For example, the coordinate distance between two corner point coordinates in the same row is the length of the extended screen component area, and the coordinate distance between two corner point coordinates in the same column is the width of the extended screen component area. Therefore, based on the coordinate relationship between the corner point coordinates and the area coordinates of the extended screen component area, it can be determined whether the coordinate area corresponding to the corner point coordinates belongs to the extended screen component area. If the corner point coordinates are within the range of any extended screen component area, then the coordinate area corresponding to the corner point coordinates is determined to belong to the extended screen component area; if the corner point coordinates are not within the range of any extended screen component area, then the coordinate area corresponding to the corner point coordinates is determined not to belong to the extended screen component area.

[0059] Step S104: When the current gesture action of the current user matches the preset gesture operation action, a target operation is performed on the target extended screen component based on the component operation instruction corresponding to the gesture operation action;

[0060] In one embodiment, the image acquisition module can be used to capture the current user's current gestures. The current user can make preset gestures in the image acquisition screen of the image acquisition module so that the smart wearable device can recognize the current user's operation on the target extended screen component.

[0061] In one embodiment, the preset gesture operation action may include a single finger or multiple fingers, or a gesture action of the entire hand, such as a double-click action of the thumb, a sliding action of the index finger, etc.

[0062] Furthermore, a preset gesture instruction library is obtained, wherein the gesture instruction library includes at least one of the gesture operation actions and the component operation instructions corresponding to each of the gesture operation actions; based on a gesture recognition algorithm, the current gesture action is compared with each of the gesture operation actions in the gesture instruction library to determine the similarity between the current gesture action and each of the gesture operation actions; when the similarity is greater than or equal to a preset similarity, the gesture operation action that matches the current gesture action is determined based on the similarity.

[0063] In one embodiment, gesture operations corresponding to each component operation instruction can be preset based on at least one component operation instruction of the extended screen component. The smart wearable device can pre-record gesture operations using an image acquisition device and uniquely associate each gesture operation with the corresponding component operation instruction, so that each component operation instruction corresponds to a gesture operation. The gesture operation is stored in a data storage module of the smart wearable device, thereby obtaining a gesture instruction library.

[0064] In one embodiment, a component operation instruction may also correspond to multiple different gesture operation actions. For example, a component details page viewing instruction may correspond to a double-click gesture operation action. The double-click gesture operation action may be a double-click action of the thumb or a double-click action of the index finger.

[0065] In one embodiment, the same gesture operation action can also correspond to multiple component operation instructions. For example, the component details page viewing instruction and the component details page exit instruction can both correspond to a gesture double-click operation action. If a gesture double-click operation action is detected outside the component details page (i.e., without entering the component details page), a component details page viewing instruction is generated. If a gesture double-click operation action is detected within the component details page (i.e., already in the component details page), a component details page exit instruction is generated.

[0066] In one embodiment, the motion trajectory of the current gesture action can be calculated through a gesture recognition algorithm, and then the motion trajectory is compared with the motion trajectory of each gesture operation action in the gesture instruction library to calculate the similarity between the motion trajectory of the current gesture action and the motion trajectory of each gesture operation action.

[0067] If the similarity between the current gesture's trajectory and that of a particular gesture operation is greater than or equal to a preset similarity, the current gesture is determined to be that gesture operation, and the two are matched. If the similarity between the current gesture's trajectory and the trajectory of each gesture operation in the gesture instruction library is less than the preset similarity, the current gesture is determined to be unmatched with any of the gesture operations, and no component operation instruction can be generated.

[0068] In one embodiment, if the similarity between the motion trajectory of the current gesture and the motion trajectories of multiple gesture operations is greater than or equal to a preset similarity, the gesture operation operation with the highest similarity is determined to match the current gesture. The motion trajectory of the current gesture operation may be a coordinate trajectory of a key point of the hand (joints, fingertips, etc.) over time.

[0069] In one embodiment, when it is detected that the current gesture action matches the preset gesture operation action, the smart wearable device generates a corresponding component operation instruction according to the gesture operation action, and performs a corresponding operation on the target extension component according to the component operation instruction.

[0070] In one embodiment, when it is detected that the current gesture action matches the preset gesture operation action, the smart wearable device can determine the component operation instruction corresponding to the current gesture operation action based on the preset gesture instruction library, and then generate the component operation instruction. For example, a click action can indicate entering the component details page, and at this time, an instruction to open the details page of the extended screen component is generated; a swiping action indicates exiting the component details page, and at this time, an instruction to exit the details page of the extended screen component is generated, etc.

[0071] For example, if the double-click operation action corresponds to the component viewing instruction of the extended screen component, the viewing instruction of the target extended screen component is generated according to the target extended screen component pointed to by the double-click operation action, such as the component operation instruction "Music Player--View", then you can enter the details page of the music player extended screen component according to the component operation instruction.

[0072] If the sliding operation corresponds to the switching instruction of the extended screen component, when it is detected that the mobile terminal makes a sliding operation in the current extended screen component details page, a component details page switching instruction is generated to switch the current extended screen component details page to another extended screen component details page.

[0073] In one specific embodiment, the process of executing the target extended screen component operation based on the current user's current gesture action can be: when the thumb double-clicks the extended screen component, it indicates that the user wants to view the details page of the extended screen component. When the image acquisition module recognizes that the user's thumb has made a double-click action on the component page, the gesture operation action corresponding to the double-click action (thumb-double-click) is matched from the gesture instruction library. The smart wearable device generates a component operation instruction corresponding to the gesture operation action (enter component details page instruction) and enters the details page of the target extended screen component according to the component operation instruction. If the user's gesture is captured to make a sliding action (corresponding to the component switching instruction action) on the details page of the target extended screen component, the smart wearable device generates a component switching instruction to switch to display the details page of the next extended screen component. If the user's thumb is captured to make a double-click action on the details page of the extended screen component, a back-off instruction can be matched according to the "component details page-double-click action-exit component details page" and the back-off instruction is used to exit the details page of the currently displayed extended screen component.

[0074] This embodiment provides an interaction method between a smart wearable device and a mobile terminal. When the smart wearable device and the mobile terminal are in an extended screen interaction state, the method detects the current posture of the mobile terminal through an image acquisition device, and when the current posture matches a preset interactive operation action, triggers the operation recognition process of the extended screen component, thereby improving the accuracy of component operation and avoiding false triggering of operations. The image acquisition device captures the corner point coordinates of the target corner point on the mobile terminal, and then determines the target extended screen component that the current user wants to operate based on the extended screen component area where the corner point coordinates are located, thereby achieving accurate identification and selection of the extended screen component; based on the identified interactive operation action, a corresponding component operation instruction is generated, and then according to the component operation instruction, the target operation is performed on the target extended screen component, so that the current user can directly operate the extended screen component on the display interface of the smart wearable device, thereby improving the interaction efficiency between the smart wearable device and the mobile terminal and enhancing the user's usage experience.

[0075] Please refer to FIG. 2 , which is a flow chart of a second embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application.

[0076] As shown in FIG2 , based on the embodiment shown in FIG1 , before step S101 , the following steps are further included:

[0077] Step S201: Based on the image acquisition device, identify the preset unbinding action made by the current user and generate a device unbinding instruction;

[0078] In one embodiment, in extended screen mode, the mobile terminal and the extended screen assembly are bound. When the mobile terminal moves, the extended screen assembly moves synchronously with it, anchored to the periphery of the mobile terminal. In this state, the current user can only control the extended screen assembly by operating desktop applications on the mobile terminal screen. To operate the extended screen assembly on the display interface of the smart wearable device, the mobile terminal and the extended screen assembly must be unbound.

[0079] In one embodiment, the binding state between the mobile terminal and the extended screen component can be accessed by designing a specific preset unbinding action, such as double-clicking within the screen range of the mobile terminal.

[0080] In one embodiment, when the image acquisition device of the smart wearable device recognizes that the current user makes a preset unbinding action, it triggers an unbinding event between the mobile terminal and the extended screen component, and the smart wearable device generates a device unbinding instruction to release the binding status of the mobile terminal and the extended screen component.

[0081] Furthermore, as shown in FIG3 , step S201 specifically includes:

[0082] Step S2011: Based on the image acquisition device, collect hand movement pictures of the current user to obtain a hand movement picture group, wherein the hand movement picture group includes at least two hand movement pictures;

[0083] In one embodiment, the image acquisition device of the smart wearable device can be used to collect hand movement pictures of the current user to obtain a hand movement picture group. The picture group can be in the form of a video stream, that is, a group of hand movement pictures taken in chronological order.

[0084] In one embodiment, the frame rate of the image acquisition device can be set based on the tracking effect of the hand movement. For example, the frame rate can be set to 25 FPS (frames per second), that is, 25 frames of images are captured every second. This ensures that the user's hand movements can be fully captured, reduces the amount of image data processing, and improves the efficiency of hand movement recognition and action response.

[0085] Step S2012: extracting the coordinates of the target key points in the hand action image based on a preset key point tracking algorithm to obtain the time series coordinate data of the target key points;

[0086] In one embodiment, the time series coordinate data refers to the time series data of the coordinates of the target key points. Time series data is a series of data obtained at different time periods or time points in chronological order, describing the changes of the observed object over time.

[0087] In one embodiment, according to a preset key point tracking algorithm, target key points in each hand movement image, such as fingertips, joints, etc., are extracted, and the point coordinates of the target key points are tracked and calculated to obtain a set of coordinate change data of the target key points in time sequence.

[0088] Step S2013: when it is detected that there are at least two coincident point coordinates in the time series coordinate data, calculating the collection time interval of each of the coincident point coordinates based on the collection time corresponding to each of the coincident point coordinates;

[0089] In one embodiment, if at least two overlapping point coordinates are detected in the time-series coordinate data, it indicates that the current user has clicked the same location at least twice using the mobile phone. In this case, the coordinate collection time of each of the overlapping point coordinates is determined based on the time-series coordinate data, and then the collection time interval of each of the overlapping point coordinates is calculated.

[0090] In one embodiment, the coincident point coordinates may be two points with exactly the same coordinates, or two points with a distance less than a set value. For example, two points with a distance less than 5 pixels may be considered coincident point coordinates.

[0091] Step S2014: When the collection time interval is less than the preset time interval, determine that the hand movement of the current user is the preset unbinding movement, and generate the device unbinding instruction.

[0092] In one embodiment, a preset time interval is set according to a preset unbinding action. If the preset unbinding action is a double-click, the preset time interval for determining a double-click can be set to 1 second, which means that if the current user clicks the same location twice within 1 second, the preset unbinding action is triggered.

[0093] In one embodiment, when the time interval for collecting the coordinates of each overlapping point is less than the preset time interval, it can be determined that the current user's hand movement has triggered a preset unbinding action. At this time, the smart wearable device generates a device unbinding instruction and, according to the unbinding instruction, releases the binding status between the mobile terminal and the extended screen component.

[0094] For example, a double-click is two quick and consecutive clicks within the same area. The 21 key points of the human hand can be obtained through the hand key point algorithm. The position coordinates of the thumb tip and the first joint are tracked in each frame. Assuming that the interval time is defined as t frames, if the thumb tip and the first joint of the thumb appear twice in the same position within t frames and the thumb is in the screen area of ​​the mobile terminal, the preset unbinding action is triggered, and the binding state between the mobile terminal and the extended screen component is released.

[0095] In one embodiment, the area range may be a range radiating outwards a certain coordinate distance from the coordinates of the first click point as the center, that is, the determination is made based on the coordinate distance between the coordinates of the two consecutive clicks. For example, with the coordinates of the first click point as the reference point, if the coordinate distance between the coordinates of the second click point and the reference point is less than or equal to a specified value (e.g., a distance of five pixels), and the time interval between the two clicks is within (including) t frames, then it can be determined as a double-click action.

[0096] In one embodiment, when the collection time interval is greater than the preset time interval, it is determined that the current user's hand movement has not triggered the preset unbinding action. At this time, the smart wearable device does not respond and continues to collect the user's hand movement pictures through the image collection module.

[0097] Furthermore, after detecting that there are at least two overlapping point coordinates in the time series coordinate data, it also includes: determining the interval frame number between the hand movement pictures corresponding to each of the overlapping point coordinates based on the time series coordinate data; when the interval frame number is less than the preset interval frame number, determining that the hand movement of the current user is the preset unbinding action, and generating the device unbinding instruction.

[0098] In one embodiment, the images captured by the image acquisition device are arranged in order of acquisition time, and the acquisition time interval between two adjacent frames of images is the same. For example, if the frame rate is set to 25FPS, the acquisition time interval between two adjacent frames of the captured images is 1 / 25 second. Therefore, the acquisition interval time between the two frames of images can be calculated based on the number of interval frames.

[0099] In one embodiment, upon detecting the presence of at least two overlapping point coordinates in the time-series coordinate data, the number of frames between the hand action images corresponding to the overlapping point coordinates may be calculated. For example, if the preset unbinding action is a double-click, the number of frames between the hand action images corresponding to two adjacent overlapping point coordinates may be calculated. If the number of frames is less than the preset number of frames, it is determined that the current user's hand action has triggered the preset unbinding action. When the preset unbinding action is triggered, the smart wearable device generates a device unbinding instruction. If the number of frames is greater than the preset number of frames, it is determined that the current user's hand action has not triggered the preset unbinding action, and the smart wearable device does not need to respond to this hand action.

[0100] Step S202: Based on the device unbinding instruction, the binding state between the mobile terminal and the extended screen component is released, so that the smart wearable device and the mobile terminal are in the extended screen interaction state.

[0101] In one embodiment, after the mobile terminal is unbound from the extended screen assembly, the extended screen assembly is fixedly displayed on the display screen of the smart wearable device and does not move with the movement of the mobile terminal. In this case, the mobile terminal can be used as a touch device. When the user moves or taps the mobile terminal, the smart wearable device tracks and calculates the coordinates of a corner point of the mobile terminal (such as the upper left corner) in real time, and recognizes the movement of the mobile terminal.

[0102] In this embodiment, the hand movements made by the current user are identified through the image acquisition device, and when it is determined that the hand movements made by the current user are preset unbinding movements, a device unbinding instruction is triggered to be generated, and according to the device unbinding instruction, the binding state of the mobile terminal and the extended screen component is released, so that the extended screen component is fixedly mapped in the display interface of the smart wearable device, thereby facilitating the smart wearable device to track and identify the feature points of the mobile terminal, and identify the interactive movements made by the current user to manipulate the mobile terminal, so that the current user can directly manipulate the extended screen component through the display interface of the smart wearable device, thereby improving the convenience of the interactive functions between the smart wearable device and the mobile terminal.

[0103] Please refer to FIG4 , which is a flow chart of a third embodiment of a method for interaction between a smart wearable device and a mobile terminal provided in an embodiment of the present application.

[0104] As shown in FIG4 , based on the embodiment shown in FIG3 , before step S201 , the following steps are further included:

[0105] Step S301: collecting the outline coordinates of the mobile terminal based on the image acquisition device;

[0106] In one embodiment, the smart wearable device recognizes the outline of the mobile terminal through an image acquisition device, and then calculates the outline coordinates of the outline of the mobile terminal in the two-dimensional coordinate system of the smart wearable device through a computer vision algorithm.

[0107] In one embodiment, computer vision refers to the ability of computers and systems to extract meaningful information from images, videos, and other visual inputs and to take actions or provide recommendations based on that information. Computer vision training machines rely on cameras, data, and computer vision algorithms to achieve capabilities such as identifying objects, their distance, whether they are moving or still, and whether there are any issues in the image. Computer vision algorithms can include feature extraction, feature point tracking, and coordinate calculation.

[0108] In one embodiment, feature extraction and matching is an important task in many computer vision applications, and is widely used in the fields of motion structure, image retrieval, object detection, etc. Features may be specific structures in an image, such as points, edges, or objects.

[0109] In one embodiment, the feature point tracking algorithm does not track the target as a whole. Instead, it matches and tracks a set of feature points (such as boundary lines, centroids, corner points, etc.) in consecutive image frames. Feature-based tracking methods primarily involve two steps: feature extraction and matching. The main advantage of this type of tracking method is that even if the target in the scene is partially obscured, as long as the feature points are visible, the target can be continuously tracked.

[0110] In one embodiment, coordinate calculation is to calculate the position coordinates of the target feature point in the coordinate system. Because the pictures captured by the image acquisition device of the smart wearable device are all within the display interface of the smart wearable device, the pictures captured by the image acquisition device all correspond to the same coordinate system. As long as the contour feature points of the mobile terminal are extracted and the positions of the contour feature points in the picture are calculated, the contour coordinates of the mobile terminal can be obtained.

[0111] Step S302: Obtaining a relative positional relationship between at least one extended screen component and the mobile terminal;

[0112] In one embodiment, as shown in FIG5 , the relative position relationship between the extended screen components and the mobile terminal depends on the position of the applications corresponding to the extended screen components on the mobile terminal screen. The extended screen components may be distributed around the mobile terminal.

[0113] In one embodiment, when implementing the mapping of extended screen components, the extended screen components are arranged in order based on the distribution and arrangement order of the applications on the mobile terminal. When there are multiple applications, the distribution of the multiple applications on the mobile terminal is generally arranged in rows and columns, that is, the distribution position of each application on the mobile terminal can be represented by rows and columns. For example, application 1 - first row, third column, etc.

[0114] In one embodiment, the extended screen components can be mapped directly to the display screen of the smart wearable device in their original positions according to the distribution of applications, or they can be mapped to the side of the mobile terminal in columns and rows. For example, the extended screen components corresponding to the first row of applications are mapped above the mobile terminal, and the extended screen components are arranged in the order of the application arrangement; the extended screen components corresponding to the first column of applications are mapped to the left side of the mobile terminal, etc.

[0115] In one embodiment, the display interface of the smart wearable device can be arranged on the side of the mobile terminal based on the location of the desktop applications corresponding to the extended screen components on the mobile terminal screen, with the mobile terminal outline as the center. For example, if the first desktop application is located in the middle of the left side of the mobile terminal screen, the extended screen component corresponding to the first desktop application will be expanded and displayed in the middle of the left side of the mobile terminal outline on the display interface of the smart wearable device.

[0116] Step S303: Based on the outline coordinates of the mobile terminal and the relative position relationship, the extended screen component is mapped and displayed in the display interface of the smart wearable device to activate the extended screen mode, wherein in the extended screen mode, the mobile terminal and the extended screen component are in a bound state.

[0117] In one embodiment, based on the outline coordinates and relative position relationship of the mobile terminal, the regional coordinates of the extended screen component area corresponding to each extended screen component in the display interface of the smart wearable device are calculated, and then the extended screen component is mapped and displayed at the corresponding extended screen component area in the display interface to complete the activation of the extended screen mode. For example, as shown in Figure 6, the first extended screen component is located on the left side of the mobile terminal, and there is only the first extended screen component on the left side of the mobile terminal. At this time, the outline coordinates of the mobile terminal can be ((100,10), (150,10), (100,110), (150,110)). Based on the relative position relationship between the mobile terminal and the first extended screen component, the regional coordinates of the extended screen component area corresponding to the first extended screen component can be calculated as ((10,10), (100,10), (10,110), (100,110)).

[0118] Furthermore, first size information of the mobile terminal and second size information of the extended screen assembly are obtained; based on the first size information, the second size information, the relative position relationship, and the outline coordinates of the mobile terminal, the diagonal coordinate information of the extended screen assembly in the display interface is calculated; and based on the diagonal coordinate information, the extended screen assembly area corresponding to each extended screen assembly is determined. The first size information of the mobile terminal includes the actual device size of the mobile terminal, including length, width, etc.

[0119] In one embodiment, the first size information of the mobile terminal is known and can be obtained by measuring with a size measurement tool or by querying the mobile terminal manufacturer's official website. The current user can preset the second size information of each extended screen component based on the first size information of the mobile terminal, for example, by proportionally enlarging the display size based on the number of extended screen components so that the extended screen components can be fully and reasonably displayed around the mobile terminal.

[0120] For example, if there are four applications in the first row of the mobile terminal screen, the extended screen components of these four applications are all mapped to the upper side of the mobile terminal, and the first size information of the mobile terminal is 8×16cm, that is, the length dimension of the upper side of the mobile terminal is 8cm, then the second size information of the above four extended screen components can be a maximum of 2×ncm, that is, the length dimension of the four extended screen components can be a maximum of 2cm, and the width can be adjusted according to the actual application situation (as shown in Figure 7); or the length dimension of the above four extended screen components can be a maximum of 4×ncm, that is, the length dimension of the four extended screen components can be a maximum of 4cm (as shown in Figure 8), wherein the two extended screen components corresponding to the first application and the fourth application are respectively distributed at the two upper diagonal positions of the mobile terminal.

[0121] In one embodiment, the enlargement ratio of the extended screen assembly can be set according to the relative position relationship between the mobile terminal and the extended screen assembly. For example, if the mobile terminal is rectangular, the enlargement ratio of the extended screen assembly on the long side can be set larger, and the enlargement ratio of the extended screen assembly on the short side can be set smaller.

[0122] In one embodiment, based on the first size information, the second size information, the relative position relationship and the outline coordinates of the mobile terminal, the diagonal coordinate information of the component frame of each extended screen component in the display interface of the smart wearable device can be calculated, such as the upper left corner coordinates and the lower right corner coordinates; based on the diagonal coordinate information, the area range of each extended screen component in the display interface of the smart wearable device can be represented.

[0123] For example, the pixel size of the mobile terminal in the two-dimensional coordinate system is calculated based on the outline coordinates of the mobile terminal, that is, the number of pixels occupied by the length of the mobile terminal and the number of pixels occupied by the width. The ratio of the first size information and the pixel size is the size ratio of the actual size of the device and the pixel size of the two-dimensional coordinate system, and the ratio of the second size information and the size ratio is the pixel size of the extended screen component. Based on the relative position relationship and the outline coordinates of the mobile terminal, the starting point coordinates of each extended screen component in the display interface can be known. Based on the starting point coordinates and pixel size corresponding to each extended screen component, the diagonal coordinate information of the component frame of each extended screen component can be calculated, thereby obtaining the extended screen component area corresponding to each extended screen component.

[0124] In this embodiment, the outline coordinates of the mobile terminal are collected through the image acquisition device to determine the position of the mobile terminal in the display interface of the smart wearable device. Therefore, the position of the extended screen component in the display interface of the smart wearable device is determined based on the relative position relationship between the extended screen component and the mobile terminal. In addition, accurate mapping of the extended screen component is achieved, avoiding confusion in the mapping position of the extended screen component, adapting to the user's usage habits, and improving the user's experience of the interactive function between the smart wearable device and the mobile terminal.

[0125] Please refer to Figure 9, which is a structural diagram of a first embodiment of an interactive device between a smart wearable device and a mobile terminal provided in this application. The interactive device between the smart wearable device and the mobile terminal is used to execute the aforementioned interactive method between the smart wearable device and the mobile terminal.

[0126] As shown in FIG9 , the apparatus 400 for interacting with a smart wearable device and a mobile terminal includes: a posture detection module 401 , a corner point coordinate acquisition module 402 , a component determination module 403 , and a component operation module 404 .

[0127] The posture detection module 401 is used to detect the current posture of the mobile terminal when the smart wearable device is in an extended screen interaction state with the mobile terminal;

[0128] The corner point coordinate acquisition module 402 is configured to acquire the corner point coordinates of a target corner point on the mobile terminal when detecting that the current posture of the mobile terminal matches a preset interactive operation action;

[0129] A component determination module 403 is configured to determine a target extended screen component based on an extended screen component area corresponding to the corner point coordinates;

[0130] The component operation module 404 is configured to, when the current gesture action of the current user matches a preset gesture operation action, perform a target operation on the target extended screen component based on the component operation instruction corresponding to the gesture operation action.

[0131] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned embodiment of the interaction method between the smart wearable device and the mobile terminal, and will not be repeated here.

[0132] The apparatus provided in the above embodiment may be implemented in the form of a computer program, which may be run on a computer device as shown in FIG10 .

[0133] Please refer to Figure 10, which is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0134] 10 , the computer device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0135] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions that, when executed, enable the processor to execute any method for interacting with the smart wearable device and the mobile terminal.

[0136] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0137] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any method for interaction between the smart wearable device and the mobile terminal.

[0138] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that the structure shown in FIG10 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0139] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0140] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the interaction methods between the smart wearable device and the mobile terminal provided in the embodiment of the present application.

[0141] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., equipped on the computer device.

[0142] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for interaction between a smart wearable device and a mobile terminal, the method comprising: When the smart wearable device is in an extended screen interaction state with the mobile terminal, detecting the current posture of the mobile terminal; When it is detected that the current posture of the mobile terminal matches the preset interactive operation action, collecting the corner point coordinates of the target corner point on the mobile terminal; Determining a target extended screen component based on an extended screen component area corresponding to the corner point coordinates; When the current gesture action of the current user matches the preset gesture operation action, the target operation is performed on the target extended screen component based on the component operation instruction corresponding to the gesture operation action.

2. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: When the smart wearable device is in an extended screen interaction state with the mobile terminal, before detecting the current posture of the mobile terminal, the smart wearable device further includes: Based on the image acquisition device, identifying the preset unbinding action made by the current user, and generating a device unbinding instruction; Based on the device unbinding instruction, the binding state between the mobile terminal and the extended screen component is released, so that the smart wearable device and the mobile terminal are in the extended screen interaction state.

3. The method for interaction between a smart wearable device and a mobile terminal according to claim 2, wherein: The image acquisition device is based on identifying the preset unbinding action made by the current user and generating a device unbinding instruction, including: Based on the image acquisition device, the hand motion pictures of the current user are acquired to obtain a hand picture group, wherein the hand motion picture group includes at least two of the hand motion pictures; Based on a preset key point tracking algorithm, extract the point coordinates of the target key points in the hand action picture to obtain the time series coordinate data of the target key points; When it is detected that there are at least two coincident point coordinates in the time series coordinate data, based on the collection time corresponding to each of the coincident point coordinates, the collection time interval of each of the coincident point coordinates is calculated; When the collection time interval is less than the preset time interval, the hand movement of the current user is determined to be the preset unbinding movement, and the device unbinding instruction is generated.

4. The method for interaction between a smart wearable device and a mobile terminal according to claim 3, wherein: After detecting that there are at least two coincident point coordinates in the time series coordinate data, the method further includes: Based on the time series coordinate data, determining the interval frame number between the hand action pictures corresponding to the coordinates of each of the coincident points; When the interval frame number is less than the preset interval frame number, the hand action of the current user is determined to be the preset unbinding action, and the device unbinding instruction is generated.

5. The method for interaction between a smart wearable device and a mobile terminal according to claim 2, wherein: Before the image acquisition device is used to identify the preset unbinding action performed by the current user and generate the device unbinding instruction, the method further includes: Based on the image acquisition device, the outline coordinates of the mobile terminal are acquired; Acquire a relative position relationship between at least one extended screen component and the mobile terminal; Based on the outline coordinates of the mobile terminal and the relative position relationship, the extended screen component is mapped and displayed in the display interface of the smart wearable device to activate the extended screen mode, wherein in the extended screen mode, the mobile terminal and the extended screen component are in a bound state.

6. The method for interaction between a smart wearable device and a mobile terminal according to claim 5, wherein: Before mapping and displaying the extended screen component in the display interface of the smart wearable device based on the outline coordinates of the mobile terminal and the relative position relationship to activate the extended screen mode, the method further includes: Acquire first size information of the mobile terminal and second size information of the extended screen assembly; Calculate the diagonal coordinate information of the extended screen component in the display interface based on the first size information, the second size information, the relative position relationship, and the outline coordinates of the mobile terminal; Based on the diagonal coordinate information, the extended screen component area corresponding to each extended screen component is determined.

7. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: When the current gesture action of the current user matches the preset gesture operation action, before performing the target operation on the target extended screen component based on the component operation instruction corresponding to the gesture operation action, the method further includes: Acquire a preset gesture instruction library, wherein the gesture instruction library includes at least one gesture operation action and the component operation instruction corresponding to each gesture operation action; Based on a gesture recognition algorithm, the current gesture action is compared with each of the gesture operation actions in the gesture instruction library to determine the similarity between the current gesture action and each of the gesture operation actions; When the similarity is greater than or equal to a preset similarity, the gesture operation action matching the current gesture action is determined based on the similarity.

8. The method for interaction between a smart wearable device and a mobile terminal according to claim 7, wherein: The comparing the current gesture action with each of the gesture operation actions in the gesture instruction library based on the gesture recognition algorithm to determine the similarity between the current gesture action and each of the gesture operation actions includes: Based on the gesture recognition algorithm, calculating the motion trajectory of the current gesture action; The similarity between the motion trajectory of the current gesture action and the motion trajectory of each of the gesture operation actions is calculated to determine the similarity between the current gesture action and the gesture operation actions.

9. The method for interaction between a smart wearable device and a mobile terminal according to claim 8, wherein: The motion trajectory includes the coordinate trajectory of the key points of the hand in time series; Wherein, the key points of the hand include joint points and / or fingertips.

10. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: When the similarity is greater than or equal to a preset similarity, determining the gesture operation action matching the current gesture action based on the similarity includes: When the similarity between the current gesture action and multiple gesture operation actions is greater than or equal to the preset similarity, it is determined that the gesture operation action with the highest similarity matches the current gesture action.

11. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The detecting that the current posture of the mobile terminal matches a preset interactive operation action includes: Calculating the similarity between the current posture and the interactive operation action; When the similarity between the current posture and the interactive operation action is greater than or equal to a similarity threshold, it is determined that the current posture of the mobile terminal matches the preset interactive operation action.

12. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein the step of collecting the corner point coordinates of the target corner point on the mobile terminal comprises: Calculating the coordinates of intersections of the edges of the mobile terminal based on the outline coordinates of the mobile terminal; Based on the correspondence between the coordinates of the corner points of the arc angles of the mobile terminal, the coordinates of the arc points on each arc angle that are closest to the corresponding corner point coordinates are calculated to obtain the corner point coordinates.

13. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The determining the target extended screen component based on the extended screen component area corresponding to the corner point coordinates includes: When the landing point coordinates of the target corner point point to the same extended screen component area within a preset time, the extended screen component corresponding to the extended screen component area is determined to be the target extended screen component.

14. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The determining the target extended screen component based on the extended screen component area corresponding to the corner point coordinates further includes: When the intersection coordinates of the target corner points stay in any extended screen component area for a time that reaches a preset time threshold, the extended screen component corresponding to the extended screen component area is determined to be the target extended screen component.

15. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The determining the target extended screen component based on the extended screen component area corresponding to the corner point coordinates further includes: When the corner point coordinates do not belong to any of the extended screen component areas, calculating the distance between the corner point coordinates and the midpoint position of each of the extended screen component areas; The extended screen component corresponding to the extended screen component area with the smallest distance is determined as the target extended screen component.

16. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The interactive operation actions include double-clicking, maintaining a prohibited state for a preset time, and sliding.

17. The method for interaction between a smart wearable device and a mobile terminal according to claim 1, wherein: The detecting the current posture of the mobile terminal comprises: When the mobile terminal moves into a capture screen of an image capture device, identifying contour information, corner point coordinate information, and a moving trajectory of the mobile terminal based on the image capture device; Based on the outline information, the corner point coordinate information and the moving trajectory, the current posture of the mobile terminal is determined.

18. An interactive device between a smart wearable device and a mobile terminal, wherein: The interactive device between the smart wearable device and the mobile terminal includes: The posture detection module is used to detect the current posture of the mobile terminal when the smart wearable device is in an extended screen interaction state with the mobile terminal; A corner point coordinate acquisition module, configured to acquire the corner point coordinates of a target corner point on the mobile terminal when detecting that the current posture of the mobile terminal matches a preset interactive operation action; A component determination module, used to determine a target extended screen component based on an extended screen component area corresponding to the corner point coordinates; The component operation module is used to perform a target operation on the target extended screen component based on the component operation instruction corresponding to the gesture operation when the current gesture action of the current user matches the preset gesture operation action.

19. A computer device, wherein: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method for interaction between a smart wearable device and a mobile terminal as described in any one of claims 1 to 17 are implemented.

20. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for interaction between a smart wearable device and a mobile terminal as described in any one of claims 1 to 17 are implemented.

Images