Unlock interaction processing method, device and equipment

Abstract

The embodiments of this specification disclose an unlocking interaction processing method, apparatus, and device, which are applied to a locked smart device. The solution includes: displaying a virtual object on the locked smart device to represent the distance status; if the mobile terminal device used by the user to unlock is detected through the short-distance identification area contained in the locked smart device, the virtual object begins to be displayed with a visual effect of approaching the short-distance identification area to represent that the distance between the virtual object and the mobile terminal device is shortening; during the approaching process, the user's user identification is obtained by identifying the mobile terminal device; after obtaining the user identification, the virtual object is moved closer to the short-distance identification area until it moves out of the locked smart device, and accordingly moves across the device into the display area of ​​the mobile terminal device to represent that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; and an unlocking instruction is obtained based on the user identification to unlock the locked smart device.

Description

Technical Field

[0001] This specification relates to the field of user interaction technology, and in particular to unlocking interaction processing methods, devices, and equipment. Background Art

[0002] With the development of Internet technology and the popularization of smart phones, more and more businesses are conducted on smart phones through corresponding applications. With the rise of the Internet of Things, some IoT smart devices also require users to interact through smartphones so that users can obtain the services provided by IoT smart devices. A typical example is vending machines. Generally, vending machines pay for a product in the machine by scanning a code. Then, the machine automatically spits out the product from the delivery port below, so that the user can get the product.

[0003] In order to enable users to select products more freely and conveniently, another type of vending machine has come into being. Users can scan the code of this type of vending machine to trigger the unlocking of the vending machine. Then users can open the door of the machine and reach into the machine more freely to select the required products. After closing the door, they can make the payment.

[0004] For the above-mentioned vending machines that allow users to unlock and open the door, the current unlocking interaction method is still relatively cumbersome, and because users scan the code through the air, the interaction feels weak. Not only that, if multiple people want to try to use the vending machine at the same time, it is easy to cause confusion and uncertainty in the operation.

[0005] Based on this, for vending machines or other similar IoT smart devices that allow users to unlock and open their doors, a simpler, more interactive unlocking interaction solution is needed to help enhance the user's sense of certainty. Summary of the Invention

[0006] One or more embodiments of this specification provide unlocking interaction processing methods, devices, equipment, and storage media to solve the following technical problems: For vending machines or other similar IoT smart devices that allow users to unlock and open doors, a simpler, more interactive unlocking interaction solution is needed that helps enhance the user's sense of certainty.

[0007] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0008] One or more embodiments of this specification provide an unlocking interaction processing method, which is applied to a locked smart device. The method includes:

[0009] Displaying a virtual object for representing the distance status on the locked smart device;

[0010] If the mobile terminal device used by the user to unlock the device is detected through the short-distance identification area included in the locked smart device, a visual effect of the virtual object approaching the short-distance identification area is displayed to indicate that the distance between the virtual object and the mobile terminal device is shortening;

[0011] During the approaching process, obtaining a user identification of the user by identifying the mobile terminal device;

[0012] After obtaining the user identification, the virtual object is moved closer to the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0013] An unlocking instruction is acquired according to the user identification, and in response to the unlocking instruction, the locked smart device is unlocked so as to facilitate use by the user.

[0014] One or more embodiments of this specification provide an unlocking interaction processing method, which is applied to a user's mobile terminal device, and includes:

[0015] The user moves the mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user ID of the user by identifying the mobile terminal device;

[0016] During the approaching process, a virtual object representing the locked smart device displayed on the mobile terminal device moves from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0017] After the locked smart device is unlocked according to the acquired user identification, the locked smart device responds accordingly according to the user's use of the locked smart device.

[0018] One or more embodiments of this specification provide an unlocking interaction processing device, which is applied to a locked smart device. The device includes:

[0019] A virtual object display module, which displays a virtual object for representing the distance status on the locked smart device;

[0020] an approach effect response module, which, upon detecting the mobile terminal device used by the user to unlock the device through the short-distance identification area included in the locked smart device, begins to display a visual effect of the virtual object approaching the short-distance identification area to indicate that the distance between the virtual object and the mobile terminal device is shortening;

[0021] A user identification recognition module, which obtains the user identification of the user by identifying the mobile terminal device during the approach process;

[0022] a virtual object removal module, after acquiring the user identifier, causing the virtual object to approach the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0023] In response to the unlocking module, an unlocking instruction is obtained according to the user identification, and in response to the unlocking instruction, the locked smart device is unlocked so that the user can use it.

[0024] One or more embodiments of this specification provide an unlocking interaction processing device, which is applied to a user's mobile terminal device, and includes:

[0025] The user identification matching module is used to move the user's handheld mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user's user identification by identifying the mobile terminal device;

[0026] a virtual object moving-in module, which, during the approaching process, displays a virtual object representing the locked smart device on the mobile terminal device, and moves it from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0027] The unlocking and use response module responds accordingly according to the user's use of the locked smart device after the locked smart device is unlocked according to the acquired user identification.

[0028] At least one of the above technical solutions adopted in one or more embodiments of this specification can achieve the following beneficial effects: providing a new unlocking interaction method across physical and digital media, and a corresponding locked smart device (e.g., a vending machine); displaying a virtual object on the locked smart device, and in conjunction with the process of the user bringing the mobile terminal device close to the short-range identification area (e.g., based on NFC identification) contained in the locked smart device, correspondingly displaying the visual effect of the virtual object approaching the short-range identification area, and after successfully identifying the user identifier, causing the virtual object to further move out of the locked smart device and correspondingly move across the device into the display area of ​​the mobile terminal device, which The feeling that this kind of interaction process gives to the user is based on the clear prompts or guidance provided by the above visual effects. The virtual object representing the locked smart device runs accurately from the locked smart device to the mobile terminal device like a tangible object, as if it has become a physical lock to be unlocked, dynamically and actively stuck into the user's hand to be unlocked by the user, and actually triggers the locked smart device to automatically unlock. The whole process is simple for the user to operate, and the interaction feels stronger, which helps to enhance the user's sense of certainty. It can be more clear which user is triggering the unlocking and which state the unlocking process is in at the moment, which helps to avoid user misoperation and is also interesting. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0030] Figure 1 A flowchart of an unlocking interaction processing method for a locked smart device provided in one or more embodiments of this specification;

[0031] Figure 2 A flowchart of a virtual object cross-device transfer effect solution provided in one or more embodiments of this specification;

[0032] Figure 3 A flowchart of a virtual object movement effect solution provided in one or more embodiments of this specification;

[0033] Figure 4 A flowchart of a virtual object transfer simulacrum effect solution in an application scenario provided by one or more embodiments of this specification;

[0034] Figure 5A flowchart of a solution for delayed unlocking of a locked smart device based on a virtual object according to one or more embodiments of this specification;

[0035] Figure 6 A flowchart of an unlocking interaction processing method applied to a user's mobile terminal device provided in one or more embodiments of this specification;

[0036] Figure 7(a) to Figure 7(d) In an application scenario provided by one or more embodiments of this specification, Figure 1 A schematic diagram showing the effect of a specific implementation scheme of the method;

[0037] Figure 8 A schematic diagram of the structure of an unlocking interaction processing device for a locked smart device provided in one or more embodiments of this specification;

[0038] Figure 9 A schematic diagram of the structure of an unlocking interaction processing device applied to a user's mobile terminal device provided in one or more embodiments of this specification;

[0039] Figure 10 A schematic diagram of the structure of an unlocking interaction processing device for a locked smart device provided in one or more embodiments of this specification;

[0040] Figure 11 This is a schematic structural diagram of an unlocking interaction processing device applied to a user's mobile terminal device provided in one or more embodiments of this specification. DETAILED DESCRIPTION

[0041] The embodiments of this specification provide unlocking interaction processing methods, devices, equipment, and storage media.

[0042] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0043] Regarding the issues mentioned in the background technology, this application considers giving priority to the use of short-range wireless communication methods such as NFC to unlock IoT smart devices such as vending machines, rather than the more cumbersome code scanning unlocking method; it should be noted that not only short-range wireless communication methods are used here, but more importantly, a new unlocking interaction method based on cross-physical and digital media is proposed, which makes the cooperation process between the unlocking device and the unlocked device more interactive and dynamic for the user, and helps to enhance certainty and fun.

[0044] To implement the solution of this application, the new smart lock device provided by this application can be used, or existing smart lock devices can be improved, or corresponding improvements can be made on the user's mobile terminal device. If these devices already support the above-mentioned short-range wireless communication method, the solution of this application can be implemented through software upgrades, which is relatively low-cost.

[0045] The solution of this application is described in detail below. The solution involves a locked smart device and a user's mobile terminal device, and is mainly described from the perspective of the locked smart device.

[0046] Figure 1 A flowchart of an unlocking interaction processing method provided for one or more embodiments of this specification. The method can be applied to a locked smart device. From a software perspective, the execution subject of the process includes the application client or server on the locked smart device. From a hardware perspective, the execution subject of the process includes the locked smart device itself. The locked smart device can be an upgraded version of an existing service device such as a vending machine, a smart water dispenser, or a smart hair dryer, or it can also be a newly built locked smart device. The user's mobile terminal device can typically be a smartphone or a smart watch. In addition, it can also be other mobile terminal devices such as a tablet computer, a portable game console, etc.

[0047] The lock on a smart device with a lock can be a physical hardware lock or a software lock. For a physical lock, the smart device can have a door that can be locked by the physical lock, such as a vending machine door. For a software lock, however, the device may not have a corresponding door. This application focuses specifically on the former scenario and provides more dynamic interactive unlocking methods for this scenario.

[0048] Figure 1 The process in includes the following steps:

[0049] S102: Displaying a virtual object for representing a distance status on the locked smart device.

[0050] In one or more embodiments of the present specification, skeuomorphic or anthropomorphic virtual objects are used to more vividly and intuitively represent dynamic interaction methods, so that users can understand and cooperate more accurately.

[0051] Taking skeuomorphism as an example, the degree of skeuomorphism can be adjusted as needed: for example, it can be packaged as a virtual object that primarily displays a contour shape (e.g., a circle, oval, arrow, etc.) without necessarily showing the object's specific identity; another example is a virtual, more concrete spherical object (a glass ball, bubble, sports ball, etc.) or a virtual metal object that can be attracted by a magnet (a coin, medal, etc.); and so on. When presenting visual effects related to movement, the virtual object can be made to exhibit dynamic effects of elastic deformation to more vividly convey the movement process.

[0052] Virtual objects can be used to represent the locked smart device itself, or to represent related information that requires interaction (for example, device information, authentication information, order information, etc.). In the latter case, in particular, they can be more deeply materialized or personified to create a more vivid and flexible style image for these information, thereby weakening the abstract feeling that these information may originally bring to people as a bunch of text, numbers or tables, thereby making various visual effects more natural and smooth.

[0053] In one or more embodiments of this specification, a locked smart device includes a short-range identification area for identifying a user's mobile terminal device. A virtual object can represent the distance between the content it represents and the short-range identification area. If the locked smart device detects a mobile terminal device approaching the short-range identification area for identification, the virtual object can represent the distance between the content it represents and the mobile terminal device. This distance can be a relatively precise relative distance or simply represent the trend of distance change.

[0054] Based on a triggering condition, the locked smart device can initially display a virtual object with a transfer waiting visual effect (for example, stationary, slightly shaking, or with medium power accumulation. To more clearly show that it can be transferred, a guide line or other effects can also be presented). The predetermined condition may be that the locked smart device is in a bright screen standby state, or that the locked smart device has just discovered the mobile terminal device. This transfer waiting visual effect can, on the one hand, give the user the feeling that the corresponding interactive information can be taken out of the locked smart device (the degree of such hint or explicitness can be adjusted according to actual needs). On the other hand, it can even optionally indicate the corresponding transfer direction. Based on the transfer direction, the user can intuitively understand where they should move the mobile terminal device closer.

[0055] S104: If the mobile terminal device used by the user to unlock the device is found through the short-distance identification area included in the locked smart device, a visual effect of the virtual object approaching the short-distance identification area is displayed to show that the distance between the virtual object and the mobile terminal device is shortened.

[0056] In one or more embodiments of this specification, a locked smart device displays a virtual object through its display area. The locked smart device also includes a short-range recognition area, which is a separate area from the display area. These two areas can be positioned relatively close to each other to facilitate a smoother and more natural transition across the device without being abrupt.

[0057] The short-range recognition area can be, for example, an NFC recognition area that performs recognition based on induction, or an area that performs recognition based on other short-range wireless communication methods (e.g., Bluetooth, etc.). Alternatively, a camera can be used for short-range image recognition, which can be used in conjunction with short-range wireless communication.

[0058] In one or more embodiments of this specification, a time of flight (TOF) sensor deployed on or near the locked smart device is used to detect the surrounding environment so that the locked smart device can detect an approaching mobile terminal device. If the mobile terminal device is successfully detected, the display area of ​​the locked smart device can be triggered to start displaying a visual effect of a virtual object approaching the short-range recognition area. To improve realism, the visual effect can be displayed when the locked smart device is relatively close to the short-range recognition area (at this time, it may not be successfully recognized) and there is room for further approach.

[0059] This visual effect also serves as a prompt for users. As expected, the user should hold the mobile terminal device and move it close to the short-distance recognition area for identification. Therefore, this action process is more consistent with the visual effect. Of course, the visual effect can be displayed more synchronously according to the progress of the action, thereby improving the sense of consistency.

[0060] Instead of directly indicating the corresponding transfer direction in the transfer waiting visual effect, the approaching direction (i.e., the transfer direction) can be indicated when displaying the aforementioned approaching visual effect. In this case, the transfer waiting visual effect can focus more on expressing the feeling of waiting for the user to take the item, effectively grabbing the user's attention and encouraging them to make accurate and timely actions.

[0061] Furthermore, in the aforementioned waiting, approaching, and other states, additional auxiliary effects such as lighting and vibration can be used to enhance the interaction. In this case, the locked smart device does not even need to display text messages directly prompting the user to interact; the user can correctly understand the interaction based on the visual effects, additional lighting effects, or other auxiliary effects.

[0062] S106: During the approaching process, the user identification of the user is obtained by identifying the mobile terminal device.

[0063] As the mobile terminal device approaches the short-range identification zone, when the distance between the two is close enough (for example, a distance of several centimeters is considered close enough for NFC identification), the short-range identification zone can successfully identify the mobile terminal device and, based on the successful identification, further obtain the user ID of the corresponding user logged in on the mobile terminal device. As a result, both parties or even more parties (for example, other users who also want to unlock the locked smart device) can clearly know which user is currently performing the unlocking interaction, avoiding the confusion that may be caused by the code scanning unlocking method.

[0064] In one or more embodiments of this specification, the short-range identification area includes a short-range wireless communication identification area, which is used for identification by bringing a mobile terminal device close to the area. In this case, the user identification of the user can be read from the mobile terminal device using the short-range wireless communication method corresponding to the short-range wireless communication identification area, such as by NFC.

[0065] S108: After obtaining the user identifier, the virtual object is moved closer to the short-distance identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device.

[0066] After obtaining the user ID, the locked smart device can be automatically unlocked for the corresponding user. However, the focus here is on using further more drastic changes in the virtual object to show the user that the current unlockable state has been achieved.

[0067] In one or more embodiments of the present specification, the visual effect of approaching, moving out, and then moving in across devices can more vividly show on the locked smart device the direction in which the virtual object is currently moving and the final destination of the transfer. The direction can especially include the direction of the short-distance identification area. Since the mobile terminal device is close to the short-distance identification area at this time, the visual effect actually shows that the virtual object is dynamically transferring from the locked smart device to the mobile terminal device, as if the digital medium (i.e., the content represented by the virtual object) breaks through the limitations of the virtual digital world and is about to escape from a physical medium (i.e., the locked smart device) and cross the real space to rush to another physical medium (i.e., the mobile terminal device), so as to dynamically prompt and promote the completion of unlocking. This high-tech virtual interactive effect is more in line with user intuition, giving users a strong sense of immediacy and certainty, and helping users understand what is happening and which specific stage they have reached.

[0068] In one or more embodiments of the present specification, after approaching and moving out, the mobile terminal device may also cooperate accordingly to present the visual effect of moving in across devices. In order to improve visual coordination and scene consistency, the visual effect of approaching and moving out and the visual effect of moving in across devices may belong to a unified set of effects. The mobile terminal device may also upgrade the corresponding application in advance so as to be able to present the visual effect of moving in across devices.

[0069] Furthermore, after the virtual object is received and displayed on the mobile terminal device, the user can more conveniently know that he is currently in control of the lock of the locked smart device. Furthermore, the user can actively confirm (for example, the user clicks a confirmation button on the virtual object or actively triggers face recognition to indicate confirmation, etc.), or the lock can be automatically unlocked directly and adaptively, without the need for the user to actively confirm again, so that the operation is simpler. If active confirmation is required, visual effects can also be added to the confirmation stage, such as using a virtual object in the shape of a bubble, and the user clicks to pop the bubble to indicate confirmation of unlocking, etc., which helps to make the entire process better closed-loop in the dimension of visual expression.

[0070] In one or more embodiments of this specification, the identification of a mobile terminal device by a locked smart device and the cross-device transfer of a virtual object after successful identification may be performed using the same method (e.g., both based on the short-range wireless communication method corresponding to the short-range wireless communication identification area) or different methods (e.g., identification using image recognition and transfer using short-range wireless communication; or, another example, identification and transfer using two different short-range wireless communication methods). The specific implementation method may be determined based on cost or efficiency requirements.

[0071] S110: Obtain an unlocking instruction according to the user identification, and unlock the locked smart device in response to the unlocking instruction so that the user can use it.

[0072] In one or more embodiments of the present specification, the unlocking instruction is issued by the mobile terminal device based on the interaction process, or the locked smart device is automatically triggered to generate according to the user identification. The unlocking instruction thus obtained is generated specifically for the user, and the corresponding unlocking is also unlocked for the user. Afterwards, before relocking, the business performed on the unlocked locked smart device (for example, selling goods, taking a drinking cup, using the hair dryer function, etc.) belongs to the user. After the business is completed, the user can manually relock it, for example, by locking it with the corresponding operation on the mobile terminal device, or closing the cabinet door to automatically lock it. If payment is required, the user can be automatically charged or the user can pay manually.

[0073] Taking a locked smart device including a vending machine as an example, when unlocking, the vending machine can be unlocked and the door of the vending machine can be automatically opened, or the vending machine can be unlocked and the user is prompted that the door of the vending machine can be opened, and then the user can manually open the unlocked door.

[0074] pass Figure 1 A novel unlocking interaction method across physical and digital media is provided, along with a corresponding locked smart device (e.g., a vending machine). A virtual object is displayed on the locked smart device. As a user approaches a short-range identification zone (e.g., based on NFC recognition) within the locked smart device, a visual effect of the virtual object approaching the short-range identification zone is displayed. Upon successful recognition of the user's identifier, the virtual object is moved out of the locked smart device and then into the display zone of the mobile device. This interactive process gives the user the distinct feeling of a tangible object, as the visual effect provides clear guidance. The virtual object, representing the locked smart device, moves precisely from the locked smart device to the mobile device, as if transformed into a physical lock to be unlocked, dynamically and proactively placed in the user's hand to be manually unlocked. This in turn triggers the automatic unlocking of the locked smart device. The entire process is user-friendly and interactive, enhancing the user's sense of certainty. It provides a clearer understanding of the user's current unlocking status and helps prevent user errors. Furthermore, the process is engaging and engaging.

[0075] based on Figure 1 This specification also provides some specific implementation plans and extension plans of the method, which will be described below.

[0076] In order to improve the user's sense of instant interaction and interactive feel, one or more embodiments of this specification provide a flow diagram of a virtual object cross-device transfer effect solution, see Figure 2 In practical applications, it is also possible to selectively use only part of the steps in this process.

[0077] Figure 2 The process in includes the following steps:

[0078] S202: When the visual effect of the virtual object approaching the short-distance recognition area is displayed, a distance change between the mobile terminal device and the locked smart device is specifically detected.

[0079] When the distance is relatively far, the distance change can be detected immediately through TOF. When the distance is close enough, the distance change can be detected immediately through the short-distance recognition area.

[0080] S204: Start displaying a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change.

[0081] In one or more embodiments of the present specification, the distance change detected in real time is made as consistent as possible with the visual effect of approaching. Under this premise, the scale of the visual effect of approaching can be scaled to improve synchronization.

[0082] S206: When the virtual object approaches the short-distance recognition area, and when the virtual object has moved across devices into the display area of ​​the mobile terminal device, different dynamic light effects are used in the short-distance recognition area to prompt or guide.

[0083] More obviously, the dynamic light effect mode can be switched to correspondingly represent the switch between the approaching or not-yet-moved state and the moving-in state. Similarly, the mobile terminal can also provide prompts or guidance.

[0084] S208: When the virtual object approaches the short-distance recognition area, trigger the mobile terminal device to vibrate.

[0085] To convey the reality of a virtual object approaching, the mobile device can be triggered to vibrate. Specifically, at the moment a virtual object moves in or out, a more specialized vibration pattern can be used, such as a vibration similar to the sudden break of a stretched rubber band. This effectively captures the user's attention, enhances their tactile experience, and allows them to immediately notice the virtual object's entry into the mobile device.

[0086] Furthermore, in the above-mentioned transfer process of the virtual object, the transfer effects of the specific action modes and angles that can be adopted are also diverse, and a scheme with a stronger sense of dynamics and interactivity and a more vivid image can be selected as much as possible. A flowchart of a virtual object transfer motion effect scheme provided in one or more embodiments of this specification is shown in FIG. Figure 3 .

[0087] Figure 3 The process in includes the following steps:

[0088] S302: When starting to display the visual effect of the virtual object approaching the short-distance recognition area, specifically start to display the visual effect of the virtual object being attracted to the short-distance recognition area.

[0089] S304: When the virtual object is moved closer to the short-distance recognition area until it is moved out of the locked smart device, the virtual object is sucked out of the locked smart device in the direction of the short-distance recognition area.

[0090] The effects of suction and exhaust can be diverse, for example, making the virtual object shake or pull invisibly in a specified direction, showing a schematic line of air flow in the suction direction, making the virtual object deform in a specified direction, and so on.

[0091] From the user's perspective, the visual effect here is that the user brings the mobile terminal device close to the locked smart device, as if the mobile terminal device is an object such as an air pump, a water pump, or a magnet, and sucks out a virtual object that seems to be a physical medium from the locked smart device into the mobile terminal device. The method is novel and interesting, and can also bring users a more certain and immediate sense of interaction.

[0092] In addition to the suck-in and suck-out method, similarly, methods such as flying out and flying in, jumping out and jumping in, flashing out and flashing in can also be used to achieve the above-mentioned visual effects.

[0093] From these listed expressions, it can be seen that the more the dynamic interaction methods of some real entities can be imitated, the easier it will be for users to understand and cooperate more accurately.

[0094] In addition to the operation mode of the virtual object, this application also involves the virtual object itself in accordance with the operation mode and the scene in which it is located, so as to make the entire transfer process more natural and immersive. Taking the locked smart device as a device with a door handle, such as a vending machine, as an example, one or more embodiments of this specification provide a flow diagram of the virtual object transfer simulation effect solution in this application scenario, see Figure 4 .

[0095] Figure 4 The process in includes the following steps:

[0096] S402: Start displaying a visual effect of the virtual object approaching the virtual cooperating object displayed on the mobile terminal device in the short-distance recognition area, wherein the virtual object includes a virtual door handle for representing the door handle of the locked smart device, and the virtual cooperating object includes a virtual human hand. After the virtual door handle moves into the display area of ​​the mobile terminal device, the virtual human hand will hold the virtual door handle to display the visual effect of opening the door.

[0097] In this case, the process of unlocking the locked smart device by bringing the mobile terminal device close to it is transformed into a virtual human hand approaching the virtual door handle in terms of interactive vision. At the same time, the virtual door handle also caters to the virtual human hand, and through the instant change of distance, a two-way interactive feeling is achieved. In particular, the moment the virtual door handle moves across devices into the display area of ​​the mobile terminal device and completes the grip and docking with the virtual human hand gives the user a strong sense of certainty and rhythm. This moment is also the moment when the door can be successfully unlocked. The whole process maintains a consistent experience of virtual and real interaction.

[0098] S404: After the locked smart device is unlocked in response to the unlock instruction, detecting the commodity taken by the user from the locked smart device, generating and displaying a virtual commodity corresponding to the commodity on the locked smart device, wherein the locked smart device includes a vending cabinet.

[0099] S406: In cooperation with the user taking the product out of the locked smart device, the virtual product is moved closer to the short-range identification area until it is removed from the locked smart device, and is then moved across the device into the display area of ​​the mobile terminal device, so that payment is made based on the virtual product in the display area of ​​the mobile terminal device.

[0100] In one or more embodiments of this specification, in the case of a vending machine, the user's taken goods are detected immediately, and then, in a manner similar to unlocking, the goods are displayed as corresponding virtual goods in the display area of ​​the vending machine. As the user takes out the goods, the virtual goods move closer to the locked smart device and are then transferred across devices to the display area of ​​the mobile terminal device. The display area can, for example, display a virtual shopping basket for accommodating the virtual goods, thus making it convenient for the user to take one or more goods. In this case, the user can clearly and intuitively see the goods to be paid, without even having to use text to describe the product information. Afterwards, for example, they can close the door and make a non-sensical automatic payment. In this way, throughout the entire process, the user does not even need to perform any click operations on the mobile phone, which provides a good experience and strong reliability.

[0101] In one or more embodiments of the present specification, a variety of light effects can be used on a locked smart device to provide prompts or guidance for different states of a virtual object in the unlocking interaction process. In particular, a variety of light effects can be used in the short-distance recognition area to provide prompts or guidance. In this way, the short-distance recognition area is more eye-catching, and the user can bring the mobile terminal device close to the short-distance recognition area to easily observe the changes in light effects.

[0102] For example, a corresponding light effect display module can be pre-deployed on the locked smart device for the short-range recognition zone. The light effect display module can be deployed around the short-range recognition zone and dynamically display light effects around the short-range recognition zone. This can more clearly show the boundary outline of the short-range recognition zone to the user, making the short-range recognition zone more prominent, helping the user to operate in one go, and also facilitates the design of different dynamic light effects that help reflect transfer relationships and business conditions.

[0103] For the new type of locked smart device provided in this application, the above functional areas and light effect presentation module have been deployed before leaving the factory, and the software can be updated as needed to support the presentation of different designed visual effects and additional light effects.

[0104] Compared to the completed or waiting state, the approaching and exiting states can use more dynamic lighting effects to prompt or guide the user. For example, the lighting effects can change regularly to indicate the corresponding transfer direction. Conversely, a more stable lighting effect can be used, such as a constant light effect.

[0105] Similarly, for different states, corresponding lighting effects can be designed in advance based on the degree of distinction required or the importance of the state itself to the user, so as to match the visual effects in the corresponding state.

[0106] In addition, in the above process, in addition to the visual effect presentation, corresponding sound effects or voice prompts can also be used to further improve reliability.

[0107] Based on the above interaction method, this application further considers using this vivid interaction method similar to physical transmission to create a delayed unlocking solution for locked smart devices, especially for achieving one-time remote unlocking capabilities to balance security and convenience. Based on this idea, one or more embodiments of this specification provide a flow chart of a solution for delayed unlocking of locked smart devices based on virtual objects, see Figure 5 .

[0108] Figure 5 The process in includes the following steps:

[0109] S502: After the virtual object is moved closer to the short-range identification area until it is out of the locked smart device and correspondingly moved across devices into the display area of ​​the mobile terminal device, a persistent holding request for the virtual object is received from the mobile terminal device.

[0110] In one or more embodiments of this specification, a locked smart device may be, in particular, a public device such as a smart vending machine, which may be a public device that is not frequently used by a single user. In such cases, there is currently a lack of user-level remote unlocking solutions. For example, a user's school-age child may not have a smartphone but may want to use the smart vending machine. However, the user may be at home and not near the smart vending machine, making it difficult for the child to use the smart vending machine. Therefore, this application considers allowing a user to request to hold a moved virtual entity for a longer period of time as a one-time (or a smaller number of times) remote unlocking method.

[0111] After the virtual object is moved across devices to the mobile terminal device, the automatic unlocking may not be triggered first, but the user may be allowed to hold the virtual object more permanently. It should be noted that the persistent holding may not be requested separately, but may be set as the user's default setting.

[0112] S504: Authorize and confirm the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object.

[0113] For example, in some of the previous examples, unlocking can be triggered immediately after the virtual object is moved in. However, as the holding period increases, it can be extended to hours, days, or even longer, maintaining its validity for unlocking. Different users of the same locked smart device can hold different virtual objects, and the locked smart device or its server can maintain the corresponding relationship between users and their virtual objects.

[0114] S506: After the authorization confirmation, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device.

[0115] After persistent holding, the user can unlock the device by triggering the virtual object near the locked smart device, or by triggering the virtual object from a farther distance. After unlocking, the virtual object can be deactivated, temporarily preventing the user from remotely unlocking the device, providing better security.

[0116] Furthermore, based on the above-mentioned convenient virtual entity interaction method, users can also share their virtual objects with other users for other users to use. Specifically, after the persistent holding request is authorized and confirmed, if a designated trigger operation for the virtual object is received from the shared device (for example, another user's mobile terminal device, etc.), the locked smart device is unlocked, wherein the virtual object is shared to the shared device by the mobile terminal device through the visual effect of moving close to it and then moving across the device. Taking a smartphone as an example, in this case, the smartphone of the shared user can play the role of a locked smart device. With the authorization of the shared user, other users can visually suck the virtual object out of the shared user's smartphone and into their own smartphone by bringing their own smartphone close to the shared user's smartphone through NFC communication, thereby obtaining the corresponding one-time unlocking capability.

[0117] The above process is mainly described from the perspective of a locked smart device. Based on the same idea, one or more embodiments of this specification also provide a flowchart of an unlocking interaction processing method applied to a user's mobile terminal device, which is described from the perspective of a mobile terminal device. Figure 6 .

[0118] Figure 6 The process in includes the following steps:

[0119] S602: The user moves the mobile device close to a short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user ID of the user by identifying the mobile terminal device.

[0120] S604: During the approaching process, a virtual object representing the locked smart device is displayed on the mobile terminal device and moves from the direction of the short-distance identification area into the display area of ​​the mobile terminal device to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device.

[0121] S606: After the locked smart device is unlocked according to the acquired user identifier, respond accordingly according to the user's use of the locked smart device.

[0122] It should be noted that the solution can also be implemented by mobile terminal devices themselves. If the mobile terminal device itself does not support the presentation of the aforementioned visual effects (for example, the mobile terminal device has not been upgraded or the mobile terminal device may not even have a user-facing display), then the corresponding visual effects can be presented only on the mobile terminal device. This will still help improve the user experience to a certain extent, while also increasing the flexibility and applicability of the solution.

[0123] This is just a brief description from the perspective of mobile terminal devices. The specific implementation details can be understood by referring to the previous description and will not be repeated here.

[0124] According to the above description, more intuitively, one or more embodiments of this specification provide an application scenario in which: Figure 1 A schematic diagram of the effect of a specific implementation scheme of the method is shown in FIG. Figure 7(a) to Figure 7(d) .

[0125] In this application scenario, a new type of smart vending machine that supports NFC identification and unlocking is provided. The cabinet body is omitted here, and the display and unlocking interactive module of the smart vending machine is mainly shown. As shown in the figure, the display and unlocking interactive module includes a display area on the upper side and an NFC identification area (as the aforementioned short-range identification area) on the lower side. Accompanied by the prompts and guidance of novel interactive visual effects, users can quickly and securely unlock the door by bringing their NFC-enabled smartphone or other portable smart device close to the NFC identification area. The entire process is clear and distinct, with a sense of movement and technology, forming a novel virtual interactive experience that transcends physical media.

[0126] If no user has yet unlocked the phone, the standby interface can be displayed, as shown in the left sub-image of Figure 7(a). The display area can schematically display some prompts, such as "Please unlock your phone" or "Keep your phone close to the screen." Surrounding the NFC recognition area, for example, can be a default standby light effect, or a dynamically changing "NFC" light effect within the NFC recognition area. Additionally, company or product logos can be added where appropriate, though these are omitted in the figure.

[0127] If a user approaches, the user or the user's mobile phone can be detected by TOF, and then it can be awakened from the standby state, as shown in the middle sub-figure of Figure 7(a). The display area shows the above-mentioned virtual object, for example, it can be a spherical virtual object like a bubble or a pinball (for the sake of convenience of description, it will be referred to as a bubble below), which shows prompt information such as "Unlock the phone and stick it to open the door". Of course, the bubble can also contain more detailed information related to unlocking or business, and not all of them must be shown on the bubble. At this time, for example, the following light effect is used as an example: a blue semicircular light effect is presented around the NFC recognition area for circular guidance, and the "NFC" word diffuse light effect is played in a loop. At the same time, a sound effect can also be issued for prompting, for example, "Unlock the phone and stick it to open the door" is played, and if it times out (for example, 3 seconds), the guidance can be automatically jumped out and returned to standby.

[0128] Furthermore, if the user's mobile phone is close enough to the NFC identification area (and may be getting closer), so that NFC can read the induction coil, as shown in the right sub-image of Figure 7(a). The bubble is displayed near the NFC identification area, and the dynamic trajectory lines in the style of "wind direction line" or "assembly line" vividly show the effect of the bubble starting to be sucked towards the NFC identification area. In actual application, the effect is dynamic, and the bubble is dynamically deformed with the suction force, and the dynamic trajectory lines flow towards the NFC identification area to indicate that the transfer direction is from the bubble to the NFC identification area. At this time, for example, the following light effect is used as an example: a blue semicircular light effect is displayed around the NFC identification area for circular guidance, and the "NFC" word diffusion light effect is played in a loop. At the same time, sound effects can also be emitted for prompts, for example, playing "Keep the phone sticking", and the user's phone can be triggered to vibrate. After that, if it times out (for example, 10 seconds), the guidance can be automatically jumped out and returned to standby.

[0129] After reading the induction coil, the smartphone attempts to obtain the user ID. As shown in the left sub-image of Figure 7(b), the bubble has been sucked down, as if the smartphone is in the process of sucking it out. This process will continue until the entire bubble is sucked out (this is a continuous, dynamic process, and only one frame is shown as an example). For example, the following lighting effects are used: a blue semicircle of light loops around the NFC recognition area, and a diffuse light effect of the word "NFC" loops.

[0130] After successfully acquiring the user ID, the bubble will be completely sucked out of the display area of ​​the unlocking interaction module, and the door opening command can be obtained to achieve automatic unlocking, as shown in the middle sub-image of Figure 7(b). The bubble is no longer displayed in the display area, and the user is prompted that the door can be opened. At this time, for example, the following light effect is used: a blue ring is displayed around the NFC recognition area, filled with light effects, and is always on (for example, for 5 seconds). At the same time, a sound effect can also be emitted as a prompt, for example, playing "Pop, please open the door and select the product."

[0131] After opening the door, users can take items from the container. When finished, the door can be closed and locked automatically, or locked after payment is completed, as shown in the right sub-image of Figure 7(b). For example, text is used to display the purchased item information, and automatic payment is being processed. After successful payment, for example, the effect shown in the left sub-image of Figure 7(c) can be displayed.

[0132] If the user unlocks the door but doesn't make a purchase and instead closes the door, the effect shown in the middle sub-image of Figure 7(c) can be displayed. If the unlock fails, the effect shown in the right sub-image of Figure 7(c) can be displayed. Other abnormal situations, such as payment failures, can be handled similarly. For example, the following lighting effect can be used in these abnormal situations: a red ring surrounding the NFC recognition area fills the area with light and remains on to alert the user.

[0133] Looking at Figure 7(d) again, it shows the cooperative display effect on the user's mobile phone side. In the first sub-image on the left side of Figure 7(d), it shows a bubble being sucked out from the NFC identification area and about to be sucked in. At this time, it is still a small bubble and has not yet stabilized. After the user identification is successfully recognized, as shown in the second sub-image on the left side of Figure 7(d), the bubble is already stably in the mobile phone (at this time, the bubble displayed on the smart vending machine has disappeared), and a prompt "Recognition successful" is displayed. Next, it switches to the prompt interface waiting for unlocking, as shown in the third sub-image on the left side of Figure 7(d). After the unlocking is completed, it switches to the interface of the corresponding ISV applet used for this purchase, as shown in the rightmost sub-image of Figure 7(d).

[0134] If NFC unlocking fails, you can also switch to other supported unlocking methods, such as scanning the code to unlock, entering the verification code to unlock, etc.

[0135] The novel smart vending machine and unlocking interaction process described above provide users with a novel virtual interactive experience across physical media, making the entire unlocking and self-service shopping process both efficient and engaging. Furthermore, the exemplary user interface design is user-friendly, with both visual cues and operational processes simple and easy to understand. This allows users to experience the convenience of modern technology while ensuring the security and accuracy of self-service, resulting in a smoother and more enjoyable user experience.

[0136] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figures 8 to 11 The apparatus and device can accordingly execute the above method and related optional solutions.

[0137] Figure 8 This is a schematic diagram of a structure of an unlocking interaction processing device for a locked smart device provided in one or more embodiments of this specification, the device comprising:

[0138] A virtual object display module 802 displays a virtual object for representing a distance status on the locked smart device;

[0139] The approach effect response module 804 starts to display a visual effect of the virtual object approaching the short-distance identification area, so as to indicate that the distance between the virtual object and the mobile terminal device is shortening, if the mobile terminal device used by the user to unlock the locked smart device is detected through the short-distance identification area included in the locked smart device;

[0140] A user identification recognition module 806, which obtains the user identification of the user by identifying the mobile terminal device during the approach process;

[0141] The virtual object removal module 808, after obtaining the user identifier, moves the virtual object closer to the short-range identification area until it moves out of the locked smart device, and accordingly moves the virtual object across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0142] The response cooperation unlocking module 810 obtains an unlocking instruction according to the user identification, and responds to the unlocking instruction to unlock the locked smart device so that the user can use it.

[0143] Optionally, the proximity effect response module 804 detects a change in the distance between the mobile terminal device and the locked smart device;

[0144] The virtual object begins to display a visual effect of approaching the short-distance recognition area accordingly according to the distance change.

[0145] Optionally, it also includes:

[0146] The auxiliary effect response module 812 uses different dynamic light effects in the short-distance recognition area to prompt or guide when the virtual object approaches the short-distance recognition area or when the virtual object has moved across devices into the display area of ​​the mobile terminal device;

[0147] When the virtual object approaches the short-distance recognition area, the mobile terminal device is triggered to vibrate.

[0148] Optionally, the proximity effect response module 804 starts to display a visual effect of the virtual object being attracted to the short-distance recognition area;

[0149] The virtual object removal module 808 causes the virtual object to be sucked out of the locked smart device in the direction of the short-range recognition area.

[0150] Optionally, the lockable smart device includes a vending cabinet;

[0151] The response cooperates with the unlocking module 810 to unlock the vending cabinet and automatically open the cabinet door of the vending cabinet, or to unlock the vending cabinet and prompt the user that the cabinet door of the vending cabinet can be opened.

[0152] Optionally, the smart device with a lock is a device with a door handle, and the virtual object includes a virtual door handle for representing the door handle;

[0153] The approach effect response module 804 starts to display a visual effect of the virtual object approaching the virtual matching object displayed on the mobile terminal device in the short-distance recognition area;

[0154] The virtual cooperation object includes a virtual human hand. After the virtual door handle is moved into the display area of ​​the mobile terminal device, the virtual human hand will hold the virtual door handle to show the visual effect of opening the door.

[0155] Optionally, the lockable smart device includes a vending cabinet;

[0156] The virtual object removal module 808 detects the commodity taken from the locked smart device by the user after unlocking the locked smart device in response to the unlock instruction, and generates and displays a virtual commodity corresponding to the commodity on the locked smart device;

[0157] In conjunction with the user taking the product out of the locked smart device, the virtual product is moved closer to the short-range identification area until it is moved out of the locked smart device, and then moved across the device into the display area of ​​the mobile terminal device, so that payment can be made based on the virtual product in the display area of ​​the mobile terminal device.

[0158] Optionally, the response cooperation unlocking module 810 receives a persistent holding request for the virtual object sent by the mobile terminal device after the virtual object is moved closer to the short-range identification area until it is out of the locked smart device and then correspondingly moved across devices into the display area of ​​the mobile terminal device;

[0159] Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object;

[0160] After the authorization confirmation, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device.

[0161] Optionally, the response cooperation unlocking module 810 unlocks the locked smart device if a designated triggering operation for the virtual object is received from the shared device after the authorization confirmation of the persistent holding request;

[0162] The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

[0163] Optionally, the short-range identification area includes a short-range wireless communication identification area for proximity identification of a mobile terminal device;

[0164] The user identification recognition module 806 reads the user identification of the user from the mobile terminal device using the short-range wireless communication method corresponding to the short-range wireless communication identification area.

[0165] Figure 9 This is a schematic diagram of a structure of an unlocking interaction processing device for a user's mobile terminal device provided in one or more embodiments of this specification, wherein the device includes:

[0166] The user identification matching module 902 is configured to allow the user to move the mobile terminal device close to the short-range identification area of ​​the locked smart device under handheld mobile operation, so that the locked smart device can detect the mobile terminal device and obtain the user identification of the user by identifying the mobile terminal device;

[0167] A virtual object moving module 904 is configured to display a virtual object representing the locked smart device on the mobile terminal device during the approach process, and move the virtual object from the direction of the short-range identification area into the display area of ​​the mobile terminal device to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device.

[0168] The unlocking and use response module 906 responds accordingly according to the user's use of the locked smart device after the locked smart device is unlocked according to the acquired user identification.

[0169] Figure 10 This is a schematic diagram of a structure of an unlocking interaction processing device for a locked smart device provided in one or more embodiments of this specification, the device comprising:

[0170] at least one processor; and,

[0171] a memory communicatively connected to the at least one processor; wherein,

[0172] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0173] Displaying a virtual object for representing the distance status on the locked smart device;

[0174] If the mobile terminal device used by the user to unlock the device is detected through the short-distance identification area included in the locked smart device, a visual effect of the virtual object approaching the short-distance identification area is displayed to indicate that the distance between the virtual object and the mobile terminal device is shortening;

[0175] During the approaching process, obtaining a user identification of the user by identifying the mobile terminal device;

[0176] After obtaining the user identification, the virtual object is moved closer to the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0177] An unlocking instruction is acquired according to the user identification, and in response to the unlocking instruction, the locked smart device is unlocked so as to facilitate use by the user.

[0178] Figure 11 This is a schematic diagram of a structure of an unlocking interaction processing device for a user's mobile terminal device provided in one or more embodiments of this specification, wherein the device includes:

[0179] at least one processor; and,

[0180] a memory communicatively connected to the at least one processor; wherein,

[0181] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0182] The user moves the mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user ID of the user by identifying the mobile terminal device;

[0183] During the approaching process, a virtual object representing the locked smart device displayed on the mobile terminal device moves from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device;

[0184] After the locked smart device is unlocked according to the acquired user identification, the locked smart device responds accordingly according to the user's use of the locked smart device.

[0185] Based on the same idea, one or more embodiments of this specification further provide a non-volatile computer storage medium storing computer-executable instructions, which is applied to a locked smart device, wherein the computer-executable instructions are configured as the corresponding method described above.

[0186] Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium storing computer executable instructions, which is applied to a user's mobile terminal device, and the computer executable instructions are configured as the corresponding method described above.

[0187] After the user identification is unlocked, a corresponding response is given according to the user's use of the locked smart device.

[0188] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. An unlocking interaction processing method, applied to a locked smart device, comprising: Displaying a virtual object for representing the distance status on the locked smart device; If the mobile terminal device used by the user to unlock the device is detected through the short-distance identification area included in the locked smart device, a visual effect of the virtual object approaching the short-distance identification area is displayed to indicate that the distance between the virtual object and the mobile terminal device is shortening; During the approaching process, obtaining a user identification of the user by identifying the mobile terminal device; After obtaining the user identification, the virtual object is moved closer to the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; Obtaining an unlock instruction according to the user identifier, and unlocking the locked smart device in response to the unlock instruction so that the user can use it; The step of starting to display the visual effect of the virtual object approaching the short-distance recognition area specifically includes: Detecting a change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object represents the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to represent that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user. After the virtual object is moved closer to the short-range identification area until it is out of the locked smart device and then correspondingly moved across devices into the display area of ​​the mobile terminal device, the method further includes: receiving a persistent holding request for the virtual object sent by the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; After the authorization confirmation of the persistent holding request, the method further includes: If a designated trigger operation for the virtual object is received from the shared device, unlocking the locked smart device; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

2. The method of claim 1, further comprising: When the virtual object approaches the short-distance recognition area, or when the virtual object has moved across devices into the display area of ​​the mobile terminal device, different dynamic light effects are used in the short-distance recognition area to prompt or guide the virtual object; When the virtual object approaches the short-distance recognition area, the mobile terminal device is triggered to vibrate.

3. The method according to claim 1, wherein the step of starting to display the visual effect of the virtual object approaching the short-distance recognition area comprises: Starting to display a visual effect of the virtual object being attracted to the short-distance recognition area; The step of causing the virtual object to approach the short-distance identification area until it moves out of the locked smart device specifically includes: The virtual object is sucked out of the locked smart device in the direction of the short-distance recognition area.

4. The method of claim 1, wherein the locked smart device comprises a vending cabinet; Unlocking the locked smart device specifically includes: The vending cabinet is unlocked and the cabinet door of the vending cabinet is automatically opened, or the vending cabinet is unlocked and the user is prompted that the cabinet door of the vending cabinet can be opened.

5. The method according to claim 1, wherein the smart device with a lock is a device with a door handle, and the virtual object includes a virtual door handle for representing the door handle; The step of starting to display the visual effect of the virtual object approaching the short-distance recognition area specifically includes: Starting to display a visual effect of the virtual object approaching the virtual matching object displayed on the mobile terminal device in the short-distance recognition area; The virtual cooperation object includes a virtual human hand. After the virtual door handle is moved into the display area of ​​the mobile terminal device, the virtual human hand will hold the virtual door handle to show the visual effect of opening the door.

6. The method according to claim 1 or 5, wherein the locked smart device comprises a vending cabinet; After unlocking the locked smart device in response to the unlock instruction, the method further includes: detecting a commodity taken by the user from the locked smart device, generating and displaying a virtual commodity corresponding to the commodity on the locked smart device; In conjunction with the user taking the product out of the locked smart device, the virtual product is moved closer to the short-range identification area until it is moved out of the locked smart device, and then moved across the device into the display area of ​​the mobile terminal device, so that payment can be made based on the virtual product in the display area of ​​the mobile terminal device.

7. The method according to any one of claims 1 to 5, wherein the short-range identification area comprises a short-range wireless communication identification area for identification by a mobile terminal device brought close to the area; The obtaining of the user identifier of the user specifically includes: The user identification of the user is read from the mobile terminal device using the short-range wireless communication method corresponding to the short-range wireless communication identification area.

8. An unlocking interaction processing method, applied to a user's mobile terminal device, comprising: The user moves the mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user ID of the user by identifying the mobile terminal device; During the approaching process, a virtual object representing the locked smart device displayed on the mobile terminal device moves from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; After the locked smart device is unlocked according to the acquired user identifier, respond accordingly according to the user's use of the locked smart device; If the locked smart device finds the mobile terminal device, it detects the change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object represents the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to represent that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user. After the mobile terminal device is moved into the display area, the locked smart device further executes: receiving a persistent holding request for the virtual object sent by the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; After the authorization confirmation of the persistent holding request, if a designated trigger operation for the virtual object is received from the shared device, the locked smart device is unlocked; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

9. An unlocking interaction processing device, applied to a locked smart device, comprising: A virtual object display module, which displays a virtual object for representing the distance status on the locked smart device; an approach effect response module, which, upon detecting the mobile terminal device used by the user to unlock the device through the short-distance identification area included in the locked smart device, begins to display a visual effect of the virtual object approaching the short-distance identification area to indicate that the distance between the virtual object and the mobile terminal device is shortening; A user identification recognition module, which obtains the user identification of the user by identifying the mobile terminal device during the approach process; a virtual object removal module, after acquiring the user identifier, causing the virtual object to approach the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; In response to the unlocking module, obtaining an unlocking instruction according to the user identifier, and unlocking the locked smart device in response to the unlocking instruction so that the user can use it; The proximity effect response module detects the change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object will show the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to indicate that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user; The response cooperation unlocking module receives a persistent holding request for the virtual object sent by the mobile terminal device after the virtual object is moved closer to the short-range identification area until it is out of the locked smart device and then correspondingly moved across the device into the display area of ​​the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; The response cooperation unlocking module unlocks the locked smart device if a designated triggering operation for the virtual object is received from the shared device after the authorization confirmation of the persistent holding request; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

10. The apparatus of claim 9, further comprising: an auxiliary effect response module, which uses different dynamic light effects in the short-distance recognition area to prompt or guide when the virtual object approaches the short-distance recognition area or when the virtual object has moved across devices into the display area of ​​the mobile terminal device; When the virtual object approaches the short-distance recognition area, the mobile terminal device is triggered to vibrate.

11. The device of claim 9, wherein the proximity effect response module starts to display a visual effect of the virtual object being attracted to the short-distance recognition area; The virtual object removal module causes the virtual object to be sucked out of the locked smart device in the direction of the short-distance recognition area.

12. The apparatus of claim 9, wherein the locked smart device comprises a vending cabinet; The response cooperates with the unlocking module to unlock the vending cabinet and automatically open the door of the vending cabinet, or to unlock the vending cabinet and prompt the user that the door of the vending cabinet can be opened.

13. The apparatus according to claim 9, wherein the smart device with a lock is a device with a door handle, and the virtual object comprises a virtual door handle for representing the door handle; The approach effect response module starts to display a visual effect of the virtual object approaching the virtual matching object displayed on the mobile terminal device in the short-distance recognition area; in, The virtual cooperation object includes a virtual human hand. After the virtual door handle moves into the display area of ​​the mobile terminal device, the virtual human hand will hold the virtual door handle to show the visual effect of opening the door.

14. The apparatus of claim 9 or 13, wherein the locked smart device comprises a vending cabinet; The virtual object removal module, after unlocking the locked smart device in response to the unlock instruction, detects the commodity taken from the locked smart device by the user, generates and displays a virtual commodity corresponding to the commodity on the locked smart device; In conjunction with the user taking the product out of the locked smart device, the virtual product is moved closer to the short-range identification area until it is moved out of the locked smart device, and then moved across the device into the display area of ​​the mobile terminal device, so that payment can be made based on the virtual product in the display area of ​​the mobile terminal device.

15. The device according to any one of claims 9 to 13, wherein the short-range identification area comprises a short-range wireless communication identification area for identification by a mobile terminal device brought close to the area; The user identification recognition module reads the user identification of the user from the mobile terminal device using the short-range wireless communication method corresponding to the short-range wireless communication identification area.

16. An unlocking interaction processing device, applied to a user's mobile terminal device, comprising: The user identification matching module is used to move the user's handheld mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user's user identification by identifying the mobile terminal device; a virtual object moving-in module, which, during the approaching process, displays a virtual object representing the locked smart device on the mobile terminal device, and moves it from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; an unlocking and using response module, which responds accordingly to the user's use of the locked smart device after the locked smart device is unlocked according to the acquired user identifier; If the locked smart device finds the mobile terminal device, it detects the change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object represents the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to represent that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user. After the mobile terminal device is moved into the display area, the locked smart device further executes: receiving a persistent holding request for the virtual object sent by the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; After the authorization confirmation of the persistent holding request, if a designated trigger operation for the virtual object is received from the shared device, the locked smart device is unlocked; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

17. An unlocking interaction processing device, applied to a locked smart device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform: Displaying a virtual object for representing the distance status on the locked smart device; If the mobile terminal device used by the user to unlock the device is detected through the short-distance identification area included in the locked smart device, a visual effect of the virtual object approaching the short-distance identification area is displayed to indicate that the distance between the virtual object and the mobile terminal device is shortening; During the approaching process, obtaining a user identification of the user by identifying the mobile terminal device; After obtaining the user identification, the virtual object is moved closer to the short-range identification area until it moves out of the locked smart device, and correspondingly moves across the device into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; Obtaining an unlock instruction according to the user identifier, and unlocking the locked smart device in response to the unlock instruction so that the user can use it; The step of starting to display the visual effect of the virtual object approaching the short-distance recognition area specifically includes: Detecting a change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object represents the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to represent that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user. After the virtual object is moved closer to the short-range identification area until it is out of the locked smart device and then correspondingly moved across devices into the display area of ​​the mobile terminal device, the method further includes: receiving a persistent holding request for the virtual object sent by the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; After the authorization confirmation of the persistent holding request, the method further includes: If a designated trigger operation for the virtual object is received from the shared device, unlocking the locked smart device; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

18. An unlocking interaction processing device, applied to a user's mobile terminal device, comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, wherein the instructions are executed by the at least one processor to enable the at least one processor to perform: The user moves the mobile device close to the short-range identification area of ​​the locked smart device so that the locked smart device can detect the mobile terminal device and obtain the user ID of the user by identifying the mobile terminal device; During the approaching process, a virtual object representing the locked smart device displayed on the mobile terminal device moves from the direction of the short-range identification area into the display area of ​​the mobile terminal device, so as to indicate that the distance between the virtual object and the mobile terminal device is close enough to trigger the unlocking of the locked smart device; After the locked smart device is unlocked according to the acquired user identifier, respond accordingly according to the user's use of the locked smart device; If the locked smart device finds the mobile terminal device, it detects the change in the distance between the mobile terminal device and the locked smart device; Starting to display a visual effect of the virtual object moving closer to the short-distance recognition area in accordance with the distance change; If the locked smart device detects a mobile terminal device approaching the short-range identification zone for identification, the virtual object represents the distance between the content it represents and the mobile terminal device. The distance status here is a relatively accurate relative distance, or a trend of distance change. It is used to represent that the physical lock to be unlocked is dynamically and actively placed in the user's hand and is about to be unlocked by the user. After the mobile terminal device is moved into the display area, the locked smart device further executes: receiving a persistent holding request for the virtual object sent by the mobile terminal device; Performing authorization confirmation on the persistent holding request so that the mobile terminal device can extend the holding time of the virtual object; After the authorization is confirmed, if a designated trigger operation for the virtual object is received from the mobile terminal device, the locked smart device is unlocked, wherein the designated trigger operation can be effectively performed when the mobile terminal device is away from the locked smart device; After the authorization confirmation of the persistent holding request, if a designated trigger operation for the virtual object is received from the shared device, the locked smart device is unlocked; The virtual object is shared with the shared device by the mobile terminal device through the visual effect of moving closer and then moving across the device.

Images