Interaction method and system, electronic equipment, storage medium and program product

Abstract

The invention provides an interaction method and system, electronic equipment, a storage medium and a program product, and relates to the technical field of computers. The method is applied to a split device, the split device comprises a first sub-device and a second sub-device, and the method comprises the steps that if the first sub-device detects that a target object exists in a set range, first interaction information and first multimedia data containing the target object are displayed in an interface of the second sub-device; obtaining a selection operation triggered for each operation item based on the first multimedia data in an interface of the second sub-device, and determining a selected target operation item; and outputting second interaction information corresponding to the target operation item through the first sub-device. The problem of low interaction efficiency in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and specifically, to an interaction method, system, electronic device, storage medium and program product. Background Art

[0002] With the development of Internet of Things technology, the application of smart devices is becoming more and more extensive. Take smart door locks as an example. They can not only be used with outdoor doorbells or cameras, but also with indoor electronic central control screens through network and other communication methods.

[0003] However, the interaction operations between existing smart devices are not convenient enough, and the interaction operation path is long, resulting in low interaction efficiency. Summary of the invention

[0004] The present application provides an interaction method, device, electronic device and storage medium, which can solve the problem of low interaction efficiency in related technologies. The technical solution is as follows:

[0005] According to one aspect of the present application, an interaction method is applied to a split device, the split device including a first sub-device and a second sub-device, the method including: if the first sub-device detects the presence of a target object within a set range, displaying first interaction information and first multimedia data containing the target object in an interface of the second sub-device; the first interaction information includes at least one operation item that can be selected; obtaining a selection operation triggered for each operation item based on the first multimedia data in the interface of the second sub-device, and determining a selected target operation item; outputting second interaction information corresponding to the target operation item through the first sub-device; the second interaction information indicates interaction with the target object.

[0006] According to one aspect of the present application, an interaction device is applied to a split device, the split device includes a first sub-device and a second sub-device, including: a display module, which is used to display first interaction information and first multimedia data containing the target object in the interface of the second sub-device if the first sub-device detects the presence of a target object within a set range; the first interaction information includes at least one operation item that can be selected; a determination module, which is used to obtain a selection operation for each operation item based on the first multimedia data in the interface of the second sub-device, and determine the selected target operation item; an interaction module, which is used to output second interaction information corresponding to the target operation item by the first sub-device; the second interaction information indicates interaction with the target object.

[0007] In an exemplary embodiment, the target operation item includes a quick reply item; the second interaction information includes reply information corresponding to the quick reply item; the interaction module is also used to: display the reply information corresponding to the quick reply item in the interface of the first sub-device; and / or play voice information corresponding to the reply information through the first sub-device.

[0008] In an exemplary embodiment, the interaction device is also used to display an interaction operation item in the interface of the first sub-device if the target object is detected to exist within the set range; the interaction operation item is used to prompt the target object to perform the target operation to interact with the second sub-device through the first sub-device; if the first sub-device detects the target object's triggering operation on the interaction operation item, the first sub-device initiates a call connection request to the second sub-device to request to establish a call connection with the second sub-device.

[0009] In an exemplary embodiment, the interaction device is further used to push a target event corresponding to the target object to the user terminal to instruct the first multimedia data containing the target object to be displayed in the interface of the user terminal.

[0010] In an exemplary embodiment, the interaction device is also used to receive feedback information returned when a target operation is performed on the target object contained in the first multimedia data in the interface of the user terminal; and control the first sub-device to perform a corresponding target action according to the feedback information to interact with the target object.

[0011] In an exemplary embodiment, the interaction device is also used to, if the feedback information includes the target operation item, cause the first sub-device to output second interaction information corresponding to the target operation item; if the feedback information includes a device control instruction, control the first sub-device to respond to the device control instruction and perform the target action corresponding to the device control instruction; if the feedback information includes a call instruction, control the first sub-device to establish a call connection with the user terminal.

[0012] In an exemplary embodiment, the interaction device is also used to detect the duration of a stay of the target object within the set range if the target object is detected to exist within the set range; if the stay time is less than the set time, control the first sub-device to enter a sleep state until the target object is detected to exist within the set range again, and control the first sub-device to switch from the sleep state to a working state.

[0013] In an exemplary embodiment, the interaction device is also used to display a custom control in the interface of the second sub-device; the custom control is used to add the operation item; in response to an input operation triggered by the custom control, the newly added operation item is obtained; and the first interaction information including the newly added operation item is displayed in the interface of the second sub-device.

[0014] In an exemplary embodiment, the interaction device is also used to display a viewing control for viewing the target object in the interface of the second sub-device; in response to a viewing operation triggered by the viewing control, the first multimedia data containing the target object is displayed in the interface of the second sub-device, and the interaction control is displayed in the interface of the second sub-device; the interaction control is used to trigger the display of the first interaction information in the interface of the second sub-device.

[0015] According to one aspect of the present application, an interaction method is applied to a user terminal, wherein the user terminal interacts with a split device, wherein the split device includes a first sub-device and a second sub-device, and the method includes: if a target event corresponding to a target object detected by the first sub-device is received, first interaction information and first multimedia data containing the target object are displayed in an interface of the user terminal, so as to synchronously respond to the display of the first interaction information and the first multimedia data in an interface of the second sub-device; the target event is sent by the first sub-device upon detecting the existence of a target object within a set range; the first interaction information includes at least one selectable operation item; in response to an operation triggered for the target operation item in the interface of the user terminal, second interaction information corresponding to the target operation item is output to the first sub-device; the second interaction information indicates interaction with the target object.

[0016] According to one aspect of the present application, an interactive device is applied to a user terminal, which interacts with a split device, and the split device includes a first sub-device and a second sub-device. The device includes: an event receiving module, which is used to receive a target event corresponding to a target object; the target event is sent by the first sub-device when it detects the existence of a target object within a set range; a data display module, which is used to display first multimedia data containing the target object in an interface of the user terminal based on the received target event, so as to synchronously respond to the display of the first interactive information and the first multimedia data in an interface of the second sub-device, and / or the first sub-device outputs second interactive information corresponding to a target operation item; the target operation item is determined based on at least one selectable operation item in the first interactive information and the first multimedia data; the second interactive information indicates interaction with the target object.

[0017] In an exemplary embodiment, the interactive device is also used to display a custom control in the interface of the user terminal; the custom control is used to add the operation item; in response to an input operation triggered by the custom control, the newly added operation item is obtained; and the newly added operation item is sent to the split device.

[0018] According to one aspect of the present application, an interactive system includes a split device and a user terminal, wherein the split device includes a first sub-device and a second sub-device; wherein:

[0019] The first sub-device is used to detect whether there is a target object within a set range; the second sub-device is used to display first interaction information and first multimedia data containing the target object in the interface of the second sub-device if the first sub-device detects that the target object exists within the set range; the first interaction information includes at least one operation item that can be selected; the second sub-device is also used to obtain a selection operation for each operation item based on the first multimedia data in the interface of the second sub-device, and determine the selected target operation item; the first sub-device is also used to output second interaction information corresponding to the target operation item; the second interaction information indicates interaction with the target object; the user terminal is used to receive a target event corresponding to the target object, and display the first multimedia data containing the target object in the interface of the user terminal based on the received target event; the target event is sent by the first sub-device when it detects that there is a target object within a set range.

[0020] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the interaction method as described above is implemented.

[0021] According to one aspect of the present application, a storage medium stores a computer program thereon, and when the computer program is executed by one or more processors, the interaction method described above is implemented.

[0022] According to one aspect of the present application, a computer program product includes a computer program, and when the computer program is executed by one or more processors, the interaction method described above is implemented.

[0023] The beneficial effects of the technical solution provided by this application are:

[0024] In the above technical solution, the user can view the first multimedia data monitored by the first sub-device through the second sub-device, and select different operation items according to the first multimedia data, thereby reducing the communication burden between the user and the visitor and enhancing the convenience of operation; further, since the second interaction information is obtained based on the target operation item selected by the user, personalized interaction information can be provided to the visitor according to the actual situation, thereby improving the interaction efficiency between the visitor and the user, thereby effectively solving the problem of low interaction efficiency existing in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0026] Figure 1 It is a schematic diagram of the implementation environment involved in this application;

[0027] Figure 2 is a structural block diagram of an interactive system involved in the corresponding embodiment;

[0028] Figure 3 is a flow chart of an interaction method according to an exemplary embodiment;

[0029] Figure 4 is a schematic diagram of a specific implementation of the reply information involved in the corresponding embodiment;

[0030] Figure 5 yes Figure 1 A flowchart of step 310 in one embodiment of the corresponding embodiment;

[0031] Figure 6 yes Figure 1 The steps after step 310 in the corresponding embodiment are a flowchart of an embodiment;

[0032] Figure 6a yes Figure 6 A flowchart of step 430 in one embodiment corresponding to the embodiment;

[0033] Figure 7 is a specific implementation diagram of a page where input operations are located in a corresponding embodiment;

[0034] Figure 8 is a specific implementation diagram of a page where a viewing operation is located involved in a corresponding embodiment;

[0035] Fig. 9is a flow chart of another interaction method according to an exemplary embodiment;

[0036] Figure 9a yes Fig. 9 A schematic diagram of a specific implementation of a page where input operations are located according to a corresponding embodiment;

[0037] Fig.10 It is a schematic diagram of a specific implementation of an interaction method in an application scenario;

[0038] Fig.11 is a structural block diagram of an interactive device according to an exemplary embodiment;

[0039] Fig.12 is a structural block diagram of another interactive device according to an exemplary embodiment;

[0040] Fig.13 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0042] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0043] As mentioned above, the interaction operations between existing smart devices are not convenient enough, and the interaction operation path is long, resulting in low interaction efficiency.

[0044] Specifically, taking smart door locks as an example, most of them currently only use voice interaction. When the voice is ambiguous or the visitor does not hear clearly, the communication efficiency between the two will be greatly reduced, and even communication is impossible for the hearing-impaired.

[0045] Furthermore, most smart door locks need to be connected to the Internet in order to utilize user terminals to control the smart door locks for interaction. When the smart door locks are offline or the network status is poor, then the interaction between the smart door locks and visitors cannot be controlled.

[0046] In addition, the current design of interactive operations has the problem of long paths, which will greatly reduce communication efficiency.

[0047] Taking the courier as a visitor as an example, to achieve interaction between the courier and the user, first, the courier rings the doorbell of the smart door lock, and the user's mobile phone will receive a notification; the user opens the application on the mobile phone to check the identity of the courier (it may take a few seconds to load); confirm whether the courier is the expected delivery person through the video surveillance of the smart door lock; then, the user chooses to communicate with the courier through voice call or message in the application to determine the specific delivery location of the package (such as "please leave the package at the door"). The courier replies to the user by voice or text, and may need to repeat the confirmation of the details; if the user expects to put the courier in the house, he also needs to find the unlock function in the application, click to unlock the door lock, and wait for the smart door lock to connect to the server (there may be a delay). Finally, the courier can place the package and leave only after confirming that the door lock is unlocked. The user needs to confirm that the courier places the package, leaves, and closes the door before completing this interaction.

[0048] It can be seen that the above interaction process has the following problems. First, users need to go through multiple steps to complete a simple package delivery, and the whole process may take several minutes. Second, due to the use of interactive voice or text communication, misunderstandings or repeated confirmations may occur, resulting in inefficient communication. Third, if the network is unstable, users may not be able to unlock the door lock in time, affecting the courier's work efficiency.

[0049] From the above, it can be seen that the related technology still has the defect of low interaction efficiency.

[0050] To this end, the interaction method provided in the present application can effectively improve the accuracy of the interaction. Accordingly, the interaction method is applicable to an interaction device, which can be deployed on an electronic device. The electronic device can also be an electronic device with a central control function, for example, the electronic device includes a smart door lock, etc.; the electronic device can also refer to a portable and mobile electronic device, for example, the electronic device includes a smart phone, a tablet computer, etc.

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0052] Figure 1Schematic diagram of an implementation environment involved in an interactive method. The implementation environment at least includes a user terminal 110, a smart device 130, a server 170, and a network device. Figure 1 In the embodiment, the network devices include a gateway 150 and a router 190, which is not specifically limited herein.

[0053] Among them, the user terminal 110, which can also be considered as a user end or terminal, can deploy (also understood as install) the client associated with the smart device 130. This user terminal 110 can be an electronic device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart control panel, other devices with display and control functions, etc., which are not limited here.

[0054] Among them, the client is associated with the smart device 130, which essentially means that the user registers an account in the client and configures the smart device 130 in the client. For example, the configuration includes adding a device identifier to the smart device 130, so that when the client is run in the user terminal 110, it can provide the user with functions such as device display and device control of the smart device 130. This client can be in the form of an application or a web page. Accordingly, the interface for device display on the client can be in the form of a program window or a web page, which is not limited here.

[0055] The smart device 130 is deployed in the gateway 150, and communicates with the gateway 150 through its own configured communication module, and is then controlled by the gateway 150. It should be understood that the smart device 130 generally refers to one of the multiple smart devices 130, and the embodiment of the present application only takes the smart device 130 as an example, that is, the embodiment of the present application does not limit the number and device type of smart devices deployed in the gateway 150. In an application scenario, the smart device 130 accesses the gateway 150 through a local area network, and is thus deployed in the gateway 150. The process of the smart device 130 accessing the gateway 150 through the local area network includes: the gateway 150 first establishes a local area network, and the smart device 130 joins the local area network established by the gateway 150 by connecting to the gateway 150. This local area network includes, but is not limited to: ZIGBEE or Bluetooth. Among them, the smart device 130 can be a smart printer, a smart fax machine, a smart camera, a smart air conditioner, a smart door lock, a smart light, or a human body sensor equipped with a communication module, a door and window sensor, a temperature and humidity sensor, a water immersion sensor, a natural gas alarm, a smoke alarm, a wall switch, a wall socket, a wireless switch, a wireless wall sticker switch, a magic cube controller, a curtain motor, a millimeter wave radar, etc.

[0056] The interaction between the user terminal 110 and the smart device 130 can be achieved through a local area network or a wide area network. In one application scenario, the user terminal 110 establishes a communication connection between the router 190 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to WIFI, so that the user terminal 110 and the gateway 150 are deployed in the same local area network, so that the user terminal 110 can interact with the smart device 130 through the local area network path. In another application scenario, the user terminal 110 establishes a communication connection between the server 170 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to 2G, 3G, 4G, 5G, WIFI, etc., so that the user terminal 110 and the gateway 150 are deployed in the same wide area network, so that the user terminal 110 can interact with the smart device 130 through the wide area network path.

[0057] The server end 170 may also be considered as a cloud, a cloud platform, a platform end, a service end, etc. The server end 170 may be a server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers, so as to better provide background services to a large number of user terminals 110. For example, the background services include interactive services.

[0058] In one application scenario, an interactive system includes a split device and a user terminal, the split device can be the smart device 130 in the above implementation environment, which includes a first sub-device and a second sub-device; wherein the first sub-device is used to detect whether there is a target object within a set range; the second sub-device is used to display first interactive information and first multimedia data containing the target object in the interface of the second sub-device if the first sub-device detects that there is a target object within the set range; the first interactive information includes at least one optional operation item; the second sub-device is also used to obtain a selection operation for each operation item based on the first multimedia data in the interface of the second sub-device, and determine the selected target operation item; the first sub-device is also used to output second interactive information corresponding to the target operation item; the second interactive information indicates interaction with the target object; wherein the first sub-device and the second sub-device can establish a local area network connection by wireless or wired means to realize local communication between the two, without relying on an external Internet network, for example, the first sub-device and the second sub-device establish a connection by Bluetooth. No specific limitation is made here.

[0059] The user terminal, which may be the user terminal 110 of the above-mentioned implementation environment, is used to receive a target event corresponding to a target object, and display first multimedia data containing the target object in an interface of the user terminal based on the received target event; the target event is sent by the first sub-device when it detects the existence of a target object within a set range.

[0060] For example, Figure 2 A structural block diagram of an interactive system is shown, Figure 2 As shown, the interactive system includes a smart door screen outside (first sub-device), a smart door screen inside (second sub-device) and a mobile terminal (user terminal); when a visitor appears outside the door, the camera on the smart door screen outside will detect the visitor and transmit the video outside the door to the smart door screen inside through the local area network for the owner to view. Then, the owner inside the room can interact with the visitor with the help of the operation items provided by the smart door screen inside; of course, if the user is not at home, the smart door screen outside can also transmit the video to the mobile terminal carried by the owner through the Internet, so that the owner can use the mobile terminal to view it.

[0061] See also Figure 3 The embodiment of the present application provides an interaction method, which is applicable to an electronic device, for example, the electronic device may be Figure 1 A smart device 130 is shown in an implementation environment.

[0062] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0063] like Figure 3 As shown, the method may include the following steps:

[0064] Step 310: If the first sub-device detects that there is a target object within a set range, the first interaction information and the first multimedia data including the target object are displayed in the interface of the second sub-device.

[0065] First of all, it should be noted that the interaction method provided in the embodiment of the present application is applied to a split device, which can be a device composed of two or more sub-devices, and different sub-devices can realize different functions; specifically, the split device includes a first sub-device and a second sub-device, and the first sub-device and the second sub-device can be installed in different positions to realize interaction between users and visitors.

[0066] Taking the split device as a smart door lock as an example, the first sub-device can be a smart display screen, which is equipped with a microphone and a camera and is installed outdoors, and the second sub-device can also be a smart display screen, which is equipped with a microphone and is installed indoors; when a visitor visits the user's home, the smart display screen installed outdoors can be used to interact with the user, for example, by clicking the doorbell button on the smart display screen, and the user can use the smart display screen installed indoors to interact with the visitor, for example, by clicking the unlock button on the smart display screen. It should be noted that a LAN connection can be established between the first sub-device and the second sub-device in a wireless or wired manner to achieve communication between the two, for example, the first sub-device and the second sub-device are connected via Bluetooth or a USB interface. No specific limitation is made here. It can be understood that the flexible connection method between the first sub-device and the second sub-device not only facilitates the installation and use of the split device, but also ensures the stability of information transmission and ensures that the interaction between the user and the visitor can be carried out in a timely manner.

[0067] Among them, the first interaction information includes at least one operation item to be selected, the first interaction message is a message used to guide the interaction, and the first interaction message can be a reply message, for example, "place the express at the door", "hang the takeaway on the door", etc., which is not limited here.

[0068] Regarding the operation items, it may be an option that provides specific interactive selections for implementing specific interactive actions. For example, quick reply items may be provided. Different reply information contents may correspond to different quick reply items. If the user wishes to send a quick message to the first sub-device, the corresponding operation item may be triggered, thereby sending the quick message to the first sub-device for the target object to view. Specifically, if the user clicks on the quick reply item corresponding to the reply message "leave the express at the door", the first sub-device may display "leave the express at the door" in the form of voice and / or text.

[0069] The target object may be an entity that needs to be monitored in a specific application scenario, such as a visitor, an animal, a stranger, an owner, etc., which is not limited here.

[0070] Furthermore, the first sub-device may be a device that provides an operation platform for target objects whose identities are visitors, strangers, etc., and the second sub-device may be a device that provides an operation platform for target objects whose identities are owners, family members, etc. In combination with the above-mentioned example of smart door locks, the guest can operate the smart display screen installed outdoors, and the host can operate the smart display screen installed indoors, thereby realizing the interaction between the guest and the host.

[0071] Furthermore, the first multimedia data may be data corresponding to information content in various forms, and may be used to display the target object monitored by the first sub-device. For example, the first multimedia data may include at least one of text, images, video, and audio, which is not limited here.

[0072] For example, the first sub-device monitors the target object. If the first sub-device is integrated with a camera, a video including the target object can be generated to obtain first multimedia data, and the first multimedia data is sent to the second sub-device. The user can view the video including the target object through the second sub-device.

[0073] Step 330: Acquire a selection operation triggered on each operation item based on the first multimedia data in the interface of the second sub-device, and determine a selected target operation item.

[0074] First of all, it is explained that the user can view the information content included in the first multimedia data through the interface of the second sub-device, for example, view the content such as the video including the target object. It can be understood that after viewing the information content, the user can perform the desired interactive action on the information content.

[0075] The target operation item is the selected operation item among the operation items. As mentioned above, the operation item can be used to implement specific interactive actions, that is, different operation items can correspond to different interactive actions. The selection operation can be used to select the interactive action that the user expects to implement among the operation items. If the user expects to implement a certain interactive action after viewing the information content included in the first multimedia data, the user can trigger the selection operation on the corresponding operation item to implement the interactive action corresponding to the operation item. Among them, the operation item for which the selection operation is triggered is regarded as the target operation item.

[0076] Step 350: output second interaction information corresponding to the target operation item through the first sub-device.

[0077] The second interaction information is information generated and output according to the selected target operation item, and can be used to display interactive information content to the target object, that is, the second interaction information can indicate interaction with the target object.

[0078] It should be noted that the first sub-device is a device for interacting with the target object. Therefore, the second interaction information can be output on the first sub-device to display the interactive information content to the target object, and the target object can view the information content, thereby achieving interaction with the target object.

[0079] For example, when a user interacts with a visitor through the second sub-device indoors, corresponding second interaction information will be generated according to the target operation item triggered by the user, and the second interaction information will be output on the first sub-device, so that the visitor can view the text or voice feedback corresponding to the second interaction information through the first sub-device, for example, "place the express at the door."

[0080] It should be noted that the second sub-device can also trigger a selection operation for the operation item corresponding to the Do Not Disturb mode, thereby selecting to turn on the Do Not Disturb mode for the interactive system. If a visitor rings the doorbell of the first sub-device, the first sub-device will automatically display the text "Do Not Disturb". In the Do Not Disturb mode, the interactive system will not play voice, and the second sub-device will not automatically call up the real-time screen, but only retain the event record (subsequent users can view the event in the log).

[0081] Through the above process, the user can view the first multimedia data monitored by the first sub-device through the second sub-device, and select different operation items according to the first multimedia data, thereby reducing the communication burden between the user and the visitor and enhancing the convenience of operation; further, since the second interaction information is obtained based on the target operation item selected by the user, personalized interaction information can be provided to the visitor according to the actual situation, thereby improving the interaction efficiency between the visitor and the user.

[0082] In an exemplary embodiment, after the first sub-device detects the target object, it can also identify the target object to confirm the identity type of the target object, which can include the owner, visitor, relative, friend, stranger, courier, waiter, etc. Specifically, the face of the target object can be recognized by calling the module with camera function of the first sub-device; the target object's clothing can also be identified to infer the target object's identity type, for example, the uniforms worn by couriers and takeaways can be identified to infer their specific identity types, which are not specifically limited here.

[0083] It can be understood that for target objects of different identity types, the first sub-device can perform different operations to provide adaptive services for target objects of different identity types. Specifically, for the owner type, the first sub-device can perform an unlocking operation to control the smart door lock to unlock so that the owner can go home; for the visitor type target object, a video including the visitor can be displayed in the second sub-device so that the owner can operate on the second sub-device, thereby displaying the second interactive information for the visitor on the first sub-device, and realizing the interaction between the owner and the guest.

[0084] Of course, the user can set different permissions for target objects of different identity types in advance, for example, the target objects of the owner type, the relative type, and the friend type can directly enter the house, etc. Furthermore, messages can be pre-set for target objects of identity types such as couriers and deliverymen, for example, placing a courier / takeout at the doorstep; then, when the first sub-device detects that the courier or deliveryman has stopped, the above message can be displayed on the first sub-device for them to view, without limitation here.

[0085] Through the above process, the interaction process can be further reduced and the interaction efficiency between visitors and users can be improved.

[0086] In an exemplary embodiment, step 370 may also include the following steps: displaying the reply information corresponding to the quick reply item in the interface of the first sub-device; and / or, playing the voice information corresponding to the reply information through the first sub-device.

[0087] The target operation item includes a quick reply item, and the second interaction information includes reply information corresponding to the quick reply item.

[0088] It can be understood that the target operation item can have multiple quick reply items, each quick reply item has corresponding reply information, the information content of the reply information can be pre-set by the manufacturer, and can also be pre-set by the user according to needs. Each reply information can correspond to a usage scenario.

[0089] Then, the user can trigger a selection operation on a quick reply item, thereby generating second interaction information including corresponding reply information.

[0090] It should be noted that, in order to enable the target object (for example, visitor) to clearly receive the second interaction information, the second interaction information may be output in text form and / or voice form.

[0091] For example, the user may select the quick reply item of "leave the express at the door". When the quick reply item is selected, the interface of the first sub-device will display the reply information corresponding to this quick reply item, display the text prompt "Please leave the express at the door" on the device screen, and / or play "Please leave the express at the door" in voice form.

[0092] Figure 4 A specific implementation diagram of a reply message is shown in FIG. Figure 4As shown, the page 401 of the second sub-device displays a video 402 including a courier and the first interactive information 403. Click the quick reply item "Please put the courier at the door" 404 displayed in the first interactive information, and the text "Please put the courier at the door" is displayed through the page 405 of the first sub-device, and the voice of "Please put the courier at the door" is played through the speaker of the first sub-device. Further, during the voice playback process, the page 405 of the first sub-device displays text and sound wave graphics. After a single voice is played once, the sound wave graphics on the page 405 of the first sub-device will change to a clickable "re-listen" button 406. Visitors can click "re-listen" to play the voice again. If the user clicks the quick reply item 404 multiple times on the second sub-device, it can be regarded as sending multiple voices. The first sub-device will automatically play the next voice after the previous voice is played, and the text will change synchronously. If the first sub-device has played all voices and no subsequent operations are detected, then the first sub-device can turn off the screen.

[0093] It should be noted that the voice message can be recorded by the user himself or generated by the system based on the text message, which is not limited here.

[0094] Under the effect of the above-mentioned embodiment, the user can quickly select the appropriate quick reply item, and the system automatically generates relevant second interaction information, reducing the tediousness of manual input by the user and improving the intelligent experience; in addition, the interactive system can convey the second interaction information simultaneously through visual and auditory means, ensuring that visitors can receive the corresponding information content whether by reading screen prompts or listening to voice prompts, thereby making communication between users and visitors smoother and ensuring the accuracy and timeliness of information transmission.

[0095] See also Figure 5 In an exemplary embodiment, step 310 may further include the following steps:

[0096] Step 311: If it is detected that there is a target object within the set range, an interactive operation item is displayed in the interface of the first sub-device.

[0097] The set range is used to determine whether the target object enters the interactive area. The setting of the set range enables the interactive system to automatically screen out the target objects that need to be interacted with, ensuring that no invalid operations are performed on all entities that enter the monitoring range of the first sub-device. For example, the interaction process will only be triggered when the visitor approaches a certain distance.

[0098] The interactive operation item is used to interact with the second sub-device by operating the first sub-device. It can be understood that the interactive operation item can be regarded as an operation item provided for the target object to interact with the second sub-device, that is, an operation item for realizing the interaction between the target object and the user. For example, the interactive operation item can be a doorbell operation item, a call operation item, etc., which is not limited here.

[0099] For example, if a visitor comes to a user's door, the second sub-device (smart display screen outside the door) detects that the visitor has entered the set range. At this time, the interactive system displays different interactive operation items on the second sub-device for the target object to select and trigger.

[0100] Step 313: If the first sub-device detects a triggering operation of the target object on the interactive operation item, the first sub-device initiates a call connection request to the second sub-device to request to establish a call connection with the second sub-device.

[0101] As mentioned above, the interactive operation item can be a call operation item. Then, if the target object expects to establish a call connection with the second sub-device, the call operation item displayed on the first sub-device can be triggered to generate a call connection request, and sent to the second sub-device to request to establish a call connection with the second sub-device.

[0102] The call connection request is a request initiated by one party to establish a call connection with at least another party, for example, a first sub-device initiates a communication connection request to a second sub-device to request to establish a call connection with the second sub-device.

[0103] For example, the first sub-device (a smart display screen installed outdoors) will display multiple interactive operation items, such as doorbell operation items, call operation items, message operation items, etc., which are not limited here; if the target object wants to talk to the user, the target object can click on the call operation item. After the target object triggers the call operation item, the first sub-device will generate a call connection request, which will be sent by the first sub-device to the second sub-device (a smart display screen installed indoors) to request to establish a call connection.

[0104] Under the effect of the above-mentioned embodiment, by setting a range, it can be ensured that only the target object entering this set range will trigger the interaction process, thereby avoiding triggering unnecessary interaction processes for entities (such as pedestrians or animals) entering the monitoring range of the first sub-device. In addition, the target object can communicate with the user in real time by selecting an interaction operation item, such as a call operation item, thereby simplifying the communication process between the user and the visitor. Real-time calls enable users to communicate with target objects (such as visitors) instantly and solve problems quickly.

[0105] In an exemplary embodiment, after step 310, the method may further include the following steps: pushing a target event corresponding to the target object to the user terminal to instruct to display the first multimedia data containing the target object in the interface of the user terminal.

[0106] Among them, the target event refers to an event that meets specific conditions related to the target object, and is used to push information related to the target object to the user terminal. Specifically, when the target object enters, leaves, or performs certain specific actions, the interactive system generates a target event and pushes it to the user terminal.

[0107] As mentioned above, the first sub-device and the second sub-device are part of a split device, that is, the second sub-device is not easy to carry. Therefore, if the user is not near the second sub-device, when the target object appears within the monitoring range of the first sub-device, the user cannot view information about the target object through the second sub-device in a timely manner.

[0108] Based on this, in order to avoid the above situation, when it is detected that the target object appears within the set range of the first sub-device, a target event corresponding to the target object can be pushed to the user terminal to instruct the first multimedia data containing the target object to be displayed in the interface of the user terminal for the user to view.

[0109] For example, if the first sub-device detects that a visitor appears outdoors, it obtains a video containing the visitor (first multimedia data), uploads the first multimedia data to the cloud, generates a corresponding target event "visitor visiting", and pushes the target event to the user terminal. The user views the target event through the user terminal carried. If the user wants to view the details, he can click on the push information of the target event, and the user terminal will obtain the first multimedia data from the cloud, thereby displaying the video containing the visitor for the user to view.

[0110] In one possible implementation, an event preservation period can be set for the target event. If the target event is pushed to the user terminal after the event preservation period has passed, then when the user clicks on the push information corresponding to the target event, the user terminal will not obtain the first multimedia data from the cloud, but will enter the log page and display the target event that occurred in history on the log page; on the contrary, if the target event has not passed the event preservation period, the user clicks on the push information corresponding to the target event, the user terminal will obtain the first multimedia data from the cloud, and display the information content corresponding to the first multimedia data on the user terminal. It can be understood that the event preservation period can reduce unnecessary resource consumption, and can also effectively distinguish between real-time events and historical events, improve users' attention to events and processing efficiency, optimize resource usage, and enhance user experience.

[0111] Through the above process, based on the push target time at the user terminal, even if the user is not near the second sub-device, the user terminal can receive the pushed target event and the first multimedia data in time, which can realize remote monitoring, timely interaction and improved security; further, through the user terminal combined with the second sub-device, the remote flexibility of the user terminal and the localization advantage of the second sub-device complement each other, ensuring that the user can fully control the safety and interaction process at home and respond anytime and anywhere. This significantly improves the user experience and makes the interactive system more intelligent and humane.

[0112] See also Figure 6 In an exemplary embodiment, after pushing the target event corresponding to the target object to the user terminal, the method may further include the following steps:

[0113] Step 410: receiving feedback information returned when a target operation is performed on a target object included in the first multimedia data in an interface of a user terminal.

[0114] Among them, the feedback information is information generated according to triggering the target operation on the user terminal, and can be used to display interactive information content to the target object, that is, the feedback information can also indicate the interaction with the target object.

[0115] The target operation is a specific action performed by the user on the user terminal, which can be used to trigger a specific interaction process. Specifically, the target operation can guide a specific interaction action to achieve interaction with the target object. For example, the target operation may include "talk to a visitor", "send a message" or "open the door", etc., which are not specifically limited here.

[0116] Then, the user can view the information content included in the first multimedia data through the interface of the user terminal, for example, view the video including the target object. It can be understood that after viewing the information content, the user can perform the desired target operation on the information content, thereby generating a feedback message and sending it to the first sub-device.

[0117] Step 430: Control the first sub-device to perform a corresponding target action according to the feedback information to interact with the target object.

[0118] The target action may be an action performed according to the guidance of the feedback information, and may be used to guide the target object to interact.

[0119] As mentioned above, the feedback information may also indicate the interaction with the target object, that is, the feedback information may not only include the operation results of the previous interaction process, for example, the target visitor requests to establish a communication connection with the user terminal through the first sub-device, and the user performs the target operation of agreeing to the call through the user terminal, then, the feedback information may include an instruction on the first sub-device to display "Call connected"; it may also include further interaction indications between the user and the target object, for example, the feedback information may include an instruction on the first sub-device to display "The door is open, please enter".

[0120] Through the above process, feedback information can make the interaction between users and target objects more vivid and efficient. The target object can not only see the operation results of the previous interaction process through feedback information, but also perform the next action according to the guidance of the feedback information. The generation and transmission of feedback information makes the interaction process more coherent and natural.

[0121] See also Figure 6a In an exemplary embodiment, step 430 may further include the following steps:

[0122] Step 431: If the feedback information includes a target operation item, the first sub-device outputs second interaction information corresponding to the target operation item.

[0123] As mentioned above, an operation item may be an option that provides specific interactive selections and is used to implement a specific interactive action. Multiple operation items may be displayed not only on the second sub-device, but also on the user terminal, so that the user can select an operation item on the user terminal to determine the target operation item. The target operation item is used to indicate the interactive action that the user expects to implement.

[0124] Furthermore, the second interaction information is information generated and output according to the target operation item selected in the user terminal, and can be used to display the interactive information content to the target object, that is, the second interaction information can indicate the interaction with the target object. Specifically, the first sub-device is a device for interacting with the target object. If the feedback information includes the target operation item, then the second interaction information can be output on the first sub-device to display the interactive information content to the target object, and the target object can view the information content, thereby realizing the interaction with the target object.

[0125] Step 433: If the feedback information includes a device control instruction, control the first sub-device to respond to the device control instruction and perform a target action corresponding to the device control instruction.

[0126] The device control instruction is a specific instruction generated according to the user triggering the target operation on the user terminal, and is used to instruct the first sub-device to perform an action that conforms to the feedback information.

[0127] Specifically, after the first sub-device receives a feedback message including a device control instruction sent by the user terminal, the feedback message can be parsed to identify the content of the device control instruction, which may specifically include a target action. The first sub-device can then execute a corresponding target action in response to the identified device control instruction.

[0128] For example, if the user selects the "open door" operation item on the user terminal, a corresponding unlocking instruction is generated and sent to the first sub-device. After receiving the unlocking instruction, the first sub-device controls the door lock to perform the unlocking action.

[0129] Step 435: If the feedback information includes a call instruction, control the first sub-device to establish a call connection with the user terminal, so as to conduct a call with the target object through the established call connection.

[0130] Among them, the call instruction is an instruction for instructing to establish a call connection with the first sub-device. If the user expects to have a call with the target object, the call instruction can be generated through the target operation, and the call instruction can be sent to the first sub-device to instruct to establish a call connection with the first sub-device, so that the user can have a call with the target object through the established call connection.

[0131] In combination with the above embodiments, different interaction processes between the user terminal and the first sub-device can be implemented through the target operation items, device control instructions, and call instructions included in the feedback information, thereby realizing the interaction between the user and the target object.

[0132] In an exemplary embodiment, before step 310, the method may further include the following steps: if the target object is detected to exist within the set range, then detecting the residence time of the target object within the set range; if the residence time is less than the set time, controlling the first sub-device to enter a sleep state until the target object is detected to exist within the set range again, and controlling the first sub-device to switch from the sleep state to the working state.

[0133] Among them, the set time is used to determine whether the target object enters the time range of interaction. The setting of the set time can enable the interactive system to automatically screen out the target objects that need to interact, and ensure that no invalid operations are performed on all entities that temporarily stay within the set range of the first sub-device. For example, the interaction process will only be triggered when the visitor approaches the set range and the stay time exceeds the set range.

[0134] Regarding the first sub-device, the sleep state can be a low power consumption mode of the first sub-device. The first sub-device in the sleep state can maintain basic monitoring functions of the target object so that it is ready to be reactivated at any time; the working state is the active state of the first sub-device when it detects the target object and meets the stay time requirement. In the working state, the first sub-device operates with full functions and can perform various interaction and monitoring tasks.

[0135] It should be noted that if the target object only stays temporarily, it means that the target object does not need to interact with the user, and the first sub-device is used to realize interaction with the second sub-device and / or interaction with the user terminal. That is to say, since the target object does not need to continue to interact, the first sub-device does not need to be in a working state, thereby avoiding energy waste.

[0136] Through the above process, if the target object's residence time is less than the set time, the first sub-device will enter a dormant state to save energy and resources and avoid ineffective workload. When the target object enters the set range again and meets the residence time requirement, the first sub-device will automatically switch from the dormant state to the working state to respond and interact in time. This mechanism can optimize the energy efficiency of the first sub-device, extend the service life of the device, and maintain sensitivity to important events.

[0137] In an exemplary embodiment, after step 310, the method may further include the following steps: displaying a custom control in the interface of the second sub-device, and obtaining a newly added operation item in response to an input operation triggered by the custom control; and displaying first interaction information including the newly added operation item in the interface of the second sub-device.

[0138] Among them, the custom control is used to add new operation items, and the input operation is used to customize the newly added operation items.

[0139] As mentioned above, the operation items provide specific interactive selection options for implementing specific interactive actions. If the user expects to add a custom operation item to implement a new interactive action, the user can input the corresponding interactive action content through the input operation triggered by the custom control to obtain the newly added operation item.

[0140] After the operation item is added, the first interaction information including the newly added operation item can be displayed on the interface of the second sub-device.

[0141] For example, a new quick reply item "leave the express at the door" is added. After completing the addition of the quick reply item, the first interactive information including the newly added quick reply item "leave the express at the door" can be viewed in the interface of the second sub-device.

[0142] Figure 7 A schematic diagram of a specific implementation of an input operation is shown in FIG. Figure 7 As shown, there is a custom control 702 for the quick reply item on page 701 of the second sub-device. Click on the custom control 702 to trigger the input operation and enter the page 703 of the newly added quick reply item. Enter the reply message "OK, I got it" through the keyboard and confirm it. Then the new quick reply item is completed and the newly added quick reply item "OK, I got it" is displayed on page 704 of the second sub-device.

[0143] Through the above process, users can customize operation items according to their needs, improve the flexibility and personalization of interaction, and by obtaining new operation items in real time, users can quickly access and select new operation items. The interactive system can adapt to changes in user needs, thereby enhancing the interactive experience.

[0144] In an exemplary embodiment, after step 310, the method may further include the following steps: displaying a viewing control for viewing the target object in the interface of the second sub-device; in response to a viewing operation triggered on the viewing control, displaying first multimedia data containing the target object in the interface of the second sub-device, and displaying an interactive control in the interface of the second sub-device.

[0145] The viewing control is used to view the first multimedia data including the target object, and the viewing operation is used to trigger the viewing control. If the user wants to view the first multimedia data, the user can view the first multimedia data including the target object in the interface of the second sub-device by performing the viewing operation on the viewing control in the interface of the second sub-device.

[0146] Regarding the interactive control, it is used to trigger the display of the first interactive information in the interface of the second sub-device. If the user wants to view the first interactive message, the first interactive information can be displayed in the interface of the second sub-device by triggering the interactive control.

[0147] For example, in the interface of the second sub-device, an icon of a viewing control is displayed, allowing the user to manually click the icon of the viewing control to view the monitoring screen (first multimedia data) of the second sub-device. When the user clicks the icon of the viewing control, the interactive system detects the viewing operation and displays the first multimedia data containing the target object in the interface of the second sub-device. At the same time, an icon of the interactive control is displayed in the interface of the second sub-device. When the icon of the interactive space is clicked, the interactive system detects the triggering operation of the interactive control, thereby displaying the first interactive information in the interface of the second sub-device.

[0148] Figure 8 A schematic diagram of a viewing operation is shown in FIG. Figure 8As shown, multiple spaces are displayed in the page 801 of the second sub-device, including a time control, a weather control, an event control, a viewing control, etc. Click the viewing control 802 to enter the video page 803. The first multimedia data containing the target object is displayed in the video page 803 of the second sub-device. The video page 803 also displays a trigger entry 804 of the interactive control. Click the trigger entry 804 to display multiple operation items of the first interactive information 806 on the video page 805.

[0149] Through the above process, the user can choose to manually view the first multimedia data to improve the efficiency of information acquisition, and the interface of the second sub-device can automatically hide the first interactive information, thereby reducing visual interference and allowing the user to focus on viewing the target object.

[0150] See also Fig. 9 The embodiment of the present application provides an interaction method, which is applicable to an electronic device, for example, the electronic device may be Figure 1 The user terminal 110 in the implementation environment is shown. The hardware structure of the electronic device can be as follows: Figure 2 shown.

[0151] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0152] like Fig. 9 As shown, the method may include the following steps:

[0153] Step 610, if a target event corresponding to the target object is detected by the first sub-device, the first interaction information and the first multimedia data containing the target object are displayed in the interface of the user terminal to synchronously respond to the display of the first interaction information and the first multimedia data in the interface of the second sub-device.

[0154] First, it is explained that the interaction method of the embodiment of the present application is applied to a user terminal, and the user terminal interacts with a split device, and the split device includes a first sub-device and a second sub-device. Among them, the target event is sent by the first sub-device when it detects that there is a target object within a set range. Specifically, when the first sub-device detects that there is a target object within the set range, it generates a target event and pushes it to the user terminal.

[0155] Regarding, the first interactive information includes at least one optional operation item, and the operation item may be an option that provides specific interactive selections for implementing specific interactive actions. Regarding the target object, it may be an entity that needs to be monitored in a specific application scenario, such as a visitor, an animal, etc., which is not limited here. Furthermore, the first multimedia data may be data corresponding to information content in various forms, which may be used to display the target object monitored by the first sub-device. For example, the first multimedia data may include text, images, and / or audio, which is not limited here.

[0156] For example, the first sub-device monitors the target object. If the first sub-device is integrated with a camera, a video including the target object can be generated to obtain first multimedia data, and the first multimedia data is sent to the user terminal. The user can view the video including the target object through the user terminal.

[0157] Regarding synchronous response, it may mean that in the interface of the user terminal and the second sub-device, the first interactive information and the first multimedia data should be updated at the same time to ensure that the information the user sees on the two devices is consistent. For example, when the target object presses the doorbell, the second sub-device immediately displays the picture outside the door, and the user terminal is also updated accordingly to display the same picture and operation items.

[0158] Step 630: In response to an operation triggered on the target operation item in the interface of the user terminal, second interaction information corresponding to the target operation item is output to the first sub-device.

[0159] The second interaction information is information generated and output according to the selected target operation item, and can be used to display interactive information content to the target object, that is, the second interaction information can indicate interaction with the target object.

[0160] As mentioned above, an operation item may be an option that provides specific interactive selections and is used to implement specific interactive actions. Multiple operation items may be displayed on a user terminal to facilitate the user to select an operation item on the user terminal, thereby determining a target operation item. The target operation item is used to indicate the interactive action that the user expects to implement.

[0161] Furthermore, the second interaction information is information generated and output according to the target operation item selected in the user terminal, and can be used to display interactive information content to the target object, that is, the second interaction information can indicate interaction with the target object.

[0162] It can be understood that the first sub-device is a device for interacting with the target object. Therefore, the second interaction information can be output on the first sub-device to display the interactive information content to the target object, and the target object can view the information content, thereby achieving interaction with the target object.

[0163] For example, when a user interacts with a visitor through a user terminal, corresponding second interaction information will be generated according to the target operation item triggered by the user, and the second interaction information will be output on the first sub-device, so that the visitor can view the text or voice feedback corresponding to the second interaction information through the first sub-device, for example, "leave the express at the door."

[0164] Through the above process, the user can view the first multimedia data monitored by the first sub-device through the user terminal, and select different operation items according to the first multimedia data, thereby reducing the communication burden between the user and the visitor and enhancing the convenience of operation; further, since the second interaction information is obtained based on the target operation item selected by the user, personalized interaction information can be provided to the visitor according to the actual situation, thereby improving the interaction efficiency between the visitor and the user.

[0165] In addition, the synchronous response between the second sub-device and the user terminal ensures that the user can obtain the same key information on any device, avoiding misunderstandings or missing important notifications due to inconsistent information. In addition, the user can freely choose which device to use for operation and interact on any device, providing greater flexibility.

[0166] In an exemplary embodiment, the interaction method may further include the following steps: displaying a custom control in an interface of a user terminal; acquiring a newly added operation item in response to an input operation triggered on the custom control; and sending the newly added operation item to a split device.

[0167] Among them, the custom control is used to add new operation items, and the input operation is used to customize the newly added operation items.

[0168] As mentioned above, the operation items provide specific interactive selection options for implementing specific interactive actions. If the user expects to add a custom operation item to implement a new interactive action, the user can input the corresponding interactive action content through the input operation triggered by the custom control to obtain the newly added operation item.

[0169] After the operation items are added, the newly added operation items can be sent to the split device, so that the newly added operation items are displayed in the interface of the second sub-device.

[0170] For example, a user adds a new quick reply item "leave the express at the door" through the user terminal. After completing the addition of the quick reply item, the newly added operation item can be sent to the split device, so that the newly added quick reply item "leave the express at the door" is displayed in the interface of the second sub-device.

[0171] Figure 9a A schematic diagram of a specific implementation of an input operation is shown in FIG. Figure 9aAs shown, there is a custom control 9a2 for quick reply items in page 9a1 of the user terminal. Click on the custom control 9a2 to trigger the input operation and enter the page 9a3 for adding a new quick reply item. Enter the reply message through the keyboard and confirm it. Then the new quick reply item is completed and the new quick reply item 9a4 is displayed in page 9a1.

[0172] Through the above process, users can customize operation items on the user terminal according to their needs, thereby improving the flexibility and personalization of interaction. By sending newly added operation items to the split device, users can quickly access and select newly added operation items on the second sub-device. The interactive system can adapt to changes in user needs, thereby enhancing the interactive experience.

[0173] Fig.10 This is a schematic diagram of a specific implementation of an interaction method in an application scenario. This application scenario applies the above interaction method, which is suitable for Figure 2 The interactive system shown in the embodiment and the application of the interactive method in the application scenario are as follows:

[0174] Specifically, the interactive system includes a smart door inner screen, a smart door outer screen and a mobile terminal. The smart door outer flat door is installed outdoors, the smart door inner screen is installed indoors, and the mobile terminal is the mobile phone carried by the user. Among them, the smart door outer screen and the smart door inner screen belong to the same split device, namely the smart door.

[0175] Through step 701, if the outdoor smart door screen detects that the visitor stays for a set time, the smart door screen is awakened and the doorbell control is displayed on the smart door screen.

[0176] Through step 703, if the smart door outer screen detects that the visitor clicks the doorbell control, the camera module of the smart door outer screen is called to shoot a video including the visitor, and the video is sent to the smart door inner screen indoors.

[0177] Specifically, when a visitor clicks the doorbell button on the smart door's outer screen, the smart door's outer screen will display a call animation and the doorbell will ring; at this time, the smart door's inner screen will automatically display the image outside the door and quick reply messages; at the same time, the mobile phone will also receive a push notification that "someone is ringing the doorbell."

[0178] Through step 705, the video including the visitor is displayed on the screen inside the smart door, and the quick reply items are displayed on the screen inside the smart door.

[0179] Through step 707, in response to the selection operation of the quick reply item, the corresponding second interaction information is generated and sent to the smart door external screen.

[0180] Among them, each quick reply item can correspond to a quick reply sentence.

[0181] Through step 709, the smart door screen receives the second interactive information and prompts the visitor in text and voice forms.

[0182] The second interaction information may include a quick reply message.

[0183] Specifically, the owner clicks on a preset quick reply through the smart door inner screen, and the smart door outer screen can receive the corresponding second interactive information, play the voice corresponding to the quick reply through the speaker, and display the text corresponding to the quick reply on the smart door outer screen.

[0184] Then, visitors can get quick replies through text or voice on the smart door screen. During voice playback, the smart door screen displays text and sound wave graphics. After a single voice is played once, the sound wave graphics on the smart door screen will become a clickable "Replay" button, and visitors can click "Replay" to play the voice again. If the owner clicks the quick reply item on the smart door screen multiple times, it will be regarded as sending multiple voices corresponding to quick replies. The smart door screen will automatically play the next voice after the previous voice is played, and the text will change synchronously.

[0185] It should be noted that when the smart door external screen finishes playing all voices, if the visitor does not perform any operation on the smart door external screen, the external screen will automatically turn off after 3 seconds.

[0186] Through step 711, according to the instruction of the second interaction information, the smart door controls the door lock to perform an unlocking operation.

[0187] The second interactive message may also include a device control instruction. Specifically, the owner clicks the unlock button through the inner screen of the smart door, and the smart door can receive the corresponding unlock instruction, thereby controlling the door lock to perform the unlock operation.

[0188] Of course, in addition to the smart internal screen automatically displaying the image outside the door when someone presses the doorbell button, the owner can also manually click on the "camera image" on the smart door internal screen to check the status outside the door.

[0189] In addition, when a visitor triggers the doorbell control, the mobile phone can respond synchronously, specifically, including the following steps:

[0190] Through step 801, a target event generated by a visitor is detected based on the smart door screen, and the target event and video are uploaded to the cloud.

[0191] Through step 803, the cloud pushes the target event to the mobile phone.

[0192] Specifically, when the smart door screen detects a visitor ringing the doorbell, the user's mobile phone will receive a relevant push card. The user clicks on the push card to automatically jump to the corresponding page. Among them, the smart home app will receive two types of push for "visitor events": someone ringing the doorbell or someone passing by.

[0193] Of course, the above push notifications are global to the app, and users can receive these two types of push notifications at any time when using the app.

[0194] It should be noted that after entering the smart home app through a mobile phone, it is necessary to check the network status. If the network connection is normal, it will automatically jump to the real-time picture taken by the smart door external screen; if the network is abnormal, the abnormal status will be displayed. It can be understood that the entire mobile interaction needs to communicate through the cloud, and instructions cannot be sent to the smart door without a network.

[0195] Through step 805, if the user views the target event through a mobile phone, the video including the visitor is obtained from the cloud through the mobile phone, and the quick reply item is displayed on the mobile phone.

[0196] Through step 807, in response to the selection operation of the quick reply item, the corresponding second interaction information is generated and sent to the smart door external screen via the mobile phone.

[0197] Each quick reply item can correspond to a quick reply. The preset voice (i.e. quick reply item) can be displayed in the app. Each voice corresponds to a quick reply. Click the voice, and the app can send the voice to the smart door, which will be played and displayed on the smart door screen. In addition to receiving relevant push notifications when someone rings the doorbell, the owner can also manually click "camera screen" in the app to check the status outside the door.

[0198] Through step 809, the smart door screen receives the quick reply and prompts the visitor in text and voice form.

[0199] In this application scenario, through the linkage of the smart door screen, the smart door screen, and the mobile phone, users can obtain the corresponding information of visitors in real time, and can make corresponding interactive operations to visitors regardless of whether the user is indoors, which improves the efficiency of information synchronization and response and ensures seamless communication between users and visitors. Furthermore, the operation items are displayed on the smart door screen and the mobile phone, and users can choose preset reply content to interact with visitors, or directly operate (such as unlocking), which reduces the tediousness of manual input by users and makes the interaction process more intuitive and fast.

[0200] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0201] The following is an embodiment of the device of the present application, which can be used to execute the interactive method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the interactive method involved in the present application.

[0202] See also Fig.11 In an embodiment of the present application, an interactive device 900 is provided, which is applied to a split device, wherein the split device includes a first sub-device and a second sub-device, and the interactive device includes but is not limited to: a display module 910, a determination module 930, and an interactive module 950.

[0203] Among them, the display module 910 is used to display the first interaction information and the first multimedia data containing the target object in the interface of the second sub-device if the first sub-device detects that there is a target object within a set range; the first interaction information includes at least one optional operation item.

[0204] The determination module 930 is used to obtain a selection operation for each operation item based on the first multimedia data in the interface of the second sub-device, and determine a selected target operation item.

[0205] The interaction module 950 is configured to output, by the first sub-device, second interaction information corresponding to the target operation item; the second interaction information indicates interaction with the target object.

[0206] In an exemplary embodiment, the target operation item includes a quick reply item; the second interaction information includes reply information corresponding to the quick reply item; the interaction module 910 is also used to: display the reply information corresponding to the quick reply item in the interface of the first sub-device; and / or play voice information corresponding to the reply information through the first sub-device.

[0207] In an exemplary embodiment, the interactive device 900 is also used to display an interactive operation item in the interface of the first sub-device if a target object is detected within a set range; the interactive operation item is used to prompt the target object to perform the target operation to interact with the second sub-device through the first sub-device; if the first sub-device detects a trigger operation of the target object on the interactive operation item, the first sub-device initiates a call connection request to the second sub-device to request to establish a call connection with the second sub-device.

[0208] In an exemplary embodiment, the interaction device 900 is further used to push a target event corresponding to the target object to the user terminal to instruct the first multimedia data containing the target object to be displayed in the interface of the user terminal.

[0209] In an exemplary embodiment, the interaction device 900 is also used to receive feedback information returned when a target operation is performed on a target object contained in the first multimedia data in the interface of the user terminal; and control the first sub-device to perform a corresponding target action according to the feedback information to interact with the target object.

[0210] In an exemplary embodiment, the interaction device 900 is also used to, if the feedback information includes a target operation item, cause the first sub-device to output second interaction information corresponding to the target operation item; if the feedback information includes a device control instruction, control the first sub-device to respond to the device control instruction and perform a target action corresponding to the device control instruction; if the feedback information includes a call instruction, control the first sub-device to establish a call connection with the user terminal.

[0211] In an exemplary embodiment, the interaction device 900 is also used to detect the residence time of the target object within the set range if the target object is detected within the set range; if the residence time is less than the set time, the first sub-device is controlled to enter a sleep state until the target object is detected within the set range again, and the first sub-device is controlled to switch from the sleep state to the working state.

[0212] In an exemplary embodiment, the interaction device 900 is also used to display a custom control in the interface of the second sub-device; the custom control is used to add a new operation item; in response to an input operation triggered by the custom control, the new operation item is obtained; and the first interaction information including the new operation item is displayed in the interface of the second sub-device.

[0213] In an exemplary embodiment, the interactive device 900 is also used to display a viewing control for viewing the target object in the interface of the second sub-device; in response to a viewing operation triggered by the viewing control, first multimedia data containing the target object is displayed in the interface of the second sub-device, and an interactive control is displayed in the interface of the second sub-device; the interactive control is used to trigger the display of first interactive information in the interface of the second sub-device.

[0214] See also Fig.12 In an embodiment of the present application, an interactive device 1000 is provided, which is applied to a user terminal, and the user terminal interacts with a split device, and the split device includes a first sub-device and a second sub-device. The interactive device includes but is not limited to: an event receiving module 1010, and a data display module 1030.

[0215] The event receiving module 1010 is used to receive a target event corresponding to a target object; the target event is sent by the first sub-device when it detects that there is a target object within a set range.

[0216] The data display module 1030 is used to display first multimedia data including a target object in the interface of a user terminal based on the received target event, so as to synchronously respond to displaying first interaction information and first multimedia data in the interface of a second sub-device, and / or the first sub-device outputs second interaction information corresponding to a target operation item; the target operation item is determined based on at least one selectable operation item in the first interaction information and the first multimedia data; the second interaction information indicates interaction with the target object.

[0217] In an exemplary embodiment, the interactive device 1000 is also used to display a custom control in the interface of the user terminal; the custom control is used to add a new operation item; in response to an input operation triggered by the custom control, the new operation item is obtained; and the new operation item is sent to the split device.

[0218] It should be noted that, when the interactive device provided in the above embodiment is interacting, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the interactive device will be divided into different functional modules to complete all or part of the functions described above.

[0219] In addition, the embodiments of the interaction device and the interaction method provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0220] See also Fig.13 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 may include: a split device and a user terminal.

[0221] exist Fig.13 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0222] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0223] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0224] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0225] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store a computer program in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to this.

[0226] The memory 4003 stores a computer program, and the processor 4001 can read the computer program stored in the memory 4003 through the communication bus 4002 .

[0227] The computer program is executed by one or more processors 4001 to implement the interaction methods in the above-mentioned embodiments.

[0228] In addition, an embodiment of the present application provides a storage medium, on which a computer program is stored. The computer program is executed by one or more processors to implement the interaction method as described above.

[0229] A computer program product is provided in an embodiment of the present application, including a computer program, which is executed by one or more processors to implement the interaction method as described above.

[0230] Compared with the related art, in the present solution, the user can view the first multimedia data monitored by the first sub-device through the second sub-device, and select different operation items according to the first multimedia data, thereby reducing the communication burden between the user and the visitor and enhancing the convenience of operation; further, since the second interaction information is obtained based on the target operation item selected by the user, personalized interaction information can be provided to the visitor according to the actual situation, thereby improving the interaction efficiency between the visitor and the user.

[0231] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An interactive method, characterized in that: Applied to a split device, the split device includes a first sub-device and a second sub-device, and the method includes: If the first sub-device detects that there is a target object within a set range, first interaction information and first multimedia data containing the target object are displayed in the interface of the second sub-device; the first interaction information includes at least one selectable operation item; Acquire a selection operation triggered on each of the operation items based on the first multimedia data in the interface of the second sub-device, and determine a selected target operation item; Second interaction information corresponding to the target operation item is outputted through the first sub-device; the second interaction information indicates interaction with the target object.

2. The method according to claim 1, characterized in that The target operation item includes a quick reply item; the second interaction information includes reply information corresponding to the quick reply item; The outputting, through the first sub-device, second interaction information corresponding to the target operation item includes: Displaying reply information corresponding to the quick reply item in the interface of the first sub-device; and / or, The voice information corresponding to the reply information is played through the first sub-device.

3. The method according to claim 1, characterized in that The method further comprises: If the target object is detected to exist within the set range, an interactive operation item is displayed in the interface of the first sub-device; the interactive operation item is used to prompt the target object to perform a target operation to interact with the second sub-device through the first sub-device; If the first sub-device detects a triggering operation of the target object on the interactive operation item, the first sub-device initiates a call connection request to the second sub-device to request to establish a call connection with the second sub-device.

4. The method according to claim 1, characterized in that If the first sub-device detects that there is a target object within a set range, the method further includes: A target event corresponding to the target object is pushed to the user terminal to instruct the first multimedia data including the target object to be displayed in the interface of the user terminal.

5. The method according to claim 4, characterized in that After pushing the target event corresponding to the target object to the user terminal, the method further includes: Receiving feedback information returned by executing a target operation on the target object included in the first multimedia data in an interface of the user terminal; The first sub-device is controlled to perform a corresponding target action according to the feedback information to interact with the target object.

6. The method according to claim 5, characterized in that The controlling the first sub-device to perform a corresponding target action according to the feedback information to interact with the target object includes: If the feedback information includes the target operation item, the first sub-device outputs second interaction information corresponding to the target operation item; If the feedback information includes a device control instruction, controlling the first sub-device to respond to the device control instruction and perform a target action corresponding to the device control instruction; If the feedback information includes a call instruction, the first sub-device is controlled to establish a call connection with the user terminal.

7. The method according to claim 1, characterized in that If the first sub-device detects that there is a target object within a set range, before displaying the first interaction information and the first multimedia data containing the target object in the interface of the second sub-device, the method further includes: If it is detected that the target object exists within the set range, detecting the stay time of the target object within the set range; If the stay time is less than the set time, the first sub-device is controlled to enter a sleep state until the target object is detected within the set range again, and the first sub-device is controlled to switch from the sleep state to a working state.

8. The method according to claim 1, characterized in that After displaying the first interaction information and the first multimedia data containing the target object in the interface of the second sub-device, the method further includes: In the interface of the second sub-device, a custom control is displayed; the custom control is used to add the operation item; In response to an input operation triggered on the custom control, obtaining the newly added operation item; The first interaction information including the newly added operation item is displayed on the interface of the second sub-device.

9. The method according to any one of claims 1 to 8, characterized in that If the first sub-device detects that there is a target object within a set range, the method further includes: In the interface of the second sub-device, displaying a viewing control for viewing the target object; In response to a viewing operation triggered by the viewing control, first multimedia data containing the target object is displayed in the interface of the second sub-device, and an interactive control is displayed in the interface of the second sub-device; the interactive control is used to trigger display of the first interactive information in the interface of the second sub-device.

10. An interactive method, characterized in that: Applied to a user terminal, the user terminal interacts with a split device, the split device includes a first sub-device and a second sub-device, the method includes: If a target event corresponding to a target object detected by the first sub-device is received, first interaction information and first multimedia data containing the target object are displayed in the interface of the user terminal, so as to synchronously respond to the display of the first interaction information and the first multimedia data in the interface of the second sub-device; the target event is sent by the first sub-device when the target object is detected within a set range; the first interaction information includes at least one optional operation item; In response to an operation triggered for a target operation item in an interface of the user terminal, second interaction information corresponding to the target operation item is output to the first sub-device; the second interaction information indicates interaction with the target object.

11. The method according to claim 10, characterized in that The method further comprises: In the interface of the user terminal, a custom control is displayed; the custom control is used to add the operation item; In response to an input operation triggered on the custom control, obtaining the newly added operation item; The newly added operation item is sent to the split device.

12. An interactive system, characterized in that: The system includes a split device and a user terminal, wherein the split device includes a first sub-device and a second sub-device; wherein, The first sub-device is used to detect whether a target object exists within a set range; the second sub-device is configured to display first interaction information and first multimedia data containing the target object in an interface of the second sub-device if the first sub-device detects that the target object exists within the set range; the first interaction information includes at least one selectable operation item; The second sub-device is further used to obtain a selection operation for each of the operation items based on the first multimedia data in an interface of the second sub-device, and determine a selected target operation item; The first sub-device is further configured to output second interaction information corresponding to the target operation item; the second interaction information indicates interaction with the target object; The user terminal is used to receive a target event corresponding to the target object, and display first multimedia data containing the target object in the interface of the user terminal based on the received target event; the target event is sent by the first sub-device when it detects the existence of a target object within a set range.

13. An electronic device comprising at least one processor and at least one memory, wherein: The memory stores a computer program, wherein the computer program, when executed by the processor, implements the interaction method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by one or more processors, the interactive method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by one or more processors, the interactive method according to any one of claims 1 to 11 is implemented.

Images