Data interaction method, terminal equipment and storage medium
By establishing a communication connection between the terminal device and the virtual scene, the interaction between virtual control requests and response data is realized, and the problem of information interaction between the terminal device in the real scene and the virtual scene is solved, and the comprehensiveness and effectiveness of the interaction are improved.
Patent Information
- Application Number
- CN202311424684.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-30
- Publication Date
- 2025-05-02
AI Technical Summary
The prior art is difficult to realize efficient information interaction between real-scene terminal devices and virtual scenes, especially the ability to control virtual devices and acquire virtual data.
By providing a data interaction method, the terminal device can obtain a virtual control request, establish a communication connection to interact with the target virtual device, send a virtual control request, and receive response data of the target virtual device.
It is realized that the terminal device can control the virtual device and obtain data in the virtual scene, thereby improving the comprehensiveness and effectiveness of information interaction between the virtual scene and the real scene.
Smart Images
Figure CN119916925A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data interaction method, a terminal device, and a storage medium. Background Art
[0002] Virtual scenes are network worlds based on simulated environments and using virtual characters / equipment avatars as carriers, allowing users to live and communicate in them. Users of virtual scenes can choose virtual digital models as their avatars, move by walking, flying, riding public transportation, and communicate through various media such as text, images, sound, and video.
[0003] The virtual scene is independent of the real scene. How the objects in the real scene control the objects in the virtual scene and obtain the virtual data generated by the objects in the virtual scene simulating the activities of the objects in the real scene is a technical difficulty that computer developers currently need to overcome. Summary of the invention
[0004] In order to overcome the problems existing in the related art, the present disclosure provides a data interaction method, a terminal device, and a storage medium. Through the data interaction method proposed in the present disclosure, the terminal device has the ability to control the virtual device and obtain the virtual data in the virtual scene, further improving the comprehensiveness of the information interaction between the virtual scene and the real scene.
[0005] According to a first aspect of an embodiment of the present disclosure, a data interaction method is provided, the data interaction method comprising:
[0006] Obtain a virtual control request for a target virtual device;
[0007] In the case where a communication connection is established between the terminal device and the target virtual device, the virtual control request is sent to the target virtual device, wherein the target virtual device is a device in the virtual scene having the same function as the terminal device;
[0008] The virtual data returned by the target virtual device in response to the virtual control request is received, where the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
[0009] In some embodiments, the method further comprises:
[0010] Acquiring communication information by calling a debugging tool of the terminal device;
[0011] Based on the communication information, a communication connection is established between the terminal device and the target virtual device.
[0012] In some embodiments, the acquiring communication information by calling a debugging tool of the terminal device includes:
[0013] The communication information is obtained by calling the debugging tool of the terminal device to scan the barcode corresponding to the target virtual device in the virtual scene.
[0014] In some embodiments, the communication information includes an engine address and a virtual identifier of the target virtual device; and establishing a communication connection between the terminal device and the target virtual device based on the communication information includes:
[0015] Sending a communication request to a scene server that builds the virtual scene based on the engine address;
[0016] Receiving a connection result returned by the scene server in response to the communication request;
[0017] When the connection result indicates that the connection is successful, the virtual identifier is sent to the scene server, so that the scene server establishes a communication connection between the terminal device and the target virtual device based on the virtual identifier.
[0018] In some embodiments, obtaining a virtual control request for a target virtual device includes:
[0019] When it is detected that the designated function in the function check card for the target virtual device is in a checked state, the virtual control request is generated, wherein the virtual control request is used to instruct the target virtual device to implement the designated function in response to the virtual control request.
[0020] In some embodiments, the method further comprises:
[0021] When the terminal device does not detect that the designated function is in the checked state, or when the terminal device and the target virtual device have not established a communication connection, the virtual control request is responded to by calling an application program interface of the terminal device.
[0022] In some embodiments, the method further comprises:
[0023] In a case where the virtual control request is directed to performing linkage control on the terminal device and the target virtual device, the terminal device is adjusted based on the virtual data.
[0024] In some embodiments, the receiving virtual data returned by the target virtual device in response to the virtual control request includes:
[0025] In a case where a virtual communication connection is established between the target virtual device and the virtual device to be controlled in the virtual scene, the virtual data returned by the target virtual device in response to the virtual control request to control the virtual device to be controlled is received.
[0026] In some embodiments, the receiving virtual data returned by the target virtual device in response to the virtual control request includes:
[0027] The virtual data returned by the target virtual device invoking a virtual application program interface in response to the virtual control request is received.
[0028] According to a second aspect of an embodiment of the present disclosure, a terminal device is provided, the terminal device comprising:
[0029] An acquisition module, configured to acquire a virtual control request for a target virtual device;
[0030] a sending module, configured to send the virtual control request to the target virtual device when a communication connection is established between the terminal device and the target virtual device, wherein the target virtual device is a device having the same function as the terminal device in the virtual scene;
[0031] The receiving module is configured to receive virtual data returned by the target virtual device in response to the virtual control request, wherein the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
[0032] According to a third aspect of an embodiment of the present disclosure, a terminal device is further provided, the terminal device comprising:
[0033] processor;
[0034] a memory for storing processor-executable instructions;
[0035] Wherein, the processor is configured to execute the data interaction method as described in the first aspect above.
[0036] According to the fourth aspect of the embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided. When the instructions in the storage medium are executed by the processor of the terminal device, the terminal device can execute the data interaction method described in the first aspect above.
[0037] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:
[0038] In the disclosed embodiments, when the terminal device establishes a communication connection with the target virtual device in the virtual scene, it can control the target virtual device with the same function to respond to the virtual control request issued by the terminal device, and obtain the virtual data obtained by the target virtual device in response to the virtual control request; in this way, on the one hand, the terminal device in the real scene has the ability to control the virtual device and obtain the virtual data in the virtual scene, further improving the comprehensiveness and effectiveness of the information interaction between the virtual scene and the real scene; on the other hand, based on the simulation of the same function of the terminal device by the target virtual device in the virtual scene, the time cost and financial cost of developing / testing the terminal device in the real scene can be reduced, and the efficiency of developers / tester in testing the designated functions of the terminal device can be improved.
[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0041] Figure 1 is a flowchart of a data interaction method according to an exemplary embodiment;
[0042] Figure 2 is a schematic diagram of a scene interface of a virtual scene according to an exemplary embodiment;
[0043] Figure 3 is a schematic diagram of a process of establishing a communication connection between a terminal device and a target virtual device according to an exemplary embodiment;
[0044] Figure 4 is a schematic diagram of a process of data interaction between a terminal device and a target virtual device according to an exemplary embodiment;
[0045] Figure 5 is a schematic diagram of the structure of a terminal device according to an exemplary embodiment;
[0046] Figure 6 It is a structural block diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION
[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0048] The present disclosure provides a data interaction method. Figure 1 , Figure 1 is a flow chart of a data interaction method according to an exemplary embodiment; Figure 1 As shown, the data interaction method is applied in a terminal device, and the terminal device can execute the data interaction method by the following steps:
[0049] S101, obtaining a virtual control request for a target virtual device;
[0050] S102: When a communication connection is established between the terminal device and the target virtual device, a virtual control request is sent to the target virtual device, wherein the target virtual device is a device in the virtual scene having the same function as the terminal device;
[0051] S103: Receive virtual data returned by the target virtual device in response to the virtual control request, where the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
[0052] In the embodiments of the present disclosure, the data interaction method can be applied to a terminal device of a real scene (an active scene in the real world). The terminal device has the ability to control a virtual device of a virtual scene (an active scene in the virtual world). The terminal device can be a device used by a developer, a device used by a tester, or a device used by a user, and the present disclosure does not impose any further restrictions on this. When the terminal device obtains a virtual control request, it can control the target virtual device and obtain the virtual data generated in the virtual scene based on the communication connection established with the target virtual device. In this way, through the data interaction method proposed in the present disclosure, the terminal device has the ability to control the virtual device and obtain the virtual data in the virtual scene, thereby improving the comprehensiveness and effectiveness of the information interaction between the virtual scene and the real scene.
[0053] Here, the terminal devices in the real scenarios proposed in the present disclosure include but are not limited to fixed terminals, mobile terminals and portable electronic devices. The fixed terminals include but are not limited to desktop computers and household appliances; the mobile terminals include but are not limited to mobile phones and tablets; the portable electronic devices include but are not limited to smart wearable devices such as smart watches.
[0054] In step S101, the application processing layer of the terminal device obtains a virtual control request for a target virtual device.
[0055] Here, the target virtual device is a virtual object created in the virtual scene. The target virtual device has the same functions as the terminal device in the real scene. For example, the terminal device has a camera function and an audio and video playback function, and the corresponding target virtual device can also have the same camera function and audio and video playback function.
[0056] In the embodiments of the present disclosure, the types and models of the target virtual device and the terminal device may be the same or different. For example, the terminal device is a mobile phone, and the target virtual device is a simulated mobile phone created in a virtual scene. The appearance of the target virtual device and the terminal device may be the same or different. The functions in the target virtual device may be exactly the same as those of the terminal device, or may only have some of the functions of the terminal device.
[0057] In the present disclosure, when the application processing layer detects a trigger event of a specified function developed by the holder of the terminal device for the terminal device, a virtual control request for the target device is generated. Here, the virtual control request may carry an identification code of the specified function, call logic information, etc. The terminal device may be a development / test device. After determining the virtual control request through the trigger event of the terminal device, the application processing layer further determines whether to control the target virtual device to execute the corresponding specified function.
[0058] Exemplarily, after the holder of the terminal device clicks the shutter button of the terminal device to start the camera function, the application processing layer generates a virtual control request for the target device, which is used to control the target virtual device to take pictures in the virtual scene (i.e., capture the panoramic image corresponding to the current shooting angle of the simulated camera of the target virtual device).
[0059] In step S102, when the application processing layer of the terminal device determines that a communication connection has been established between the terminal device and the target virtual device, the application processing layer of the terminal device sends the virtual control request to the target virtual device.
[0060] In the present disclosure, the target virtual device is an object created in the virtual scene after the scene server builds the virtual scene. Specifically, after the scene server builds the virtual scene, a server address is generated, which can be text and / or QR code information; the virtual device in the virtual scene (such as the target virtual device of the present disclosure) can open the login page of the virtual scene based on the access server address, and after logging in through the ID of the virtual device, the scene server confirms the login information and creates an engine service for the target virtual device; here, the creation of the engine service is characterized by opening a copy of the virtual scene for the virtual device, and the virtual device can perform a series of activities such as socializing and exercising in the copy of the virtual scene to simulate the terminal device in the real scene.
[0061] Here, the application processing layer of the terminal device determines whether there is a communication connection between the terminal device and the target virtual device when determining the virtual control request. The communication connection is a point-to-point connection between the target virtual device in the virtual scene copy built by the engine service based on the communication protocol connection established between the terminal device and the scene server. If the application processing layer determines that there is the communication connection, the virtual control request is sent to the target virtual device.
[0062] It should be noted that, when the application processing layer determines that the terminal device currently has no communication connection established with the target virtual device, it directly responds to the virtual control request and uses the terminal device itself to start / execute the specified function.
[0063] In the disclosed embodiment, the virtual scene is a scene constructed by the above-mentioned scene server; the user can access the server address of the scene server in the above-mentioned terminal device or other electronic device to browse the virtual scene and view the target virtual device in the virtual scene.
[0064] In some examples, the virtual scene can be a simulated scene that simulates a real scene. For example, for a real home bedroom or office, the virtual scene can be a simulated bedroom that simulates a real home bedroom, or a virtual office that simulates a real office. Here, a mobile phone in the simulated bedroom can perform activities such as taking photos and recording videos in the virtual scene to simulate the activities of taking photos and recording videos by a mobile phone in a real home bedroom. Figure 2 , Figure 2 is a schematic diagram of a scene interface of a virtual scene according to an exemplary embodiment; Figure 2 As shown, the virtual scene is the simulated bedroom scene, and the target virtual device in the simulated bedroom can be Figure 2 The mobile phone 201 shown is capable of acquiring images within the acquisition range of the camera.
[0065] In other examples, the virtual scene may also be a scene that constructs a virtual world for users to experience, for example, the virtual scene may be a scene of a virtual game world; the scene of the virtual game world may be a scene corresponding to a game interface opened by a terminal device or other electronic device through a game application, such as a game scene in a game interface opened through a mobile phone or computer, or a game scene corresponding to a game screen viewed through virtual reality glasses, etc. In step S103, after receiving the virtual control request based on the communication connection, the target virtual device parses the virtual control request to obtain the identification code of the specified function, call logic information and other data, then responds to the virtual control request, starts the specified function, and sends the response result of starting / running the specified function back to the terminal device through the communication connection.
[0066] In the present disclosure, the virtual data of the target virtual device starting / running the specified function can be generated instantly in response to the virtual control request, or it can be generated in real time during the operation of the specified function, or it can be obtained after the specified function is completed; for example, when the specified function is to start a chat software, the virtual data can be a confirmation message that the chat software has been opened, or it can be a chat message sent and received in the chat software, etc.
[0067] In the embodiment of the present disclosure, when a communication connection is established between a terminal device and a target virtual device, the terminal device can subscribe to the activity information of the target virtual device activity. The subscribed activity information can be the virtual data obtained by responding to the virtual control request as described above, or it can be the acquired data obtained by the terminal device directly sending a data acquisition request to the target virtual device; here, the acquired data can be the spatial parameters, time parameters, natural environment parameters, device information of the target virtual device, etc. in the virtual scene where the target virtual device is located. Here, the spatial parameters include but are not limited to the position of the target virtual device in the virtual scene, the relative position between the target virtual device and other virtual objects, etc.; the time parameters include the activity time of the target virtual device, the simulated time of the simulated real scene timeline, etc.; the natural environment parameters include but are not limited to the simulated temperature and simulated humidity of the virtual scene, etc.; the device information of the target virtual device includes the device ID, device type, device model, etc.
[0068] In the disclosed embodiments, when the terminal device establishes a communication connection with the target virtual device in the virtual scene, it can control the target virtual device with the same function to respond to the virtual control request issued by the terminal device, and obtain the virtual data obtained by the target virtual device in response to the virtual control request; in this way, on the one hand, the terminal device in the real scene has the ability to control the virtual device and obtain the virtual data in the virtual scene, further improving the comprehensiveness and effectiveness of the information interaction between the virtual scene and the real scene; on the other hand, based on the simulation of the same function of the terminal device by the target virtual device in the virtual scene, the time cost and financial cost of developing / testing the terminal device in the real scene can be reduced, and the efficiency of developers / tester in testing the designated functions of the terminal device can be improved.
[0069] In some embodiments, the data interaction method proposed in the embodiments of the present disclosure further includes:
[0070] Obtain communication information by calling the debugging tool of the terminal device;
[0071] Based on the communication information, a communication connection is established between the terminal device and the target virtual device.
[0072] Here, the debugging tools of the terminal device include but are not limited to search engines, debuggers, etc.
[0073] Taking the application of terminal equipment in the Android application system as an example, the debugger includes an application package (Android application package, APK) with debugging function. Port mapping or information scanning can be performed through the debugger APK, and communication information can be obtained based on the port mapping result or the information scanning result; for example, the debugger APK can be a scanning program set in the mobile phone, or a port mapping program set in the mobile phone, etc.
[0074] Here, the search engine can be installed in the browser software installed in the terminal device, and obtain the communication information of the target virtual device by searching the server address of the scene server.
[0075] In the present disclosure, after the communication information is acquired, the communication information is sent to the application processing layer (runtime layer), and the application processing layer sends a connection signal to the target virtual device based on the communication information, so as to establish a connection with the target virtual device. In the embodiment of the present disclosure, the debugging tool of the terminal device can be used to help establish a communication connection with the target virtual device, so that the virtual control request can be forwarded to the target virtual device, and the virtual data fed back by the target virtual device can be acquired in real time, thereby improving the effectiveness of data interaction.
[0076] In some embodiments, the above-mentioned obtaining of communication information by calling the debugging tool of the terminal device may be implemented in the following manner:
[0077] The communication information is obtained by calling the debugging tool of the terminal device to scan the barcode corresponding to the target virtual device in the virtual scene.
[0078] Here, the debugging tool includes a scanning program, and the communication information is obtained by scanning the barcode corresponding to the target virtual device in the virtual scene through the scanning program. In the present disclosure, the barcode includes but is not limited to a barcode, a QR code, etc.
[0079] It should be noted that the barcode of the target virtual device can be displayed on a terminal device or other electronic device. In some examples, the terminal device can access the server address mentioned above to view the virtual scene, and select the target virtual device in the virtual scene copy built by a certain engine service. By clicking the icon of the target virtual device, the barcode can be obtained. The terminal device saves the barcode in the mobile phone, and then scans the saved barcode based on calling the debugging tool to obtain the communication information. In other examples, a third-party electronic device (such as a computer) can be used to access the server address to view the virtual scene, and select the target virtual device in the virtual scene copy built by a certain engine service to obtain the barcode. At this time, the terminal device calls the debugging tool to scan the barcode displayed on the third-party electronic device to obtain the communication information.
[0080] The disclosed embodiment can obtain communication information by scanning the barcode of the target device, thereby establishing a communication connection between the terminal device and the target virtual device, which is helpful for subsequent information interaction between the real scene and the virtual scene.
[0081] In some embodiments, the communication information includes the engine address and the virtual identifier of the target virtual device; the communication connection between the terminal device and the target virtual device is established based on the communication information, which can be implemented in the following manner:
[0082] Send a communication request to the scene server that builds the virtual scene based on the engine address;
[0083] Receive a connection result returned by the scene server in response to the communication request;
[0084] When the connection result indicates that the connection is successful, the virtual identifier is sent to the scene server, so that the scene server establishes a communication connection between the terminal device and the target virtual device based on the virtual identifier.
[0085] Here, the engine address is the access address of the engine service corresponding to the target virtual device mentioned above, and the virtual scene copy where the target virtual device is located can be viewed through the engine address. The virtual identifier can uniquely identify the target virtual device and serve as the device ID of the target virtual device. The identity of the virtual device that interacts with the terminal device can be determined through the virtual identifier.
[0086] In the present disclosure, the debugging tool scans the barcode to obtain the engine address and virtual identifier, and then transmits them to the application processing layer (runtime layer). The application processing layer sends a communication request to the scene server that builds the virtual scene based on the full-duplex communication protocol (websocket) and the engine address, and waits for the response of the scene server; when the connection result returned by the scene server for the communication request indicates that the connection is successful, the communication handshake with the scene server is completed. At this time, based on the communication handshake result, the virtual identifier is sent to the scene server so that the engine service of the scene server marks the association between the target virtual device and the terminal device, thereby establishing a communication connection between the terminal device and the target virtual device; in the subsequent communication process, the information sent by the terminal device to the scene server will be forwarded to the target virtual device in a directional manner, and the engine service will also send the virtual data generated by the target virtual device activity to the terminal device in a directional manner.
[0087] It should be noted that in special circumstances such as data maintenance in a virtual scene or a poor network environment of the terminal device, the connection result indicates a connection failure. At this time, the terminal device cannot establish a communication connection between the target virtual devices. The application processing layer of the terminal device will actively respond to the virtual control request and start / execute the specified function of the virtual control request.
[0088] Here, the embodiment of the present disclosure can establish one-to-one communication between the target virtual device and the terminal device through the engine address and the virtual identifier, which helps to increase the success rate and effectiveness of the subsequent information interaction between the real scene and the virtual scene.
[0089] In some embodiments, obtaining the virtual control request for the target virtual device in step S101 may be implemented in the following manner:
[0090] When it is detected that the designated function in the function check card for the target virtual device is in a checked state, a virtual control request is generated, wherein the virtual control request is used to instruct the target virtual device to implement the designated function in response to the virtual control request.
[0091] Here, the function check card contains functions that the target virtual device can simulate in the virtual scene; there is a check box under each function, and when the check box corresponding to the specified function is checked, the application processing layer of the terminal device generates a virtual control request corresponding to the specified function.
[0092] In the present disclosure, the holder of the terminal device can adjust the check status corresponding to different functions according to actual needs, so as to adjust whether the terminal device controls the target virtual device to execute the specified function.
[0093] The disclosed embodiment improves the control flexibility of the terminal device over the target virtual device in the virtual scene by setting a function check card.
[0094] In some embodiments, the data interaction method proposed in the present disclosure further includes:
[0095] When the terminal device does not detect that the specified function is in a checked state, or the terminal device and the target virtual device have not established a communication connection, the virtual control request is responded to by calling the application program interface of the terminal device.
[0096] Here, an Application Programming Interface (API) is a collection of predefined method functions. By calling the API, a specified function can be fully or partially executed.
[0097] In the present disclosure, when it is detected that the check box corresponding to the specified function changes from a checked state to an unchecked state during the use of the terminal device, it is determined that the terminal device wants to execute the specified function itself. Therefore, the application processing layer calls the corresponding API in the terminal device to respond to the virtual control request.
[0098] It should be noted that if the application processing layer detects a trigger event of a specified function developed by the holder of the terminal device for the terminal device and determines that the specified function is not in the checked state or there is no communication connection between the terminal device and the target virtual device, a virtual control request is not generated at this time, and the corresponding API in the terminal device is called to respond to the trigger event to execute the specified function.
[0099] In some embodiments, the receiving of virtual data returned by the target virtual device in response to the virtual control request in step S103 may be implemented in the following manner:
[0100] The virtual data returned by the target virtual device invoking the virtual application program interface in response to the virtual control request is received.
[0101] Here, when both the target virtual device and the terminal device have a specified function, the terminal device is provided with a physical API for executing the specified function, and the target virtual device is provided with a virtual API for executing the specified function, and the function methods encapsulated in the physical API and the virtual API are consistent.
[0102] In the present disclosure, after the target virtual device receives the virtual control request, it calls the virtual API to execute the specified function, obtains the virtual data, and transmits the virtual data to the terminal device through the communication connection.
[0103] Thus, in the embodiment of the present disclosure, the terminal device can control the target virtual device to call the virtual API to execute the specified function in different states, and can also call the real API to execute the specified function.
[0104] It should be noted that the execution result of the terminal device calling the real API to execute the specified function may be the same as or different from the result of the target virtual device calling the virtual API to execute the specified function. In some examples, when the specified function is the photo function, the image captured by the target virtual device is the image of the virtual scene, and the image captured by the terminal device is the captured image of the real scene, and the execution results may be different at this time; in other instances, when the specified function is the video playback function, both the target virtual device and the virtual device can open the same video, and the execution results can be the same.
[0105] The present disclosure can enable a terminal device to have the ability to control a virtual device based on the call to a virtual API, thereby improving the functions of the terminal device and enhancing the practicality.
[0106] In some embodiments, the data interaction method proposed in the present disclosure also includes
[0107] When the virtual control request is directed to linkage control of the terminal device and the target virtual device, the terminal device is adjusted based on the virtual data.
[0108] In the application scenario of the present disclosure, the target virtual device can also feedback control the terminal device. Specifically, when the terminal device and the target virtual device are linked to each other, the terminal device controls the terminal device to execute a specified function or other functions after acquiring the virtual data.
[0109] For example, if both the terminal device and the target virtual device are air conditioning devices, when the specified function is automatic temperature adjustment, the target virtual device responds to the virtual control request and turns on the heating adjustment mode in combination with the simulated temperature of the virtual scene. At this time, the virtual data obtained by the terminal device is the temperature change result of the virtual scene. Based on this, in some examples, the terminal device can call the API corresponding to the heating adjustment mode to turn on the heating function according to the temperature change result; in some other examples, the terminal device can also adjust the temperature of the real scene to the same as the current temperature in the virtual scene according to the temperature change result and the current temperature of the real scene. At this time, the terminal device can call the heating adjustment mode or the cooling adjustment mode.
[0110] In the present disclosure, the virtual control request pointing to the linkage control of the terminal device and the target virtual device is realized through the function check card described above in the present disclosure. The check card also has an option box for linkage control at the specified function. When the option box is checked, the corresponding virtual control request instructs the terminal device and the target virtual device to perform linkage control on the specified function, that is, feedback control of the target virtual device.
[0111] In the method proposed in the embodiment of the present disclosure, the target virtual device can also affect the terminal device, thereby improving the linkage effect between the virtual scene and the real scene.
[0112] In some embodiments, the receiving of virtual data returned by the target virtual device in response to the virtual control request in step S103 may be implemented in the following manner:
[0113] In the case where a virtual communication connection is established between the target virtual device and the virtual device to be controlled in the virtual scene, virtual data returned by the target virtual device controlling the virtual device to be controlled in response to the virtual control request is received.
[0114] Here, in addition to the target virtual device, the virtual scene built by the engine service also has a virtual device to be controlled; a real wireless communication connection or a wired communication connection can be simulated between the virtual device to be controlled and the target virtual device. Here, the simulated wireless communication connection can be a virtual communication connection established based on a simulated Bluetooth signal, a wireless fidelity signal, etc.
[0115] It should be noted that the relative distance between the target virtual device and the virtual device to be controlled and the signal strength of the simulated signal will affect the result of the virtual communication connection just like in a real scenario.
[0116] When a virtual communication connection is established between the target virtual device and the virtual device to be controlled in the virtual scene, the target virtual device can control the virtual device to be controlled to respond to the virtual control request to complete the specified function. For example, the specified function can be video projection, the target virtual device is a simulated mobile phone, and the virtual device to be controlled is an analog TV. After the terminal device plays the video and selects the projection function, the simulated mobile phone controls the analog TV to display the video when it has a virtual Bluetooth connection with the analog TV.
[0117] In the disclosed embodiment, the terminal device can control the linkage of multiple devices in the virtual scene, improve the simulation effect of the virtual scene on the real scene, and make the functions of the terminal device more complete, thereby improving the effectiveness of information interaction.
[0118] At present, various IoT devices are gradually increasing, and computer technology has also made many breakthroughs. When developing the functions of terminal devices or developing the linkage of multiple device functions, how to complete the development and debugging efficiently and at low cost is a difficult problem faced by developers. In some scenarios, such as testing the linkage of 1,000 robots in a park, the time / capital cost required to build a real scene is huge, and it is difficult to observe the status of all robot devices globally. In this way, developers can build a virtual scene to simulate the linkage of 1,000 robots, but how the real terminal device can obtain the activity information of one or more robots in the virtual scene has not yet been solved. In other scenarios, when a mobile phone placed in the bedroom is connected to the virtual bedroom corresponding to the virtual scene, how to display the photos taken as images of the virtual bedroom, the developers have never found a solution.
[0119] Based on the above problems, the present disclosure embodiment proposes the above data interaction method; see Figure 3 and Figure 4 , Figure 3 is a schematic diagram of a process of establishing a communication connection between a terminal device and a target virtual device according to an exemplary embodiment; Figure 4 is a schematic diagram of a process flow of data interaction between a terminal device and a target virtual device according to an exemplary embodiment; Figure 3 and Figure 4 An exemplary application of the data interaction method proposed in the present disclosure between a terminal device and a target virtual device in a virtual scene is described.
[0120] like Figure 3 As shown, the terminal device accessing the virtual scene can be implemented by the following steps:
[0121] Step S301, start the virtual scene;
[0122] Here, the virtual scene is the scene built by the virtual server.
[0123] Step S302, displaying the server address;
[0124] Here, the server address may be text and / or QR code information.
[0125] Step S303, opening a virtual scene page based on the server address;
[0126] Step S304, sending a login request;
[0127] Step S305, creating an engine service for the target virtual device based on the login request;
[0128] Step S306, sending the engine address;
[0129] Step S307, entering the virtual scene based on the engine address;
[0130] Here, the engine service creates a copy of the virtual scene for the target virtual device, and the virtual scene also has other virtual devices, thereby simulating a real scene, such as the robot linkage scene mentioned above, or a virtual bedroom scene.
[0131] Step S308, responding to a selection event for a target virtual device;
[0132] Step S309, displaying a QR code;
[0133] Here, the QR code includes the engine address and the virtual identifier, and the virtual identifier includes the device ID of the target virtual device;
[0134] Step S310, scan the QR code to obtain communication information;
[0135] Here, the communication information includes the engine address and the device ID.
[0136] Step S311, sending the communication information to the application processing layer;
[0137] Step S312, establishing a connection between the terminal device and the target virtual device, and entering the virtual mode.
[0138] The application processing layer (runtime layer) establishes a communication connection between the terminal device and the target virtual device based on the engine address and device ID in the communication information, which serves as the communication basis for the subsequent data interaction process.
[0139] like Figure 4 As shown, the interaction process between the terminal device and the target virtual device can be implemented by the following steps:
[0140] Step S401, connecting to a target virtual device in a virtual scene built by an engine service based on a full-duplex communication protocol;
[0141] Here, the engine address and device ID are obtained through the QR code information of the target virtual device logged in the virtual scene, and a communication connection between the terminal device and the target virtual device is established in combination with the full-duplex communication protocol (websocket).
[0142] Step S402, returning the communication connection result.
[0143] Here, the communication connection result can be a successful connection or a connection failure. In special cases such as data maintenance in a virtual scene or a poor network environment of the terminal device, the engine service will return a connection failure result.
[0144] When the communication connection result indicates that the connection is successful, a communication connection is established between the terminal device and the target virtual device, and data interaction in subsequent processes is completed based on the communication connection.
[0145] Step S403, subscribing to the activity information of the target virtual device activity;
[0146] Here, the subscribed activity information may be virtual data obtained in response to a virtual control request, or may be space parameters, time parameters, natural environment parameters in a virtual scene (virtual world) where a target virtual device is located, device information of a target virtual device, etc.
[0147] Step S404, distributing the activity information to the terminal device;
[0148] Here, the virtual device may feed back the activity information to the terminal device continuously or at a preset time interval.
[0149] Step S405, sending a virtual control request;
[0150] Here, the virtual scene provides a set of virtual APIs, and the virtual API calling method and calling process are consistent with the terminal device. Specifically, when the physical API call follows the synchronous rule, the virtual API call also follows the synchronous rule, and when the physical API call follows the asynchronous rule, the virtual API call also follows the asynchronous rule. In the present disclosure, when the target virtual device responds to the virtual control request, it obtains virtual data by calling the virtual API to execute the specified function.
[0151] Step S406, returning virtual data.
[0152] Here, taking the designated function of taking photos as an example, the virtual data is a picture taken by the target virtual device in the virtual scene (ie, a panoramic image corresponding to the current shooting angle of the simulated camera of the target virtual device).
[0153] Through the data interaction method proposed in the embodiment of the present disclosure, functional tests related to terminal devices and multi-device linkage can be completed efficiently and at low cost, solving the problems faced by developers; at the same time, it also provides the ability for real devices to control virtual devices and obtain activity information in virtual scenes, enriching the experience of real devices accessing and controlling virtual scenes.
[0154] Figure 5 is a schematic diagram of the structure of a terminal device according to an exemplary embodiment; Figure 5 As shown, the terminal device 500 includes:
[0155] An acquisition module 501 is configured to acquire a virtual control request for a target virtual device;
[0156] The sending module 502 is configured to send the virtual control request to the target virtual device when a communication connection is established between the terminal device and the target virtual device, wherein the target virtual device is a device having the same function as the terminal device in the virtual scene;
[0157] The receiving module 503 is configured to receive virtual data returned by the target virtual device in response to the virtual control request, where the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
[0158] In some embodiments, the terminal device 500 further includes:
[0159] Calling module( Figure 4 (not shown), configured to obtain communication information by calling a debugging tool of the terminal device;
[0160] Connection module( Figure 4 (not shown) and is configured to establish a communication connection between the terminal device and the target virtual device based on the communication information.
[0161] In some embodiments, the calling module is further configured to obtain the communication information by calling a debugging tool of the terminal device to scan a barcode corresponding to the target virtual device in the virtual scene.
[0162] In some embodiments, the communication information includes an engine address and a virtual identifier of the target virtual device; the connection module is also configured to send a communication request to a scene server that builds the virtual scene based on the engine address; receive a connection result returned by the scene server in response to the communication request; and if the connection result indicates a successful connection, send the virtual identifier to the scene server so that the scene server establishes a communication connection between the terminal device and the target virtual device based on the virtual identifier.
[0163] In some embodiments, the acquisition module 501 is also configured to generate the virtual control request when it is detected that the specified function in the function check card for the target virtual device is in a checked state, wherein the virtual control request is used to instruct the target virtual device to respond to the virtual control request to implement the specified function.
[0164] In some embodiments, the terminal device 500 further includes:
[0165] Response Module( Figure 4 (not shown), configured to respond to the virtual control request by calling the application program interface of the terminal device when the terminal device does not detect that the specified function is in the checked state, or when the terminal device and the target virtual device have not established a communication connection.
[0166] In some embodiments, the response module is further configured to adjust the terminal device based on the virtual data when the virtual control request points to linkage control of the terminal device and the target virtual device.
[0167] In some embodiments, the receiving module 503 is further configured to receive the virtual data returned by the target virtual device controlling the virtual device to be controlled in response to the virtual control request when a virtual communication connection is established between the target virtual device and the virtual device to be controlled in the virtual scene.
[0168] In some embodiments, the receiving module 503 is further configured to receive the virtual data returned by the target virtual device invoking a virtual application program interface in response to the virtual control request.
[0169] Figure 6 6 is a block diagram of a terminal device according to an exemplary embodiment. The terminal device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.
[0170] The processing component 602 generally controls the overall operation of the terminal device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned data interaction method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.
[0171] The memory 604 is configured to store various types of data to support operations on the terminal device 600. Examples of such data include instructions for any application or method operating on the terminal device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0172] The power supply component 606 provides power to various components of the terminal device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 600.
[0173] The multimedia component 608 includes a screen that provides an output interface between the terminal device 600 and the user. The pixel circuit proposed in the present disclosure is applied on the screen. In some embodiments, the screen may include an OLED display panel, such as an AMOLED display panel, and may also be a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the terminal device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0174] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the terminal device 600 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.
[0175] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0176] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the terminal device 600. For example, the sensor assembly 614 can detect the open / closed state of the terminal device 600, the relative positioning of the components, such as the display and keypad of the terminal device 600, and the sensor assembly 614 can also detect the position change of the terminal device 600 or a component of the terminal device 600, the presence or absence of contact between the user and the terminal device 600, the orientation or acceleration / deceleration of the terminal device 600, and the temperature change of the terminal device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0177] The communication component 616 is configured to facilitate the communication between the terminal device 600 and other devices in a wired or wireless manner. The terminal device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0178] In an exemplary embodiment, the terminal device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.
[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of the terminal device 600. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0180] Here, when the instructions in the storage medium are executed by the processing component of the terminal device, the terminal device is enabled to execute a data interaction method, and the data interaction method includes:
[0181] Obtain a virtual control request for a target virtual device;
[0182] In the case where a communication connection is established between the terminal device and the target virtual device, the virtual control request is sent to the target virtual device, wherein the target virtual device is a device in the virtual scene having the same function as the terminal device;
[0183] The virtual data returned by the target virtual device in response to the virtual control request is received, where the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
[0184] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0185] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A data interaction method, characterized in that: The method comprises: Obtain a virtual control request for a target virtual device; In the case where a communication connection is established between the terminal device and the target virtual device, the virtual control request is sent to the target virtual device, wherein the target virtual device is a device in the virtual scene having the same function as the terminal device; The virtual data returned by the target virtual device in response to the virtual control request is received, where the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
2. The method according to claim 1, characterized in that The method further comprises: Acquiring communication information by calling a debugging tool of the terminal device; Based on the communication information, a communication connection is established between the terminal device and the target virtual device.
3. The method according to claim 2, characterized in that The acquiring of communication information by calling a debugging tool of the terminal device includes: The communication information is obtained by calling the debugging tool of the terminal device to scan the barcode corresponding to the target virtual device in the virtual scene.
4. The method according to claim 2, characterized in that: The communication information includes an engine address and a virtual identifier of the target virtual device; and establishing a communication connection between the terminal device and the target virtual device based on the communication information includes: Sending a communication request to a scene server that builds the virtual scene based on the engine address; Receiving a connection result returned by the scene server in response to the communication request; When the connection result indicates that the connection is successful, the virtual identifier is sent to the scene server, so that the scene server establishes a communication connection between the terminal device and the target virtual device based on the virtual identifier.
5. The method according to any one of claims 1 to 4, characterized in that: The obtaining of a virtual control request for a target virtual device includes: When it is detected that the designated function in the function check card for the target virtual device is in a checked state, the virtual control request is generated, wherein the virtual control request is used to instruct the target virtual device to implement the designated function in response to the virtual control request.
6. The method according to claim 5, characterized in that The method further comprises: When the terminal device does not detect that the designated function is in the checked state, or when the terminal device and the target virtual device have not established a communication connection, the virtual control request is responded to by calling an application program interface of the terminal device.
7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: In a case where the virtual control request is directed to performing linkage control on the terminal device and the target virtual device, the terminal device is adjusted based on the virtual data.
8. The method according to any one of claims 1 to 4, characterized in that: The receiving virtual data returned by the target virtual device in response to the virtual control request includes: In a case where a virtual communication connection is established between the target virtual device and the virtual device to be controlled in the virtual scene, the virtual data returned by the target virtual device in response to the virtual control request to control the virtual device to be controlled is received.
9. The method according to any one of claims 1 to 4, characterized in that: The receiving virtual data returned by the target virtual device in response to the virtual control request includes: The virtual data returned by the target virtual device invoking a virtual application program interface in response to the virtual control request is received.
10. A terminal device, characterized in that: include: An acquisition module, configured to acquire a virtual control request for a target virtual device; a sending module, configured to send the virtual control request to the target virtual device when a communication connection is established between the terminal device and the target virtual device, wherein the target virtual device is a device having the same function as the terminal device in the virtual scene; The receiving module is configured to receive virtual data returned by the target virtual device in response to the virtual control request, wherein the virtual data is a response result of the target virtual device in response to the virtual control request in the virtual scene.
11. A terminal device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the data interaction method as described in any one of claims 1 to 9.
12. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by the processor of the terminal device, the terminal device is enabled to execute the data interaction method as described in any one of claims 1 to 9.