Control method and device, electronic equipment and medium

Abstract

The embodiment of the invention discloses a control method and device, electronic equipment and a medium, and the method comprises the steps: adding a plurality of objects in a target scene in which first equipment runs, and creating private hierarchies corresponding to the plurality of objects in the target scene; wherein the private hierarchy corresponding to the object is used for the object to select an application to be started; for each object in the plurality of objects, receiving a starting request of starting an application triggered by the object in the corresponding private hierarchy; in response to the starting request, running the application in the target scene; obtaining degree-of-freedom information of a second device corresponding to the object, and obtaining a rendered image in the target scene according to the degree-of-freedom information; and sending the rendered image to a second device corresponding to the object for display.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of electronic devices, and more specifically, to a control method, a control device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the development of augmented reality technology, AR products such as AR glasses have appeared in more and more application scenarios, such as vehicle-mounted AR glasses. However, in the application scenarios of users in vehicles, there is a lack of single-scene multi-user multi-application usage scenarios. Summary of the invention

[0003] The purpose of the embodiments of the present disclosure is to provide a control method, device, electronic device and medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a control method is provided, the method comprising:

[0005] Adding multiple objects to a target scene running on the first device, and creating multiple private hierarchies corresponding to the objects in the target scene; wherein the private hierarchies corresponding to the objects are used for the objects to select applications to be opened;

[0006] For each of the plurality of objects, receiving a start request for starting an application triggered by the object in the corresponding private layer;

[0007] In response to the start request, running the application in the target scene;

[0008] Acquire degree of freedom information of a second device corresponding to the object, and acquire a rendered image in the target scene according to the degree of freedom information;

[0009] The rendered image is sent to a second device corresponding to the object for display.

[0010] Optionally, adding multiple objects to the target scene running on the first device includes:

[0011] For each of the plurality of objects, a second device corresponding to the object scans an information code at a corresponding position of the first device to obtain a relative position of the object in the target scene;

[0012] The object is added to the target scene according to the relative position of the object in the target scene.

[0013] Optionally, the application is a built-in application in the target scene, and in response to the start request, running the application in the target scene includes:

[0014] In response to the start request, creating a first application object corresponding to the application for the object;

[0015] Add the first application object to the private layer corresponding to the object.

[0016] Optionally, the application is a third-party application, and in response to the start request, running the application in the target scenario includes:

[0017] In response to the start request, creating a first virtual screen, and running the application on the first virtual screen;

[0018] Creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen on the first canvas;

[0019] Add the first canvas to the private hierarchy corresponding to the object.

[0020] Optionally, after adding multiple objects to the target scene running on the first device, the method further includes:

[0021] For each of the plurality of objects, a virtual camera group corresponding to the object is created.

[0022] Optionally, the acquiring degree of freedom information of a second device corresponding to the object, and acquiring a rendered image in the target scene according to the degree of freedom information, includes:

[0023] Acquire degree of freedom information of a second device corresponding to the object;

[0024] According to the degree of freedom information, the corresponding virtual camera group is controlled to collect the rendered image in the target scene.

[0025] Optionally, the application is a built-in application in the target scene, and in response to the start request, running the application in the target scene includes:

[0026] In response to the start request, creating a first application object corresponding to the application for the object;

[0027] When the access mode of the application is a personal mode, adding the first application object to a private layer corresponding to the object;

[0028] When the access mode of the application is a public mode, the first application object is added to a public layer corresponding to the plurality of objects.

[0029] Optionally, the application is a third-party application, and in response to the start request, running the application in the target scenario includes:

[0030] In response to the start request, creating a first virtual screen, and running the application on the first virtual screen;

[0031] Creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen on the first canvas;

[0032] When the access mode of the application is a personal mode, adding the first canvas to a private level corresponding to the object;

[0033] When the access mode of the application is a public mode, the first canvas is added to a public layer corresponding to the plurality of objects.

[0034] Optionally, the acquiring degree of freedom information of a second device corresponding to the object, and acquiring a rendered image in the target scene according to the degree of freedom information, includes:

[0035] When the access mode of the application is a personal mode, obtaining degree of freedom information of a second device corresponding to the object, and controlling the corresponding virtual camera group to collect a rendered image in the target scene according to the degree of freedom information;

[0036] When the access mode of the application is a public mode, the degree of freedom information of the second devices respectively corresponding to the multiple objects is obtained, and according to the degree of freedom information of the second devices respectively corresponding to the multiple objects, the virtual camera groups respectively corresponding to the multiple objects are controlled to capture the rendered image in the target scene.

[0037] Optionally, the multiple objects include a first object and a second object, and after the multiple objects are added to the target scene running on the first device and private hierarchies corresponding to the multiple objects are created in the target scene, the method further includes:

[0038] Receiving a first start request for starting a third-party application triggered by the first object in the corresponding private layer;

[0039] In response to the first start request, creating a second virtual screen, and running the application on the second virtual screen;

[0040] Creating a second canvas in the target scene, rendering and displaying the texture information of the second virtual screen on the second canvas, and adding the second canvas to the private level corresponding to the first object;

[0041] receiving a second start request for starting the third-party application triggered by the second object in the corresponding private layer;

[0042] In response to the second start request, a third canvas is created in the target scene, texture information of the second virtual screen is rendered and displayed on the third canvas, and the third canvas is added to the private level corresponding to the second object.

[0043] Optionally, the method further comprises:

[0044] When the working mode of the second device corresponding to the first object is the head-over mode, the texture information of the second virtual screen is sent to the second device corresponding to the first object, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the first object;

[0045] Sending broadcast information to a second device corresponding to the second object through a second device corresponding to the first object;

[0046] Upon receiving confirmation information returned by the second device corresponding to the second object in response to the broadcast information, the texture information of the second virtual screen is sent to the second device corresponding to the second object, so that the texture information of the second virtual screen is rendered and displayed on the display screen of the second device corresponding to the second object.

[0047] According to a second aspect of an embodiment of the present disclosure, a control device is provided, the device comprising:

[0048] A creation module, used to add multiple objects to a target scene running on the first device, and create multiple private levels corresponding to the objects in the target scene; wherein the private levels corresponding to the objects are used for the objects to select applications to be opened;

[0049] A receiving module, configured to receive, for each of the plurality of objects, a start request for starting an application triggered by the object in the corresponding private layer;

[0050] A running module, configured to run the application in the target scenario in response to the start request;

[0051] An acquisition module, used to acquire degree of freedom information of a second device corresponding to the object, and acquire a rendered image in the target scene according to the degree of freedom information;

[0052] A sending module is used to send the rendered image to a second device corresponding to the object for display.

[0053] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, the electronic device comprising:

[0054] Memory for storing executable computer instructions;

[0055] A processor is used to execute the control method according to the first aspect above under the control of the executable computer instructions.

[0056] According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the control method described in the first aspect above is executed.

[0057] A beneficial effect of the embodiments of the present disclosure is that multiple objects are added to a target scene running on a first device, and private hierarchies corresponding to the multiple objects are created in the target scene, and for each of the multiple objects, a start request for starting an application triggered by the object in the corresponding private hierarchy is received, and in response to the start request, the application is run in the target scene, and the degree of freedom information of the second device corresponding to the object is obtained, and a rendered image in the target scene is obtained based on the degree of freedom information, and the rendered image is sent to the second device corresponding to the object for display, so that multiple applications created by multiple users can coexist in the same target scene.

[0058] Other features and advantages of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the specification.

[0060] Figure 1 is a schematic diagram of the hardware configuration of a control system according to an embodiment of the present disclosure;

[0061] Figure 2 is a flow chart of a control method according to an embodiment of the present disclosure;

[0062] Figure 3 is a principle block diagram of a control device according to an embodiment of the present disclosure;

[0063] Figure 4 is a principle block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0064] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.

[0065] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0066] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0067] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0068] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0069] <Hardware Configuration>

[0070] Figure 1 The figure is a schematic diagram of the hardware configuration of a control system that can be used to implement a control method of an embodiment. Figure 1 A first device 100, a second device 200 and a network 300 are shown. The first device 100 can be connected to the network 300, and can also be connected to the second device 200 through a communication method such as Bluetooth. In one embodiment, the first device 100 is connected to the second device 200 only through a communication method such as Bluetooth. Among them, multiple servers 301 and 302 can be set in the network 300. The network 300 can be a wireless communication network or a wired communication network. The network 300 can be a local area network or a wide area network. The network 300 can be a short-range communication or a long-range communication.

[0071] In one embodiment, Figure 1 As shown, the first device 100 may include a processor 101 and a memory 102. The first device 100 also includes a communication device 103, a display device 104, a user interface 105, a camera device 106, an audio / video interface 107, a sensor 108, etc. In addition, the first device 100 may also include a power management chip 109 and a battery 110, etc.

[0072] Among them, the processor 101 can be various processors. The memory 102 can store the underlying software, system software, application software, data, etc. required for the operation of the first device 100. The memory 102 can include various forms of memory, such as ROM, RAM, Flash, etc. The communication device 103 can include, for example, a WiFi communication device, a Bluetooth communication device, a 3G, 4G and 5G communication device, etc. Through the communication device 103, the first device 100 can be arranged in the network. The display device 104 can be a liquid crystal display, an OLED display, etc. In one example, the display device 104 can be a touch screen. The user can perform input operations through the display device 104. In addition, the user can also perform fingerprint recognition through the touch screen. The user interface 105 may include a USB interface, a lightning interface, a keyboard, etc. The camera device 106 can be a single camera or multiple cameras. The audio / video interface 107 can include, for example, a speaker interface, a microphone interface, a video transmission interface such as HDMI, etc. The sensor 108 can include, for example, a gyroscope, an accelerometer, a temperature sensor, a humidity sensor, a pressure sensor, etc. For example, the posture information of the vehicle can be determined by the sensor. The power management chip 109 can be used to manage the power input to the first device 100, and can also manage the battery 110 to ensure greater utilization efficiency. The battery 110 is, for example, a lithium-ion battery.

[0073] The first device 100 may be a vehicle, and the vehicle includes an on-board vehicle system. Figure 1 The various components shown are only exemplary. The first device 100 may include Figure 1 One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The illustrated first device 100 is merely illustrative and is in no way intended to limit the embodiments herein, their application, or uses.

[0074] In one embodiment, the second device 200 may include a processor 201 and a memory 202. The second device 200 also includes a communication device 203, a display device 204, a user interface 205, a camera device 206, an audio / video interface 207, a sensor 208, etc. In addition, the second device 200 may also include a power management chip 209 and a battery 210, etc.

[0075] The second device 200 may be a head mounted display device, for example, AR (Augmented Reality) glasses and MR (Mixed Reality) glasses, etc., which are not limited in the embodiments of the present disclosure. Figure 1 The various components shown are only schematic. The second device 200 may include Figure 1One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The second device 200 shown is for illustrative purposes only and is in no way intended to limit the embodiments herein, their application, or uses.

[0076] It should be understood that although Figure 1 Only one first device 100 and second device 200 are shown, but this does not mean to limit their respective numbers, and the control system may include multiple first devices 100 and multiple second devices 200.

[0077] In the above description, the technician can design instructions according to the solution provided by the present disclosure. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0078] <Method Example>

[0079] Figure 2 A control method according to an embodiment of the present disclosure is shown. The control method can be performed by Figure 1 In the first device implementation shown, the first device may be a vehicle including an on-board vehicle system.

[0080] like Figure 2 As shown, the control method of this embodiment may include the following steps S2100 to S2500:

[0081] Step S2100: adding a plurality of objects to a target scene running on a first device, and creating a plurality of private hierarchies corresponding to the objects in the target scene.

[0082] The private level corresponding to the object is used for the object to select an application to be opened, and the application may be a built-in application of the target scene or a third-party application, which is not limited in this embodiment.

[0083] The multiple objects may be multiple users, and each user uses a second device. Taking the first device as a vehicle as an example, the front passenger, the right rear passenger, and the left rear passenger of the vehicle all wear AR glasses.

[0084] Specifically, after adding multiple objects to the target scene run by the first device, the first device can only create private hierarchies corresponding to the multiple objects in the target scene, that is, each object has an exclusive private hierarchy, which is used to render the private virtual objects of the object, and the virtual objects in the private hierarchy are only visible to the object. Moreover, the private hierarchy defaults to a menu object established for the corresponding object, and the object can select the application it wants to open through the menu object.

[0085] Specifically, after multiple objects are added to the target scene running on the first device, the first device can not only create private layers corresponding to the multiple objects in the target scene, but also create a public layer corresponding to the multiple objects in the target scene. That is to say, multiple objects share a common layer, which can be used for rendering common virtual objects of multiple objects, and the virtual objects in the common layer are visible to multiple objects.

[0086] In one embodiment, this step S2100 adds multiple objects to the target scene running on the first device, and creates multiple private levels corresponding to the objects in the target scene, which can further include: for each of the multiple objects, scanning the information code at the corresponding position of the first device by the second device corresponding to the object, to obtain the relative position of the object in the target scene; according to the relative position of the object in the target scene, add the object to the target scene.

[0087] The information code may be a QR code, and each information code carries a position information. Taking the first device as a vehicle as an example, a QR code is set at the co-pilot position of the vehicle, and the QR code carries the co-pilot position information of the vehicle; a QR code is set at the right rear position of the vehicle, and the QR code carries the right rear position information of the vehicle; a QR code is set at the left rear position of the vehicle, and the QR code carries the left rear position information of the vehicle.

[0088] In an optional embodiment, after adding multiple objects to the target scene running on the first device, the control method of the embodiment of the present disclosure further includes: for each of the multiple objects, creating a virtual camera group corresponding to the object.

[0089] The virtual camera group includes a first virtual camera and a second virtual camera, wherein the first virtual camera is used to simulate the left eye of the corresponding object, and the second virtual camera is used to simulate the right eye of the corresponding object. Specifically, after the first device creates a corresponding virtual camera group for the object, it can obtain the six degrees of freedom (6DoF) information of the corresponding second device, and control the displacement and rotation of the virtual camera group through the 6DoF information of the corresponding second device. The virtual camera group can collect and render images and transmit them to the corresponding second device, and the second device only calculates the 6DoF information and displays the rendered images.

[0090] Exemplarily, the vehicle starts the AR application, builds the vehicle scene, and the co-pilot passenger uses AR glasses to scan the QR code at the co-pilot position in the vehicle, and obtains the position information corresponding to the co-pilot passenger as the co-pilot position information. After the co-pilot passenger confirms, the AR glasses can send the co-pilot position information to the on-board vehicle system. The on-board vehicle system can use the co-pilot position information as the relative position of the co-pilot passenger in the vehicle scene, add the co-pilot passenger to the vehicle scene based on the relative position, create a new virtual camera group for the co-pilot passenger's scene rendering, and create a private level for the co-pilot passenger in the vehicle scene.

[0091] The right rear passenger uses AR glasses to scan the QR code at the right rear position in the vehicle, and obtains the position information corresponding to the right rear passenger as the right rear position information. After the right rear passenger confirms, the AR glasses can send the right rear position information to the onboard vehicle system. The onboard vehicle system can use the right rear position information as the relative position of the right rear passenger in the vehicle scene, add the right rear passenger to the vehicle scene based on the relative position, create a new virtual camera group for the scene rendering of the right rear passenger, and create a private layer for the right rear passenger in the vehicle scene.

[0092] The left rear passenger uses AR glasses to scan the QR code at the left rear position in the vehicle, and obtains the position information corresponding to the left rear passenger as the left rear position information. After the left rear passenger confirms, the AR glasses can send the left rear position information to the onboard vehicle system. The onboard vehicle system can use the left rear position information as the relative position of the left rear passenger in the vehicle scene, add the left rear passenger to the vehicle scene based on the relative position, create a new virtual camera group for the scene rendering of the left rear passenger, and create a private layer for the left rear passenger in the vehicle scene.

[0093] Of course, the onboard car computer can also create a common layer for the front passenger, right rear passenger and left rear passenger in the vehicle scene.

[0094] Step S2200: For each of the plurality of objects, receiving a start request for starting an application triggered by the object in the corresponding private layer.

[0095] In this embodiment, each object added to the target scene of the vehicle-mounted vehicle system can individually select the application to be opened in the corresponding private layer.

[0096] Step S2300: In response to the start request, run the application in the target scene.

[0097] In an optional embodiment, the vehicle-mounted vehicle system only creates a corresponding private layer for each object, and the application is a built-in application in the target scene. Then this step S2300 responds to the start request and runs the application in the target scene, which may further include: responding to the start request, creating a first application object corresponding to the application for the object; adding the first application object to the private layer corresponding to the object.

[0098] Continuing with the above example, the front passenger passenger can open built-in application 1 in the vehicle scene in the corresponding first level. The onboard vehicle system creates an application object corresponding to built-in application 1 for the front passenger passenger, and adds the application object corresponding to built-in application 1 to the first level corresponding to the front passenger passenger. The right rear passenger can open built-in application 2 in the vehicle scene in the corresponding first level. The onboard vehicle system creates an application object corresponding to built-in application 2 for the right rear passenger, and adds the application object corresponding to built-in application 2 to the first level corresponding to the right rear passenger. The left rear passenger can open built-in application 3 in the vehicle scene in the corresponding first level. The onboard vehicle system creates an application object corresponding to built-in application 3 for the left rear passenger, and adds the application object corresponding to built-in application 3 to the first level corresponding to the left rear passenger. In this way, different objects can open different built-in applications in the same target scene, allowing multiple built-in applications to coexist in the same target scene.

[0099] In an optional embodiment, the vehicle-mounted vehicle system only creates a corresponding private layer for each object, and the application is a third-party application. Then this step S2300 responds to the startup request and runs the application in the target scene, which may further include: responding to the startup request, creating a first virtual screen, and running the application on the first virtual screen; creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen to the first canvas; adding the first canvas to the private layer corresponding to the object.

[0100] Continuing with the above example, the front passenger passenger can open third-party application 1 in the corresponding first level, and the onboard vehicle system opens the third-party application 1 on the newly created virtual screen 1, and creates a new canvas 1 in the vehicle scene for the front passenger passenger, renders and displays the texture information of virtual screen 1 through canvas 1, and adds canvas 1 to the first level corresponding to the front passenger passenger. The right rear passenger can open third-party application 2 in the corresponding first level, and the onboard vehicle system opens the third-party application 2 on the newly created virtual screen 2, and creates a new canvas 2 in the vehicle scene for the right rear passenger, renders and displays the texture information of virtual screen 2 through canvas 2, and adds canvas 2 to the first level corresponding to the right rear passenger. The left rear passenger can open third-party application 3 in the corresponding first level, and the onboard vehicle system opens the third-party application 3 on the newly created virtual screen 3, and creates a new canvas 3 in the vehicle scene for the left rear passenger, renders and displays the texture information of virtual screen 3 through canvas 3, and adds canvas 3 to the first level corresponding to the left rear passenger. In this way, different objects can open different third-party applications in the same target scene, allowing multiple third-party applications to coexist in the same target scene.

[0101] In an optional embodiment, the vehicle-mounted vehicle system not only creates a corresponding private level for each object, but also creates a common public level for multiple objects, and the application is a built-in application in the target scene. Then this step S2300 responds to the start request, and running the application in the target scene may further include: in response to the start request, creating a first application object corresponding to the application for the object; when the access mode of the application is personal mode, adding the first application object to the private level corresponding to the object; when the access mode of the application is public mode, adding the first application object to the public level corresponding to multiple objects.

[0102] In this embodiment, for any object, if the object opens a built-in application in the corresponding first level, the vehicle-mounted vehicle system can create an application object corresponding to the built-in application for the object, and when the access mode of the built-in application is personal mode, add the application object corresponding to the built-in application to the private level corresponding to the object. When the access mode of the built-in application is public mode, add the application object corresponding to the built-in application to the public level.

[0103] In an optional embodiment, the onboard vehicle system not only creates a corresponding private level for each object, but also creates a common public level for multiple objects, and the application is a built-in application in the target scene. Then this step S2300 responds to the startup request, and running the application in the target scene may further include: creating a first virtual screen in response to the startup request, and running the application on the first virtual screen; creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen to the first canvas; when the access mode of the application is personal mode, adding the first canvas to the private level corresponding to the object; when the access mode of the application is public mode, adding the first canvas to the public level corresponding to multiple objects.

[0104] In this embodiment, for any object, if the object opens a third-party application in the corresponding first level, the vehicle-mounted vehicle system can create a virtual screen for the object to run the third-party application, and create a canvas in the target scene, and use the canvas to render and display the texture information of the virtual screen. If the access mode of the third-party application is personal mode, the canvas is added to the private level corresponding to the object. If the access mode of the third-party application is public mode, the canvas is added to the public level.

[0105] Step S2400, obtaining degree of freedom information of a second device corresponding to the object, and obtaining a rendered image in the target scene according to the degree of freedom information.

[0106] The degree of freedom information of the second device may be 6DoF information of the second device.

[0107] In an optional embodiment, the on-board vehicle system only creates a corresponding private level for each object. Then, step S2400 obtains the degree of freedom information of the second device corresponding to the object, and obtains the rendered image in the target scene based on the degree of freedom information. It can further include: obtaining the degree of freedom information of the second device corresponding to the object, and controlling the corresponding virtual camera group to capture the rendered image in the target scene based on the degree of freedom information.

[0108] In an optional embodiment, the onboard vehicle system not only creates a corresponding private level for each object, but also creates a common public level for multiple objects. Then this step S2400 obtains the degree of freedom information of the second device corresponding to the object, and obtains the rendered image in the target scene according to the degree of freedom information, which may further include: when the access mode of the application is a personal mode, obtain the degree of freedom information of the second device corresponding to the object, and control the corresponding virtual camera group to collect the rendered image in the target scene according to the degree of freedom information; when the access mode of the application is a public mode, obtain the degree of freedom information of the second devices corresponding to the multiple objects, and control the virtual camera groups corresponding to the multiple objects to collect the rendered image in the target scene according to the degree of freedom information of the second devices corresponding to the multiple objects.

[0109] Step S2500: sending the rendered image to a second device corresponding to the object for display.

[0110] According to an embodiment of the present disclosure, multiple objects are added to a target scene running on a first device, and private hierarchies corresponding to the multiple objects are created in the target scene. For each of the multiple objects, a start request for starting an application triggered by the object in the corresponding private hierarchy is received, and in response to the start request, the application is run in the target scene, and the degree of freedom information of the second device corresponding to the object is obtained, and a rendered image in the target scene is obtained based on the degree of freedom information, and the rendered image is sent to the second device corresponding to the object for display. In this way, multiple applications created by multiple users can coexist in the same target scene.

[0111] In one embodiment, the vehicle-mounted vehicle system only creates a corresponding private level for each object, and the multiple objects include a first object and a second object. After executing the above step S2100, multiple objects are added to the target scene running on the first device, and the private levels corresponding to the multiple objects are created in the target scene, the control method of the embodiment of the present disclosure may further include the following steps S3100 to S3500:

[0112] Step S3100: receiving a first start request for starting a third-party application triggered by the first object in the corresponding private layer.

[0113] Step S3200: creating a second virtual screen in response to the first start request, and running the application on the second virtual screen.

[0114] Step S3300: create a second canvas in the target scene, render and display the texture information of the second virtual screen on the second canvas, and add the second canvas to the private level corresponding to the first object.

[0115] Step S3400: receiving a second start request for starting the third-party application triggered by the second object in the corresponding private layer.

[0116] Step S3500: In response to the second start request, create a third canvas in the target scene, render and display the texture information of the second virtual screen on the third canvas, and add the third canvas to the private level corresponding to the second object.

[0117] It should be noted that if the onboard vehicle system not only creates a corresponding private level for each object, but also creates a common public level for multiple objects, then the above step S3300 can determine whether to add the second canvas to the private level corresponding to the first object or the public level corresponding to multiple objects according to the access mode of the third-party application. The above step S3500 can determine whether to add the third canvas to the private level corresponding to the second object or the public level corresponding to multiple objects according to the access mode of the third-party application.

[0118] It should be noted that if the first object opens the built-in application in the target scene, the onboard vehicle system can create an application object corresponding to the built-in application for the first object. If the second object also opens the built-in application in the target scene, the onboard vehicle system can also create an application object corresponding to the built-in application for the second object.

[0119] According to this embodiment, different objects in the same target scene can open the same application.

[0120] In one embodiment, the control method of the embodiment of the present disclosure further includes the following steps S3100 to S3300:

[0121] Step S3100, when the working mode of the second device corresponding to the first object is the head-over mode, the texture information of the second virtual screen is sent to the second device corresponding to the first object, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the first object.

[0122] In this embodiment, the working mode of the second device includes a 6DoF mode and a head-over-head mode. When the working mode of the second device corresponding to the first object is the 6DoF mode, the second device can send its degree of freedom information to the first device. When the working mode of the second device corresponding to the first object is the head-over-head mode, the second device will not send its degree of freedom information to the first device. At this time, the first device can send the texture information of the running second virtual screen to the second device corresponding to the first object.

[0123] Step S3200: Send broadcast information through the second device corresponding to the first object.

[0124] The broadcast information is used to notify other second devices that the working mode of the second device corresponding to the first object is the follow-up mode.

[0125] Step S3300, when the confirmation information returned by the second device corresponding to the second object in response to the broadcast information is received, the texture information of the second virtual screen is sent to the second device corresponding to the second object, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the second object.

[0126] According to this embodiment, when the object opens a third-party application, it can select the follow-up mode. In this follow-up mode, the corresponding AR glasses no longer receive the rendered image in the vehicle scene, but directly receive the texture information of the virtual screen running the third-party application. At the same time, the object can send a broadcast to share the screen it sees with other objects. After other users confirm, they can see the same screen content as the object.

[0127] <Device Example>

[0128] Figure 3 is a schematic diagram of a control device according to an embodiment, referring to Figure 3 As shown, the control device 300 includes a creation module 310 , a receiving module 320 , an operation module 330 , an acquisition module 340 and a sending module 350 .

[0129] A creation module 310 is used to add multiple objects to a target scene running on the first device, and create multiple private hierarchies corresponding to the objects in the target scene; wherein the private hierarchies corresponding to the objects are used for the objects to select applications to be opened;

[0130] The receiving module 320 is used to receive, for each of the plurality of objects, a start request for starting an application triggered by the object in the corresponding private layer;

[0131] A running module 330, configured to run the application in the target scenario in response to the start request;

[0132] An acquisition module 340 is used to acquire degree of freedom information of a second device corresponding to the object, and acquire a rendered image in the target scene according to the degree of freedom information;

[0133] The sending module 350 is used to send the rendered image to a second device corresponding to the object for display.

[0134] In one embodiment, the creation module 310 is specifically used to obtain, for each of the multiple objects, the relative position of the object in the target scene by scanning the information code at the corresponding position of the first device through the second device corresponding to the object; and adding the object to the target scene according to the relative position of the object in the target scene.

[0135] In one embodiment, the application is a built-in application in the target scene, and the running module 330 is specifically used to create a first application object corresponding to the application for the object in response to the start request; and add the first application object to the private layer corresponding to the object.

[0136] In one embodiment, the application is a third-party application, and the running module 330 is specifically used to create a first virtual screen in response to the start request, and run the application on the first virtual screen; create a first canvas in the target scene, and render and display the texture information of the first virtual screen to the first canvas; add the first canvas to the private layer corresponding to the object.

[0137] In one embodiment, the creation module 310 is further configured to create, for each of the multiple objects, a virtual camera group corresponding to the object.

[0138] In one embodiment, the acquisition module 340 is specifically used to obtain the degree of freedom information of the second device corresponding to the object, and control the corresponding virtual camera group to capture the rendering image in the target scene according to the degree of freedom information.

[0139] In one embodiment, the application is a built-in application in the target scene, and the running module 330 is specifically used to create a first application object corresponding to the application for the object in response to the start request; when the access mode of the application is a personal mode, the first application object is added to the private layer corresponding to the object; when the access mode of the application is a public mode, the first application object is added to the public layers corresponding to multiple objects.

[0140] In one embodiment, the application is a third-party application, and the running module 330 is specifically used to create a first virtual screen in response to the start request, and run the application on the first virtual screen; create a first canvas in the target scene, and render and display the texture information of the first virtual screen on the first canvas; when the access mode of the application is a personal mode, add the first canvas to the private layer corresponding to the object; when the access mode of the application is a public mode, add the first canvas to the public layer corresponding to multiple objects.

[0141] In one embodiment, the acquisition module 340 is specifically used to obtain the degree of freedom information of the second device corresponding to the object when the access mode of the application is a personal mode, and control the corresponding virtual camera group to collect the rendered image in the target scene according to the degree of freedom information; when the access mode of the application is a public mode, obtain the degree of freedom information of the second devices corresponding to the multiple objects, and control the virtual camera groups corresponding to the multiple objects to collect the rendered image in the target scene according to the degree of freedom information of the second devices corresponding to the multiple objects.

[0142] In one embodiment, the plurality of objects comprises a first object and a second object.

[0143] The receiving module 320 is further configured to receive a first start request for starting a third-party application triggered by the first object in the corresponding private layer;

[0144] The creation module 310 is further configured to create a second virtual screen in response to the first start request, and run the application on the second virtual screen; create a second canvas in the target scene, and render and display the texture information of the second virtual screen on the second canvas;

[0145] The receiving module 320 is further configured to receive a second start request for starting the third-party application triggered by the second object in the corresponding private layer;

[0146] The creation module 310 is further configured to create a third canvas in the target scene in response to the second start request, and render and display the texture information of the second virtual screen on the third canvas.

[0147] In one embodiment, the sending module 350 is also used to send the texture information of the second virtual screen to the second device corresponding to the first object when the working mode of the second device corresponding to the first object is the heel mode, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the first object; send broadcast information to the second device corresponding to the second object through the second device corresponding to the first object; and send the texture information of the second virtual screen to the second device corresponding to the second object when receiving the confirmation information returned by the second device corresponding to the second object in response to the broadcast information, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the second device.

[0148] According to an embodiment of the present disclosure, multiple objects are added to a target scene running on a first device, and private hierarchies corresponding to the multiple objects are created in the target scene. For each of the multiple objects, a start request for starting an application triggered by the object in the corresponding private hierarchy is received, and in response to the start request, the application is run in the target scene, and the degree of freedom information of the second device corresponding to the object is obtained, and a rendered image in the target scene is obtained based on the degree of freedom information, and the rendered image is sent to the second device corresponding to the object for display. In this way, multiple applications created by multiple users can coexist in the same target scene.

[0149] <Equipment Embodiment>

[0150] Figure 4 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment. Figure 4 As shown, the head mounted display device 400 includes a processor 410 and a memory 420 .

[0151] The memory 420 may be used to store executable computer instructions.

[0152] The processor 410 can be used to execute the control method described in the embodiment of the method of the present disclosure according to the control of the executable computer instructions.

[0153] The electronic device 400 may be Figure 1 The first device 100 shown may also be Figure 1 The second device 200 shown may also be a device with other hardware structures, which is not limited here.

[0154] In another embodiment, the electronic device 400 may include the above control device 300 .

[0155] In one embodiment, each module of the above control device 300 can be implemented by the processor 410 running computer instructions stored in the memory 420 .

[0156] <Computer Readable Storage Medium>

[0157] The embodiment of the present disclosure further provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the control method provided by the embodiment of the present disclosure is executed.

[0158] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0159] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0160] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0161] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0162] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0163] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0164] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0165] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0166] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A control method, characterized in that: The method comprises: Adding multiple objects to a target scene running on the first device, and creating multiple private hierarchies corresponding to the objects in the target scene; wherein the private hierarchies corresponding to the objects are used for the objects to select applications to be opened; For each of the plurality of objects, receiving a start request for starting an application triggered by the object in the corresponding private layer; In response to the start request, running the application in the target scene; Acquire degree of freedom information of a second device corresponding to the object, and acquire a rendered image in the target scene according to the degree of freedom information; The rendered image is sent to a second device corresponding to the object for display.

2. The method according to claim 1, characterized in that The adding of multiple objects to the target scene running on the first device includes: For each of the plurality of objects, a second device corresponding to the object scans an information code at a corresponding position of the first device to obtain a relative position of the object in the target scene; The object is added to the target scene according to the relative position of the object in the target scene.

3. The method according to claim 1, characterized in that The application is a built-in application in the target scene, and in response to the start request, running the application in the target scene includes: In response to the start request, creating a first application object corresponding to the application for the object; Add the first application object to the private layer corresponding to the object.

4. The method according to claim 1, characterized in that: The application is a third-party application, and in response to the start request, running the application in the target scenario includes: In response to the start request, creating a first virtual screen, and running the application on the first virtual screen; Creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen on the first canvas; Add the first canvas to the private hierarchy corresponding to the object.

5. The method according to claim 1, characterized in that After adding multiple objects to the target scene running on the first device, the method further includes: For each of the plurality of objects, a virtual camera group corresponding to the object is created.

6. The method according to claim 5, characterized in that The obtaining the degree of freedom information of the second device corresponding to the object, and obtaining the rendered image in the target scene according to the degree of freedom information, includes: Acquire degree of freedom information of a second device corresponding to the object; According to the degree of freedom information, the corresponding virtual camera group is controlled to collect the rendering image in the target scene.

7. The method according to claim 1, characterized in that The application is a built-in application in the target scene, and in response to the start request, running the application in the target scene includes: In response to the start request, creating a first application object corresponding to the application for the object; When the access mode of the application is a personal mode, adding the first application object to a private layer corresponding to the object; When the access mode of the application is a public mode, the first application object is added to a public layer corresponding to the plurality of objects.

8. The method according to claim 1, characterized in that The application is a third-party application, and in response to the start request, running the application in the target scenario includes: In response to the start request, creating a first virtual screen, and running the application on the first virtual screen; Creating a first canvas in the target scene, and rendering and displaying the texture information of the first virtual screen on the first canvas; When the access mode of the application is a personal mode, adding the first canvas to a private level corresponding to the object; When the access mode of the application is a public mode, the first canvas is added to a public layer corresponding to the plurality of objects.

9. The method according to claim 5, characterized in that The obtaining the degree of freedom information of the second device corresponding to the object, and obtaining the rendered image in the target scene according to the degree of freedom information, includes: When the access mode of the application is a personal mode, obtaining degree of freedom information of a second device corresponding to the object, and controlling the corresponding virtual camera group to collect a rendered image in the target scene according to the degree of freedom information; When the access mode of the application is a public mode, the degree of freedom information of the second devices respectively corresponding to the multiple objects is obtained, and according to the degree of freedom information of the second devices respectively corresponding to the multiple objects, the virtual camera groups respectively corresponding to the multiple objects are controlled to capture the rendered image in the target scene.

10. The method according to claim 1, characterized in that The multiple objects include a first object and a second object. After the multiple objects are added to the target scene running on the first device and private hierarchies corresponding to the multiple objects are created in the target scene, the method further includes: Receiving a first start request for starting a third-party application triggered by the first object in the corresponding private layer; In response to the first start request, creating a second virtual screen, and running the application on the second virtual screen; Creating a second canvas in the target scene, rendering and displaying the texture information of the second virtual screen on the second canvas, and adding the second canvas to the private level corresponding to the first object; receiving a second start request for starting the third-party application triggered by the second object in the corresponding private layer; In response to the second start request, a third canvas is created in the target scene, texture information of the second virtual screen is rendered and displayed on the third canvas, and the third canvas is added to the private level corresponding to the second object.

11. The method according to claim 10, characterized in that The method further comprises: When the working mode of the second device corresponding to the first object is the head-over mode, the texture information of the second virtual screen is sent to the second device corresponding to the first object, so as to render and display the texture information of the second virtual screen on the display screen of the second device corresponding to the first object; Sending broadcast information through a second device corresponding to the first object; Upon receiving confirmation information returned by the second device corresponding to the second object in response to the broadcast information, the texture information of the second virtual screen is sent to the second device corresponding to the second object, so that the texture information of the second virtual screen is rendered and displayed on the display screen of the second device corresponding to the second object.

12. A control device, characterized in that: The device comprises: A creation module, used to add multiple objects to a target scene running on the first device, and create multiple private levels corresponding to the objects in the target scene; wherein the private levels corresponding to the objects are used for the objects to select applications to be opened; A receiving module, configured to receive, for each of the plurality of objects, a start request for starting an application triggered by the object in the corresponding private layer; A running module, configured to run the application in the target scenario in response to the start request; An acquisition module, used to acquire degree of freedom information of a second device corresponding to the object, and acquire a rendered image in the target scene according to the degree of freedom information; A sending module is used to send the rendered image to a second device corresponding to the object for display.

13. An electronic device, characterized in that: The electronic device comprises: Memory for storing executable computer instructions; A processor, configured to execute the control method according to any one of claims 1 to 11 under the control of the executable computer instructions.

14. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are executed by a processor to execute the control method according to any one of claims 1 to 11.

Images