Equipment picture display method and device, equipment and storage medium

Abstract

The invention discloses an equipment picture display method and device, equipment and a storage medium, and relates to the technical field of virtualization. The method comprises the following steps: identifying one or more second XR devices in a target area; acquiring a second display picture of second XR equipment in the target area; the second display picture is displayed in the first display picture, the first display picture further displays a second XR device, and the second display picture and the second XR device are displayed in an associated mode. By means of the technical means, the problem that in the prior art, a teacher needs to manually input operation to check the display picture of the student XR equipment is solved, and the teacher can check the display picture of the student XR equipment without feeling.

Description

Technical Field

[0001] The present application relates to the field of virtual technology, and in particular to a device screen display method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of virtual technology, it can be deeply integrated with teaching scenarios to create immersive virtual classrooms and improve students' learning efficiency. In virtual classrooms, students and teachers can use XR (Extended Reality) devices to demonstrate and view virtual classroom content.

[0003] In the existing technology, the teacher's device can obtain the display screen of each student's XR device and generate a display screen and monitoring list for each student's XR device based on the content. When the teacher wants to view the display screen of any student's XR device, he or she can find the corresponding student's XR device or display screen in the monitoring list. After clicking the corresponding student's XR device or display screen, the teacher's device will zoom in and display the display content of the corresponding student's XR device. However, in the classroom, teachers do not always have free time to manually input the operation of viewing the display screen of the student's XR device, and the manual operation of viewing the display screen by the teacher may interrupt the teacher's teaching ideas and affect the teaching effect. Summary of the Invention

[0004] The present application provides a device screen display method, apparatus, device and storage medium to solve the problem in the prior art that teachers need to manually input operations to view the display screen of students' XR devices. This enables teachers to view the display screen of students' XR devices without feeling, avoids interrupting teaching ideas when teachers view students' XR devices, and ensures classroom teaching effectiveness.

[0005] In a first aspect, the present application provides a device screen display method, comprising:

[0006] identifying one or more second XR devices within the target area;

[0007] Acquire a second display image of a second XR device in the target area;

[0008] The second display screen is displayed within the first display screen, the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

[0009] In a second aspect, the present application provides a device screen display apparatus, comprising:

[0010] a device identification module configured to identify one or more second XR devices within the target area;

[0011] A picture acquisition module is configured to acquire a second display picture of a second XR device in the target area;

[0012] The screen display module is configured to display the second display screen within the first display screen, the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

[0013] In a third aspect, the present application provides a device for displaying a screen of a device, comprising:

[0014] One or more processors; a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the device screen display method as described in the first aspect.

[0015] In the present application, one or more second XR devices in the target area are identified; a second display screen of the second XR device in the target area is obtained; the second display screen is displayed in the first display screen, the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device. Through the above technical means, the second XR device used by the student in the target area can be identified, and then the second display screen of the second XR device can be obtained. The obtained second display screen is associated with the corresponding second XR device and displayed in the first display screen viewed by the teacher, so that the teacher can seamlessly view the second display screen of the second XR device used by the student sitting in front of him / her, solving the problem in the prior art that the teacher needs to manually input operations to view the second display screen of the second XR device used by the student, avoiding the interruption of the teaching train of thought when the teacher views the second XR device used by the student, and ensuring the teaching effect in the classroom. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a device screen display method provided by an embodiment of the present application;

[0017] Figure 2 This is a flowchart of another device screen display method provided by an embodiment of the present application;

[0018] Figure 3 This is a flowchart of determining the three-dimensional coordinates of a shooting area in a virtual classroom provided by an embodiment of the present application;

[0019] Figure 4 is a schematic diagram of the target device provided in an embodiment of the present application;

[0020] Figure 5 is a schematic diagram of a display screen of a first XR device provided in an embodiment of the present application;

[0021] Figure 6This is a flowchart of another device screen display method provided by an embodiment of the present application;

[0022] Figure 7 This is a structural diagram of two second cameras installed in a classroom provided by an embodiment of the present application;

[0023] Figure 8 This is a structural diagram of a device screen display apparatus provided in an embodiment of the present application;

[0024] Figure 9 This is a structural diagram of a device screen display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0026] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0027] In a more common existing implementation, the teacher's device can obtain the identity information of each student and the device information of the corresponding student XR device, and organize each student's identity information and corresponding device information into a monitoring list. When the teacher wants to view the display screen of any student's XR device, he or she can find the corresponding student's identity information in the monitoring list and click it. The teacher's XR device determines the associated device information based on the clicked identity information, and sends a request to the corresponding student's XR device based on the device information. After receiving the request, the student's XR device sends the corresponding display screen to the teacher's device, and the teacher's device displays the received display screen, thereby enabling the teacher to monitor the display screen of the student's XR device. Among them, the teacher's device can be an interactive tablet used in the classroom or an XR device used by the teacher, the student's XR device is the XR device used by the student, and the display screen is the screen displayed by the XR device. However, in the classroom, teachers do not always have free time to manually input the operation of viewing the display screen of the student's XR device, and the teacher's manual input of the operation of viewing the display screen may interrupt the teacher's teaching ideas and affect the teaching effect.

[0028] To solve the above problem, this embodiment provides a device screen display method to identify the student XR devices used by students in the target area, and display the display screen of the student XR device on the display screen of the teacher's device, so that the teacher can view the display screen of the student XR device without feeling, avoiding interruption of teaching ideas when the teacher views the student XR device, and ensuring the teaching effect in the classroom.

[0029] The device screen display method provided in this embodiment can be executed by a device screen display device, which can be implemented by software and / or hardware. The device screen display device can be composed of two or more physical entities, or it can be composed of one physical entity. In this embodiment, the device screen display device can be a teacher's device, such as an interactive tablet or an XR device used by a teacher. In addition, the device screen display device can also be a server, which connects the teacher's device and multiple student XR devices. The XR device includes glasses and a camera. The camera is used to capture images of the real world, and the glasses are used to demonstrate virtual classroom content. The display screen of the XR device is the screen displayed by the glasses. The device screen display device can also be a virtual classroom system composed of a server, a teacher's device and multiple student XR devices.

[0030] The device screen display device is installed with at least one operating system, including but not limited to Android, Linux, and Windows. The device screen display device can install at least one application based on the operating system. The application can be a native application of the operating system or an application downloaded from a third-party device or server. In this embodiment, the device screen display device has at least one application capable of executing the device screen display method.

[0031] For ease of understanding, this embodiment is described by taking a teacher device as an example of the main body executing the device screen display method.

[0032] Figure 1 This is a flow chart of a device screen display method provided by an embodiment of the present application. Figure 1 As shown, the steps of the device screen display method include:

[0033] S110 : Identify one or more second XR devices within the target area.

[0034] S120: Acquire a second display image of a second XR device in the target area.

[0035] S130: Display the second display screen within the first display screen, wherein the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

[0036] In this embodiment, the target area can be understood as the area detectable by the detection device of the teacher's device. For example, when the teacher's device is an interactive tablet, the detection device can be a rotatable camera installed on the interactive tablet or a rotatable camera installed in the classroom but communicatively connected to the interactive tablet. The imaging area of ​​the rotatable camera is the target area. The rotatable camera rotates at a preset frequency and captures an image of the target area. Based on this image, the interactive tablet identifies one or more second XR devices located within the target area. The second XR device is the XR device used by the student.

[0037] When the teacher device is a first XR device, the detection device can be a first camera or a radio frequency identifier installed in the first XR device, and the target area is the identification area of ​​the first XR device, specifically the shooting area of ​​the first camera or the signal radiation range of the radio frequency identifier. The first camera can capture an image of the target area, and the first XR device can identify one or more second XR devices located in the target area based on the image. The radio frequency identifier can transmit a radio frequency signal in the target area and receive a radio frequency signal returned by one or more second XR devices in the target area. The one or more second XR devices in the target area can be determined based on the identifier carried by the received radio frequency signal.

[0038] After the teacher's device identifies the second XR device in the target area, it can obtain the second display screen of the second XR device and the student information using the second XR device, and associate the second display screen with the corresponding student information and display it on the first display screen, so that the teacher can directly view the second display screen of the students in the target area, realizing that the teacher can seamlessly view the display screen of the student's XR device. The first display screen is the display screen of the teacher's device, that is, when the teacher's device is an interactive tablet, the first display screen is the screen displayed on the interactive tablet; when the teacher's device is the first XR device, the first display screen is the screen displayed by the glasses of the first XR device.

[0039] For ease of understanding, this embodiment is described by taking the teacher device as the first XR device and the detection device of the first XR device as the first camera as an example. Figure 2 This is a flow chart of another device screen display method provided by an embodiment of the present application. Figure 2 As shown, the device screen display method includes S210-S230:

[0040] S210: Acquire a first image captured by a first camera, and determine first position information of a shooting area of ​​the first camera according to the first image.

[0041] Among them, the first camera is a camera configured for the first XR device to capture real-life images. The shooting angle of the first camera is toward the front of the wearer (that is, the teacher using the first XR device), and the first image is the image captured by the first camera in the classroom. After the class starts, the first XR device used by the teacher captures the image in front through the first camera to generate a first image. The first XR device displays the first image through the first glasses (that is, the glasses configured for displaying virtual classroom content by the first XR device), and the teacher can see the image in front of him or her through the first glasses. The first XR device can determine the first position information of the shooting area of ​​the first camera based on the first image.

[0042] In this embodiment, the first position information can be the three-dimensional coordinates of the shooting area in the real world (i.e., the three-dimensional coordinates in the real-world coordinate system), or the three-dimensional coordinates in a pre-built virtual classroom (i.e., the three-dimensional coordinates in the virtual classroom coordinate system). In one embodiment, the first XR device can determine the three-dimensional coordinates of the first camera in the real world based on its own positioning information, and determine the relative rotation matrix between the calibration image and the first image based on the calibration image captured by the first camera at the corresponding three-dimensional coordinates. Based on the relative rotation matrix and the attitude angle of the first camera when capturing the calibration image, the attitude angle of the first camera when capturing the first image is determined. Based on the attitude angle and three-dimensional coordinates of the first camera, the three-dimensional coordinates of the shooting area of ​​the first camera in the real world are determined.

[0043] In another embodiment, Figure 3 This is a flow chart of determining the three-dimensional coordinates of the shooting area in the virtual classroom provided by the embodiment of the present application. Figure 3 As shown, the step of determining the three-dimensional coordinates of the shooting area in the virtual classroom specifically includes S2101-S2102:

[0044] S2101: Match the first image with a pre-built virtual classroom to determine a shooting position of a first camera.

[0045] Among them, the shooting posture includes the posture angle and three-dimensional coordinates of the first camera in the virtual classroom when shooting the first image. The virtual classroom is a virtual three-dimensional space generated by three-dimensional reconstruction based on images generated by shooting the classroom from different angles. Exemplarily, the first XR device can determine the three-dimensional coordinates of the first camera in the real world based on its own positioning information, and determine the three-dimensional coordinates of the first camera in the virtual classroom based on the coordinate system conversion matrix between the world coordinate system and the virtual classroom coordinate system. Based on the three-dimensional coordinates of the first camera in the virtual classroom and the various candidate posture angles of the first camera in the virtual classroom, the virtual classroom is projected into multiple two-dimensional projection images, the first image is matched with each two-dimensional projection image in turn, and the posture angle corresponding to the two-dimensional projection image with the highest matching degree is determined as the posture angle of the first camera in the virtual classroom.

[0046] In another embodiment, two-dimensional feature data of the first image and three-dimensional feature data of the virtual classroom are extracted, the two-dimensional feature data and the three-dimensional feature data are aligned using a pre-trained alignment model, the spatial transformation between the first image and the virtual classroom is determined, and the shooting posture of the first camera in the virtual classroom is determined based on the spatial transformation.

[0047] S2102: Determine first position information of a shooting area of ​​the first camera in the virtual classroom according to the shooting posture of the first camera.

[0048] Exemplarily, the first XR device determines the three-dimensional coordinates of the first camera's shooting area in the virtual classroom based on the posture angle and three-dimensional coordinates of the first camera in the virtual classroom when the first image was captured, as well as the relative positional relationship between the pre-set shooting posture of the first camera and the corresponding shooting posture. This embodiment quickly determines the three-dimensional coordinates of the first camera's shooting area in the virtual classroom by matching the first image with a pre-built virtual classroom, which helps improve the efficiency of the first XR device's subsequent display of the corresponding student's second display image.

[0049] It should be noted that when the first camera captures the first image, the image of the shooting area is the image seen by the first XR device in the first image. Therefore, the shooting area includes the student in front of the teacher. This embodiment aims to determine the student in front of the teacher based on the first position information of the shooting area, and transmit the second display image of the second XR device used by the student to the first XR device, so that the teacher can view the second display image of the second XR device currently used by the student in front of him / her through the first XR device. Accordingly, when the teacher wants to view the second display image of a student's second XR device, he / she only needs to walk in front of the student with the help of the first camera, and the first XR device can obtain the second display image transmitted by the student's second XR device, so that the teacher can view the second display image of the student's XR device without feeling.

[0050] S220: Determine the second XR device located in the shooting area as a target device based on the first location information and the second location information of each second XR device.

[0051] In this embodiment, if the currently determined first location information is the 3D coordinates of the shooting area in the real world, then the second location information is the 3D coordinates of the corresponding second XR device in the real world. Similarly, if the currently determined first location information is the 3D coordinates in the virtual classroom, then the second location information is the 3D coordinates of the corresponding second XR device in the virtual classroom.

[0052] For example, the first XR device can obtain the positioning information uploaded by each second XR device, determine the three-dimensional coordinates of the corresponding second XR device in the real world based on the positioning information, match the three-dimensional coordinates of the shooting area in the real world with the three-dimensional coordinates of each second XR device in the real world, and determine the second XR device located in the shooting area. Alternatively, before the virtual class begins, after the virtual classroom is constructed, the first XR device determines the three-dimensional coordinates of each student in the virtual classroom, and sets the three-dimensional coordinates of the student in the virtual classroom to the three-dimensional coordinates of the corresponding second XR device in the virtual classroom. The first XR device matches the three-dimensional coordinates of the shooting area in the virtual classroom with the three-dimensional coordinates of each second XR device in the virtual classroom to determine the second XR device located in the shooting area.

[0053] In this embodiment, if the second location information of the second XR device overlaps with the first location information, the second XR device is determined as the target device. It can be understood that the second location information of the second XR device is a coordinate range, and the first location information is also a coordinate range. When the two coordinate ranges overlap, it can be determined that the student using the second XR device is within the capture area. In this case, the second XR device can be determined as the target device for sending the second display image to the first XR device. Figure 3Schematic diagram of the target device provided in the embodiment of the present application. Figure 3 As shown, when teacher A, wearing the first XR device 10, walks in front of students C, B, and D, the first XR device determines that the second position information of the second XR device 20 worn by students B and C overlaps with the first position information of the capture area 12 of the first camera 11, and then determines that the second XR devices worn by students B and C are target devices. This embodiment uses the second XR device of a student who may be at the edge of the capture area as the target device, eliminating the need for the teacher to adjust the first camera to fully capture the student. This further simplifies the teacher's operation of viewing the student's second display screen and indirectly increases the number of second display screens the teacher can view.

[0054] S230: Send a screen request instruction to the target device, so that the target device sends the corresponding second display screen to the first XR device for display when receiving the screen request instruction.

[0055] The screen request instruction is a request instruction used by the first XR device to request the second XR device to transmit its second display screen to the first XR device. After the first XR device determines the second XR device as the target device based on the first image, it sends a screen request instruction to the target device, so that the target device sends the corresponding second display screen to the first XR device, and the first XR device displays the received second display screen. The second XR device that is not the target device will not receive the screen request instruction sent by the first XR device, nor will it send the corresponding second display screen to the first XR device, allowing the teacher to view the second display screen of the second XR device used by the student in front of him.

[0056] In this embodiment, the second XR device sends the screen displayed by the second glasses (i.e., the glasses configured by the second XR device to display virtual classroom content) as the second display screen to the first XR device. After the first XR device receives the second display screen sent by the second XR device, it displays the received second display screen on the first display screen through the corresponding configured first glasses. When the first XR device receives multiple second display screens, the received second display screens can be displayed separately through multiple screen windows. Figure 5 Schematic diagram of the first display screen of the first XR device provided in the embodiment of the present application. Figure 5 As shown, when receiving the second display screen A, the second display screen B and the second display screen C respectively sent by the three second XR devices, the first XR device creates three screen windows 14 in the first display screen 13 of the first glasses, and displays the second display screen A, the second display screen B and the second display screen C in the three screen windows 14, so that the teacher can view the second display screens of the second XR devices used by multiple students at the same time.

[0057] Furthermore, when it is detected that the target device has moved out of the shooting area, a stop screen instruction is sent to the target device, so that the target device stops sending the second display screen to the first XR device when receiving the stop screen instruction, and the first XR device closes the window displaying the second display screen sent by the target device. The stop screen instruction is an instruction used by the first XR device to request the second XR device to stop sending the second display screen to the first XR device. Exemplarily, the first XR device determines the first position information of the shooting area of ​​the first camera based on the first image. If the first position information does not overlap with the second position information of the current target device, it is determined that the target device has moved out of the shooting area. The first XR device sends a stop screen instruction to the target device that has moved out of the shooting area, and the target device that receives the stop screen instruction no longer sends the second display screen to the first XR device. When the first XR device does not receive the second display screen sent by the second XR device, the window used to display the second display screen in the first display screen is closed to avoid interfering with the teacher's viewing of the second display screens corresponding to other students. This embodiment sends a screen stop command to the target device that moves out of the shooting area, so as to promptly stop the first XR device from playing the second display screen corresponding to the student who is not in front of the teacher, thereby ensuring that the teacher can accurately view the second display screen of the second XR device used by the student in front of him / her.

[0058] Based on the above embodiments, Figure 6 This is a flow chart of another device screen display method provided by an embodiment of the present application. Figure 6 As shown, the device screen display method includes:

[0059] S310: Before opening the virtual classroom, obtain a second image captured by a second camera installed in the classroom, and construct a virtual classroom based on the second image.

[0060] In this embodiment, one or more second cameras may be installed in the classroom. When one second camera is installed in the classroom, the first XR device may control the second camera to rotate to capture multiple second images of the classroom, or the second camera may rotate based on a preset rotation method to capture multiple second images of the classroom, and the first XR device may obtain the multiple second images transmitted by the second camera. When multiple second cameras are installed in the classroom, the multiple second cameras are installed at different locations in the classroom to capture a single second image of the classroom, and the first XR device may obtain the multiple second images transmitted by the multiple second cameras. Figure 7 This is a structural diagram of a classroom provided by an embodiment of the present application with two second cameras installed. Figure 7As shown, a second camera 31 is installed in the upper left corner of the classroom 30, and another second camera 31 is installed in the upper right corner of the classroom 30. The shooting areas 32 of the two second cameras 31 almost cover the entire classroom. Therefore, a virtual classroom can be constructed based on the second images taken by the two second cameras 31 through a three-dimensional reconstruction algorithm.

[0061] In this embodiment, when constructing a virtual classroom, the second image taken by the second camera in the upper left corner can be stereo matched with the second image taken by the second camera in the upper right corner to determine the depth value of each pixel in the second image. Based on the depth value of each pixel and the shooting posture of the second camera, a three-dimensional point cloud corresponding to each pixel can be constructed, and a virtual classroom can be constructed based on the three-dimensional point cloud.

[0062] S320. Perform facial recognition on the second image to determine the identity information of each student and the third position information in the virtual classroom, and associate and save the identity information and the third position information of each student; or, determine the identity information and the third position information of each student in the virtual classroom based on the student seat distribution map, and associate and save the identity information and the third position information of each student.

[0063] In one embodiment, when the second camera captures the second image, the students are already seated in the classroom. Therefore, the second image contains facial information for each student. A pre-trained face detection model can be used to detect the area containing the face in the second image, and the first facial image can be captured from this area. The captured first facial image is matched with the second facial image in a facial database. The facial database stores the second facial image of each student, as well as the identity information of the student associated with the second facial image. The identity information corresponding to the second facial image that matches the first facial image is determined as the identity information of the student corresponding to the first facial image. Based on the pixel coordinates of the first facial image in the second image and the three-dimensional point cloud used to generate the virtual classroom, the third position information of the student corresponding to the first facial image in the virtual classroom (i.e., the three-dimensional coordinates of the student in the virtual classroom) can be determined. The identity information of the same student is associated with the third position information and saved so that the third position information of the corresponding student in the virtual classroom can be subsequently obtained based on the student's identity information.

[0064] In another embodiment, the student seat distribution map can represent the relative position relationship between each student and the classroom. The first XR device can determine the identity information of the students sitting in each seat in the classroom through the student seat distribution map, and based on the relative position relationship between the student's seat and the classroom, the third position information of the student in the virtual classroom can be determined, and the identity information of the same student and the third position information are associated and saved.

[0065] S330: Receive identity information and device information sent by the second XR device.

[0066] For example, after the student puts on the second XR device, he logs in to the second XR device using his own identity information and joins the virtual classroom. When the second XR device joins the virtual classroom, the identity information of the corresponding logged-in student and his own device information are sent to the first XR device, so that the first XR device can identify the second XR device used by each student based on the device information and identity information.

[0067] S340: Obtain third location information of the corresponding student in the virtual classroom according to the identity information, and determine the third location information as the second location information of the second XR device corresponding to the device information.

[0068] Exemplarily, after the first XR device receives the identity information and device information sent by the second XR device, it queries the pre-associated and saved third location information based on the identity information sent by the second XR device, and sets the third location information to the second location information corresponding to the second XR device based on the device information sent by the second XR device, so that in the virtual classroom, the first XR device can determine the second XR device located in the shooting area of ​​the first camera based on the second location information of the second XR device.

[0069] S350: After the virtual classroom is opened, a first image captured by a first camera is obtained, and first position information of a shooting area of ​​the first camera is determined according to the first image.

[0070] Exemplarily, after starting the virtual classroom, the first XR device turns on the first camera, and the first camera captures a first image. The first XR device matches the first image with the pre-built virtual classroom, determines the shooting posture of the first camera in the virtual classroom, and determines the first position information of the shooting area in the virtual classroom based on the shooting posture of the first camera and the relative position relationship between the shooting posture and the shooting area.

[0071] S360: Determine the second XR device located in the shooting area as a target device based on the first location information and the second location information of each second XR device.

[0072] Exemplarily, the second position information of each second XR device pre-set in the virtual classroom before the virtual classroom is opened is obtained, the first position information is compared with the second position information, and the second XR device corresponding to the second position information overlapping with the first position information is determined as the target device.

[0073] S370: Send a screen request instruction to the target device, so that the target device sends the corresponding second display screen to the first XR device for display when receiving the screen request instruction.

[0074] For example, a screen request instruction is sent to the target device based on the device information of the target device. In response to the screen request instruction, the target device sends the second display screen of the second glasses to the first XR device. The first XR device associates the received second display screen with the student information corresponding to the second XR device and displays it on the first display screen of the first glasses.

[0075] In summary, the device screen display method provided in the embodiment of the present application identifies one or more second XR devices in the target area; obtains the second display screen of the second XR device in the target area; displays the second display screen in the first display screen, and the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device. Through the above technical means, the second XR device used by the student in the target area can be identified, and then the second display screen of the second XR device can be obtained. The obtained second display screen is associated with the corresponding second XR device and displayed in the first display screen viewed by the teacher, so that the teacher can seamlessly view the second display screen of the second XR device used by the student sitting in front of him / her, solving the problem in the prior art that the teacher needs to manually input operations to view the second display screen of the second XR device used by the student, avoiding the interruption of the teaching train of thought when the teacher views the second XR device used by the student, and ensuring the teaching effect in the classroom.

[0076] Based on the above embodiments, Figure 8 This is a schematic diagram of the structure of a device screen display device provided in an embodiment of the present application. Figure 8 The device screen display apparatus provided in this embodiment specifically includes: a device identification module 41 , a screen acquisition module 42 and a screen display module 43 .

[0077] The device identification module 41 is configured to identify one or more second XR devices in the target area;

[0078] The image acquisition module 42 is configured to acquire a second display image of a second XR device in the target area;

[0079] The screen display module 43 is configured to display the second display screen within the first display screen, where the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

[0080] Based on the above embodiment, the first device screen is displayed on the first XR device, and the target area is the recognition area of ​​the first XR device.

[0081] On the basis of the above embodiment, the first XR device has a first camera, and the target area is the shooting area of ​​the first camera of the first XR device. The device screen display device also includes: a first position determination module, configured to obtain a first image captured by the first camera of the first XR device, and determine the first position information of the shooting area of ​​the first camera according to the first image; a target device determination module, configured to determine the second XR device located in the shooting area as the target device according to the first position information and the second position information of each second XR device; a device screen display module, configured to send a screen request instruction to the target device, so that when the target device receives the screen request instruction, it sends the corresponding second display screen to the first XR device for display.

[0082] Based on the above embodiment, the device screen display device also includes a second position determination module, which includes: an information receiving unit, configured to receive identity information and device information sent by the second XR device before opening the virtual classroom; a second position determination unit, configured to obtain the third position information of the corresponding student in the virtual classroom based on the identity information, and determine the third position information as the second position information of the second XR device corresponding to the device information.

[0083] Based on the above embodiment, the second position determination unit includes: a first construction subunit, configured to obtain a second image taken by a second camera installed in the classroom before obtaining the third position information of the corresponding student in the virtual classroom based on the identity information, and construct a virtual classroom based on the second image; a first information storage subunit, configured to perform face recognition on the second image, determine the identity information of each student and the third position information in the virtual classroom, and store the identity information and third position information of each student in association.

[0084] Based on the above embodiment, the second position determination unit includes: a second construction subunit, which is configured to obtain a second image taken by a second camera installed in the classroom before obtaining the third position information of the corresponding student in the virtual classroom based on the identity information, and construct the virtual classroom based on the second image; a second information storage subunit, which is configured to determine the identity information and the third position information of each student in the virtual classroom based on the student seat distribution map, and store the identity information and third position information of each student in association with each other.

[0085] Based on the above embodiment, the first position determination module includes: a shooting posture determination unit, configured to match the first image with a pre-built virtual classroom to determine the shooting posture of the first camera; and a first position determination unit, configured to determine the first position information of the shooting area of ​​the first camera in the virtual classroom based on the shooting posture of the first camera.

[0086] Based on the above embodiment, the device screen display device also includes: a screen stop display module, which is configured to send a screen request instruction to the target device, and then send a screen stop instruction to the target device when it is detected that the target device moves out of the shooting area, so that the target device stops sending the second display screen to the first XR device when receiving the screen stop instruction, and the first XR device closes the window displaying the second display screen sent by the target device.

[0087] As described above, the device screen display device provided in the embodiment of the present application identifies one or more second XR devices in the target area; obtains the second display screen of the second XR device in the target area; and displays the second display screen in the first display screen, wherein the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device. Through the above technical means, the second XR device used by the student in the target area can be identified, and then the second display screen of the second XR device can be obtained. The obtained second display screen is associated with the corresponding second XR device and displayed in the first display screen viewed by the teacher, so that the teacher can seamlessly view the second display screen of the second XR device used by the student sitting in front of him / her, solving the problem in the prior art that the teacher needs to manually input operations to view the second display screen of the second XR device used by the student, avoiding the interruption of the teaching train of thought when the teacher views the second XR device used by the student, and ensuring the teaching effect in the classroom.

[0088] The device screen display apparatus provided in the embodiment of the present application can be used to execute the device screen display method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0089] Figure 9 This is a schematic diagram of the structure of a device screen display device provided in an embodiment of the present application, with reference to Figure 9 The device screen display device includes: a processor 51, a memory 52, a communication device 53, an input device 54, and an output device 55. The number of processors 51 in the device screen display device can be one or more, and the number of memories 52 in the device screen display device can be one or more. The processor 51, memory 52, communication device 53, input device 54, and output device 55 of the device screen display device can be connected via a bus or other means.

[0090] The memory 52, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules, such as program instructions / modules corresponding to the device screen display method of any embodiment of the present application (for example, the device identification 41, the screen acquisition module 42, and the screen display module 43 in the device screen display device). The memory 52 may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 52 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0091] The communication device 53 is used for data transmission.

[0092] The processor 51 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 52, that is, implements the above-mentioned device screen display method.

[0093] The input device 54 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 55 may include a display device such as a display screen.

[0094] The device screen display device provided above can be used to execute the device screen display method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0095] An embodiment of the present application also provides a storage medium containing computer-executable instructions. When the computer-executable instructions are executed by a computer processor, they are used to perform a device screen display method. The device screen display method includes: identifying one or more second XR devices in a target area; obtaining a second display screen of the second XR device in the target area; displaying the second display screen in a first display screen, wherein the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

[0096] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROMs, floppy disks, or tape drives; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or it may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0097] Of course, the storage medium containing computer-executable instructions provided in the embodiment of the present application is not limited to the above device screen display method, and the computer-executable instructions can also execute related operations in the device screen display method provided in any embodiment of the present application.

[0098] The device screen display apparatus, storage medium, and device screen display device provided in the above embodiments can execute the device screen display method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the device screen display method provided in any embodiment of the present application.

[0099] The above are only preferred embodiments of the present application and the technical principles employed. The present application is not limited to the specific embodiments described herein, and any obvious changes, readjustments, and substitutions that are apparent to those skilled in the art will not depart from the scope of protection of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the claims.

Claims

1. A device screen display method, characterized in that: include: identifying one or more second XR devices within the target area; Acquire a second display image of a second XR device in the target area; The second display screen is displayed within the first display screen, the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

2. The device screen display method according to claim 1, characterized in that: The first device screen is displayed on the first XR device, and the target area is the recognition area of ​​the first XR device.

3. The device screen display method according to claim 2, characterized in that: The first XR device has a first camera, the target area is a shooting area of ​​the first camera of the first XR device, and the method includes: Acquire a first image captured by the first camera, and determine first position information of a shooting area of ​​the first camera according to the first image; determining, according to the first position information and the second position information of each of the second XR devices, a second XR device located in the shooting area as a target device; Send a picture request instruction to the target device, so that the target device sends the corresponding second display picture to the first XR device for display when receiving the picture request instruction.

4. The device screen display method according to claim 3, characterized in that: The method is applied to a virtual classroom, and specifically further comprises: Before starting the virtual classroom, receiving identity information and device information sent by the second XR device; Acquire third location information of the corresponding student in the virtual classroom according to the identity information, and determine the third location information as the second location information of the second XR device corresponding to the device information.

5. The device screen display method according to claim 4, characterized in that: Before acquiring the third position information of the corresponding student in the virtual classroom according to the identity information, the method further includes: Acquire a second image captured by a second camera installed in the classroom, and construct a virtual classroom based on the second image; Performing face recognition on the second image to determine the identity information of each student and the third position information in the virtual classroom, and then associating the identity information of each student with the third position information and saving them.

6. The device screen display method according to claim 4, characterized in that: Before acquiring the third position information of the corresponding student in the virtual classroom according to the identity information, the method further includes: Acquire a second image captured by a second camera installed in the classroom, and construct a virtual classroom based on the second image; The identity information and the third position information of each student in the virtual classroom are determined according to the student seat distribution map, and the identity information and the third position information of each student are associated and saved.

7. The device screen display method according to claim 3, characterized in that: The determining, based on the first image, first position information of a shooting area of ​​the first camera includes: Matching the first image with a pre-built virtual classroom to determine a shooting posture of the first camera; According to the shooting posture of the first camera, first position information of the shooting area of ​​the first camera in the virtual classroom is determined.

8. The device screen display method according to claim 3, characterized in that: After sending the screen request instruction to the target device, the method further includes: When it is detected that the target device moves out of the shooting area, a picture stop instruction is sent to the target device, so that the target device stops sending the corresponding second display picture to the first XR device when receiving the picture stop instruction, and the first XR device closes the window displaying the second display picture of the target device.

9. A device screen display device, characterized in that: The device comprises: a device identification module configured to identify one or more second XR devices within the target area; A picture acquisition module is configured to acquire a second display picture of a second XR device in the target area; The screen display module is configured to display the second display screen within the first display screen, the first display screen also displays the second XR device, and the second display screen is displayed in association with the second XR device.

10. A device screen display device, characterized in that: include: one or more processors; A memory stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the device screen display method according to any one of claims 1 to 8.

Images