Plane control adaptation system suitable for XR shed and method thereof

Abstract

The invention provides a plane control adaptation system suitable for an XR shed and a method thereof, and relates to the technical field of equipment control. According to the method, a cursor is deconstructed into a foreground cursor and a background cursor, the foreground cursor is used for direct display and interaction with foreground software, and position information and operation information of the background cursor are obtained in real time by capturing a moving track of the background cursor in a two-dimensional operation space; determining a two-dimensional subspace where a background cursor is located according to the background cursor position information, and determining a display module corresponding to the two-dimensional subspace as a current controlled module, thereby determining an operation object and an operation range; mapping the moving track and the operation information of the background cursor to the foreground cursor in real time to obtain corresponding operation control and act on a desktop block corresponding to the current controlled module; and the operating system performs operation control on the foreground desktop through the foreground cursor, configures the software state of the corresponding desktop block, and further generates the display content of each display module according to the software state through foreground software.

Description

Technical Field

[0001] The present invention relates to the technical field of device control, and particularly to a planar control adaptation system and method applicable to an XR booth. Background Art

[0002] An XR booth is a new type of content display device, mainly composed of five different planes. Its appearance resembles a box with one side opened. The content displayed on each plane is interrelated, thus achieving an immersive visual effect. Viewers can get an immersive viewing experience when standing at the opening.

[0003] Due to the special design of the XR booth, the content it displays is also different from traditional single-plane images. The server side needs to project the displayed content onto each plane respectively. However, the existing server-side operating systems (such as the Windows Server operating system) are mostly designed based on traditional single-plane images, which results in the inability of the original desktop operations to be compatible and adapted to the planar control of the XR booth. Especially for desktop controls such as mice, problems such as inconsistent movement with spatial relationships will occur during actual use.

[0004] In addition, due to its excellent performance, the XR booth has great potential in interactive entertainment. Many existing interactive devices are based on simulated mice (such as motion guns, etc.). However, the movement of desktop controls does not conform to spatial relationships, which leads to the inability of simulated mouse-based interactive devices to be compatible with the actual use of the XR booth. Therefore, this problem also restricts the development of the XR booth in interactive entertainment.

[0005] Therefore, it is necessary to provide a planar control adaptation system and method applicable to an XR booth to solve the technical problem that the existing desktop operating systems on the server side cannot be compatible and adapted to the planar control of the XR booth. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a planar control adaptation system and method applicable to an XR booth, aiming to achieve the compatibility and adaptation of the server's operating system to the planar control of the XR booth through a special system for the XR booth, thereby removing the development restrictions of the XR booth in interactive entertainment caused by incompatibility.

[0007] To achieve the above purpose, the present application proposes a planar control adaptation system applicable to an XR booth, including: An XR booth, configured to display display content through a corresponding display module; A server, which is deployed with an operating system and runs a foreground desktop and foreground software; wherein, the operating system controls the foreground desktop through a foreground cursor; the foreground desktop divides the desktop into blocks corresponding to each display module, and configures the software status of the corresponding desktop block according to the operation control; the foreground software generates the display content of each display module according to the software status; A two-dimensional operation space component, which runs in the operating system in the form of a background and provides a two-dimensional operation space; wherein, the two-dimensional operation space consists of a two-dimensional subspace corresponding to the display module and a background cursor; A cursor control device, which is used to obtain the operation control behavior of the user, convert it into a cursor control signal and input it into the operating system; wherein, the background cursor is controlled by the cursor control signal; A background cursor capture unit, which is used to capture the movement trajectory of the background cursor in the two-dimensional operation space, and obtain the background cursor position information and operation information in real time; A controlled module acquisition unit, which determines the two-dimensional subspace where the background cursor is located according to the background cursor position information, and determines the display module corresponding to the two-dimensional subspace as the current controlled module; A cursor operation mapping unit, which is used to map the movement trajectory and operation information of the background cursor to the foreground cursor in real time, obtain the corresponding operation control and act on the desktop block corresponding to the current controlled module.

[0008] As a further solution, the display module includes an upper module, a middle module, a lower module, a left module and a right module; the desktop block includes an upper block, a middle block, a lower block, a left block and a right block; the two-dimensional subspace includes an upper subspace, a middle subspace, a lower subspace, a left subspace and a right subspace.

[0009] As a further solution, the two-dimensional operation space stitches each subspace into a whole rectangular operation space; wherein, the left subspace is set on the left side of the rectangular operation space, the upper subspace, the middle subspace and the lower subspace are set from top to bottom in the middle part, and the right subspace is set on the right side; the boundary lines of each subspace correspond to the inner boundary lines between each display module.

[0010] As a further solution, the rectangular operation space maps the positions of each two-dimensional subspace to the corresponding desktop block by setting a corresponding position transformation matrix, and the desktop block determines the display range and the position of the display pixel points according to the resolution information of the corresponding display module.

[0011] As a further solution, an operation space selection unit is also provided, and the operation space selection unit is used to select the two-dimensional subspace to be used.

[0012] As a further solution, a cursor parameter adaptation unit is also provided; wherein, the cursor parameter adaptation unit determines the DPI parameter relationship between each two-dimensional subspace and the desktop block according to the position transformation matrix, converts the DPI parameter of the background cursor into the DPI parameter of the foreground cursor according to the DPI parameter relationship, and performs real-time adaptation and use according to the desktop block corresponding to the currently controlled module.

[0013] As a further solution, an interaction device adaptation unit is also provided, and the interaction device adaptation unit is used to adapt an interaction device implemented based on an analog mouse; wherein, the operation control behavior is bound to the control signal of the interaction device, including left-click information, right-click information, scroll wheel rolling information, custom button information, and custom operation information.

[0014] On the other hand, the present invention also provides a planar control adaptation method applicable to an XR shed, which is applied to a planar control adaptation system applicable to an XR shed as described in any one of the above, and includes the following steps: Step 1: Obtain the operation control behavior of the user through a cursor control device, convert it into a cursor control signal and input it into the operating system; wherein, the background cursor is controlled by the cursor control signal; Step 2: The background cursor capture unit captures the movement trajectory of the background cursor in the two-dimensional operation space, and obtains the background cursor position information and operation information in real time; Step 3: The controlled module acquisition unit determines the two-dimensional subspace where the background cursor is located according to the background cursor position information, and determines the display module corresponding to the two-dimensional subspace as the currently controlled module; Step 4: The cursor operation mapping unit maps the movement trajectory and operation information of the background cursor to the foreground cursor in real time, obtains the corresponding operation control and acts on the desktop block corresponding to the currently controlled module; Step 5: The operating system operates and controls the foreground desktop through the foreground cursor; Step 6: Perform virtual scene simulation on the XR shed and the viewing area through the virtual scene simulation unit; Step 7: The foreground desktop divides desktop blocks corresponding to each display module, and configures the software status of the corresponding desktop block according to the operation control; Step 8: The foreground software generates the display content of each display module according to the software status; Step 9: The XR shed displays the display content through the display module; Step 10: Repeat steps 1 to 9 to achieve planar control adaptation of the XR shed until the adaptation process ends.

[0015] Compared with the related technology, a planar control adaptation system and method applicable to an XR shed provided by the present invention have the following advantages: The present invention deconstructs the cursor into a foreground cursor and a background cursor. The foreground cursor is used for direct display and interaction with foreground software. By capturing the movement trajectory of the background cursor in the two-dimensional operation space, the position information and operation information of the background cursor are obtained in real time. Then, according to the position information of the background cursor, the two-dimensional subspace where the background cursor is located is determined, and the display module corresponding to the two-dimensional subspace is determined as the currently controlled module, thereby determining our operation object and operation range. Then, the movement trajectory and operation information of the background cursor are mapped to the foreground cursor in real time, and the corresponding operation control is obtained and applied to the desktop block corresponding to the currently controlled module. At this time, the operating system can operate and control the foreground desktop through the foreground cursor, configure the software state of the corresponding desktop block according to the operation control, and then generate the display content of each display module through the foreground software according to the software state. Brief Description of the Drawings

[0016] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the accompanying drawings required for use in the description of the embodiments or related technologies. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 Schematic structural diagram of a planar control adaptation system applicable to an XR shed provided by the present invention; Figure 2 Schematic diagram of the adaptation failure of the existing operating system to the XR shed; Figure 3 Schematic diagram of the two-dimensional operation space provided by the present invention; Figure 4 Schematic diagram of the steps of a planar control adaptation method applicable to an XR shed provided by the present invention.

[0019] The realization of the purpose of the present application, functional features, and advantages will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed Embodiments

[0020] To make the purpose, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the accompanying drawings herein can be arranged and designed in various different configurations.

[0021] Example 1 Please refer to Figure 1 , the embodiment of the present application provides a planar control adaptation system applicable to an XR shed, including: An XR shed for displaying display content through a corresponding display module; A server, deployed with an operating system and running a foreground desktop and foreground software; wherein, the operating system operates and controls the foreground desktop through a foreground cursor; the foreground desktop divides desktop blocks corresponding to each display module and configures the software state of the corresponding desktop block according to the operation control; the foreground software generates the display content of each display module according to the software state; A two-dimensional operation space component, running in the operating system in the form of a background and providing a two-dimensional operation space; wherein, the two-dimensional operation space consists of a two-dimensional subspace corresponding to the display module and a background cursor; A cursor control device for obtaining the operation control behavior of the user, converting it into a cursor control signal and inputting it into the operating system; wherein, the background cursor is controlled by the cursor control signal; A background cursor capture unit for capturing the movement trajectory of the background cursor in the two-dimensional operation space and real-time obtaining the background cursor position information and operation information; A controlled module acquisition unit for determining the two-dimensional subspace where the background cursor is located according to the background cursor position information and determining the display module corresponding to the two-dimensional subspace as the current controlled module; A cursor operation mapping unit for mapping the movement trajectory and operation information of the background cursor to the foreground cursor in real time, obtaining the corresponding operation control and acting on the desktop block corresponding to the current controlled module.

[0022] It should be noted that: Although the existing operating systems support multi-screen display, they connect the screens of each part as independent screens in series. Although we can select the display positions of each screen in the settings option; however, due to the XR shed having a certain spatial structure, the actual usage requirements cannot be met only by the screen position editor provided by the system.

[0023] As Figure 2 shown, when facing multi-screen display, it is often arranged in a side-by-side or overlapping manner. However, there are different types of boundary lines at the boundaries of the actual XR shed. For example, when the screens on the left and right are moved up, they can both reach the upper plane, only one enters from the left and the other enters from the right; however, when using the built-in screen position editor, this spatial position relationship cannot be reflected, as Figure 2Screen No. 3 is on the left, Screen No. 4 is in the middle, Screen No. 6 is on the right, and Screen No. 5 is on the upper side; we cannot reach Screen No. 5 by moving the mouse on Screens No. 3 and No. 6, and only normal horizontal movement and switching can be achieved; in fact, in the XR studio, Screen No. 5 is above Screens No. 3, No. 4, and No. 6.

[0024] Therefore, in this embodiment, the cursor is deconstructed into a foreground cursor and a background cursor. The foreground cursor is used for direct display and interaction with foreground software, while the background cursor directly receives cursor control signals and is restricted by a two-dimensional operation space. Among them, the two-dimensional operation space component provides a two-dimensional operation space that conforms to the spatial relationship for the background cursor. The two-dimensional subspace within this space is correspondingly set for the display module, so the interaction of the background cursor conforms to the spatial relationship. However, since the operating system cannot be compatible with the operation logic of the background cursor, the background cursor cannot directly act on the foreground software and still needs to interact with the operating system through the foreground cursor to achieve planar control adaptation.

[0025] Specifically, we capture the movement trajectory of the background cursor in the two-dimensional operation space and obtain the position information and operation information of the background cursor in real time. Then, we determine the two-dimensional subspace where the background cursor is located based on the position information of the background cursor, and determine the display module corresponding to the two-dimensional subspace as the currently controlled module, thereby determining our operation object and operation range. Then, we map the movement trajectory and operation information of the background cursor to the foreground cursor in real time to obtain the corresponding operation control and apply it to the desktop block corresponding to the currently controlled module. At this time, the operating system can operate and control the foreground desktop through the foreground cursor, configure the software state of the corresponding desktop block according to the operation control, and then generate the display content of each display module through the foreground software according to the software state. Among them, the forms of the foreground software are diverse, which can be the desktop mouse pattern, the game control crosshair, or other programs developed based on cursor control.

[0026] The display module includes an upper module, a middle module, a lower module, a left module, and a right module; correspondingly, the desktop block is also provided with an upper block, a middle block, a lower block, a left block, and a right block; similarly, the two-dimensional subspace is also provided with an upper subspace, a middle subspace, a lower subspace, a left subspace, and a right subspace.

[0027] Such as Figure 3As shown in the figure, in order to make the interaction conform to the spatial relationship and human intuitive logic, we piece together each sub-space into a whole rectangular operation space. Among them, the left part of the rectangular operation space is set with a left sub-space, the upper part, middle part, and lower part are set with an upper sub-space, a middle sub-space, and a lower sub-space from top to bottom respectively, and the right part is set with a right sub-space. The boundary lines of each sub-space correspond to the inner boundary lines between each display module. Among them, the black dots represent the desktop mouse, which can move arbitrarily in the rectangular operation space.

[0028] In addition, the shape of the desktop block cannot be regularly arranged and matched to the rectangular operation space. Therefore, we also need to map each position of the two-dimensional sub-space to the corresponding desktop block. The desktop block determines the display range and the position of display pixels according to the resolution information of the corresponding display module, so as to match each operation to the corresponding position and image.

[0029] To meet different display requirements, we also set up an operation space selection unit. The operation space selection unit is used to select the two-dimensional sub-space to be used, and can select the corresponding desktop block for selective adaptation according to the actual usage requirements.

[0030] In addition, since the desktop block and the two-dimensional sub-space are adapted, if the cursor parameters of the front-end cursor and the back-end cursor are not adapted, it will cause the problem of mismatched cursor sensitivity, that is, what we often call the cursor becoming unresponsive. For this reason, we also set up a cursor parameter adaptation unit. Among them, the cursor parameter adaptation unit determines the DPI parameter relationship between each two-dimensional sub-space and the desktop block according to the position transformation matrix, converts the DPI parameter of the back-end cursor into the DPI parameter of the front-end cursor according to the DPI parameter relationship, and performs real-time adaptation and use according to the desktop block corresponding to the currently controlled module.

[0031] Based on the above improvements, we can successfully implement the compatibility adaptation of the server's operating system to the plane control of the XR shed. Therefore, on this basis, we also set up an interaction device adaptation unit to lift the development restrictions of the XR shed in the aspect of interactive entertainment caused by incompatibility. The interaction device adaptation unit is used to adapt the interactive device based on the simulated mouse. Among them, the operation control behavior is bound to the control signal of the interactive device, including left-click information, right-click information, scroll wheel rolling information, custom button information, and custom operation information.

[0032] Embodiment 2 Please refer to Figure 4 , this embodiment further provides a plane control adaptation method applicable to the XR shed on the basis of Embodiment 1, including the following steps: Step 1: Obtain the operation control behavior of the user through the cursor control device, convert it into a cursor control signal and input it into the operating system. Among them, the back-end cursor is controlled by the cursor control signal; Step 2: The background cursor capture unit captures the movement trajectory of the background cursor in the two-dimensional operation space, and obtains the background cursor position information and operation information in real time; Step 3: The controlled module acquisition unit determines the two-dimensional subspace where the background cursor is located according to the background cursor position information, and determines the display module corresponding to the two-dimensional subspace as the current controlled module; Step 4: The cursor operation mapping unit maps the movement trajectory and operation information of the background cursor to the foreground cursor in real time, obtains the corresponding operation control, and applies it to the desktop block corresponding to the current controlled module; Step 5: The operating system controls and operates the foreground desktop through the foreground cursor; Step 6: The virtual scene simulation unit performs virtual scene simulation on the XR shed and the viewing area; Step 7: The foreground desktop divides the desktop blocks corresponding to each display module, and configures the software states of the corresponding desktop blocks according to the operation control; Step 8: The foreground software generates the display content of each display module according to the software state; Step 9: The XR shed displays the display content through the display module; Step 10: Repeat Steps 1 to 9 to achieve the planar control adaptation of the XR shed until the adaptation process ends.

[0033] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A planar control adaptation system applicable to an XR shed, characterized in that, Including: An XR shed for displaying display content through a corresponding display module; A server deployed with an operating system and running a foreground desktop and foreground software; wherein, the operating system operates and controls the foreground desktop through a foreground cursor; the foreground desktop divides desktop blocks corresponding to each display module and configures the software state of the corresponding desktop block according to the operation control; the foreground software generates the display content of each display module according to the software state; A two-dimensional operation space component that runs in the operating system in the form of a background and provides a two-dimensional operation space; wherein, the two-dimensional operation space consists of a two-dimensional subspace corresponding to the display module and a background cursor; A cursor control device for obtaining the operation control behavior of the user, converting it into a cursor control signal and inputting it into the operating system; wherein, the background cursor is controlled by the cursor control signal; A background cursor capture unit for capturing the movement trajectory of the background cursor in the two-dimensional operation space and real-time obtaining the background cursor position information and operation information; A controlled module acquisition unit for determining the two-dimensional subspace where the background cursor is located according to the background cursor position information and determining the display module corresponding to the two-dimensional subspace as the current controlled module; A cursor operation mapping unit for mapping the movement trajectory and operation information of the background cursor to the foreground cursor in real time to obtain the corresponding operation control and act on the desktop block corresponding to the current controlled module.

2. The planar control adaptation system applicable to the XR shed according to claim 1, wherein, The display module includes an upper module, a middle module, a lower module, a left module and a right module; the desktop block includes an upper block, a middle block, a lower block, a left block and a right block; the two-dimensional subspace includes an upper subspace, a middle subspace, a lower subspace, a left subspace and a right subspace.

3. A planar control adaptation system applicable to an XR shed according to claim 2, characterized in that, The two-dimensional operation space stitches each subspace into a whole rectangular operation space; wherein, the left part of the rectangular operation space is provided with a left subspace, the middle part is provided with an upper subspace, a middle subspace and a lower subspace from top to bottom, and the right part is provided with a right subspace; the boundary lines of each subspace correspond to the inner boundary lines between each display module.

4. A planar control adaptation system applicable to an XR shed according to claim 3, characterized in that, The rectangular operation space maps each position of the two-dimensional subspace to the corresponding desktop block by setting a corresponding position transformation matrix, and the desktop block determines the display range and display pixel position according to the resolution information of the corresponding display module.

5. A planar control adaptation system applicable to an XR shed according to claim 2, characterized in that, An operation space selection unit is also provided, and the operation space selection unit is used to select the two-dimensional subspace to be used.

6. The planar control adaptation system applicable to the XR shed according to claim 4, wherein A cursor parameter adaptation unit is also provided; wherein, the cursor parameter adaptation unit determines the DPI parameter relationship between each two-dimensional subspace and the desktop block according to the position transformation matrix, converts the DPI parameter of the background cursor into the DPI parameter of the foreground cursor according to the DPI parameter relationship, and performs real-time adaptation according to the desktop block corresponding to the current controlled module.

7. A planar control adaptation system applicable to an XR shed according to claim 1, characterized in that, An interaction device adaptation unit is also provided, and the interaction device adaptation unit is used to adapt the interaction device based on an analog mouse; wherein, the operation control behavior is bound to the control signal of the interaction device, including left click information, right click information, scroll wheel roll information, custom button information and custom operation information.

8. A planar control adaptation method applicable to an XR shed, which is applied to a planar control adaptation system for an XR shed as described in any one of claims 1 to 7, characterized in that, Including the following steps: Step 1: Obtain the user's operation control behavior through a cursor control device, convert it into a cursor control signal and input it into the operating system; wherein, the background cursor is controlled by the cursor control signal; Step 2: The background cursor capture unit captures the movement trajectory of the background cursor in the two-dimensional operation space, and obtains the background cursor position information and operation information in real time; Step 3: The controlled module acquisition unit determines the two-dimensional subspace where the background cursor is located according to the background cursor position information, and determines the display module corresponding to the two-dimensional subspace as the current controlled module; Step 4: The cursor operation mapping unit maps the movement trajectory and operation information of the background cursor to the foreground cursor in real time, obtains the corresponding operation control and acts on the desktop block corresponding to the current controlled module; Step 5: The operating system operates and controls the foreground desktop through the foreground cursor; Step 6: Perform virtual scene simulation on the XR shed and the viewing area through the virtual scene simulation unit; Step 7: The foreground desktop divides the desktop blocks corresponding to each display module, and configures the software status of the corresponding desktop blocks according to the operation control; Step 8: The foreground software generates the display content of each display module according to the software status; Step 9: The XR shed displays the display content through the display module; Step 10: Repeat steps 1 to 9 to achieve the planar control adaptation of the XR shed until the adaptation process ends.

Images