Shielding method and display method of mixed reality space, equipment and mixed reality terminal

Abstract

The embodiment of the invention discloses a mixed reality space shielding method, a display method, equipment and a mixed reality terminal. A specific embodiment of the method comprises the steps of displaying a preset shielding element list in a scene editing page in response to detection of an opening operation acting on a space shielding editing control; in response to the fact that the dragging operation acting on any preset shielding element is detected, and the dragging operation is released in the map element, displaying any preset shielding element at the release position; according to configuration information corresponding to at least one configuration operation corresponding to any preset shielding element, any preset shielding element is updated and displayed in the scene editing page, and the shielding element is used for shielding at least partial area of at least one scene element. The at least one scene element is each scene element arranged behind the shielding element in the target mixed reality space in the field angle range. According to the embodiment, mold penetration caused by modeling vulnerabilities can be avoided, and the sense of reality when experiencing the mixed reality space is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of mixed reality space, and in particular to a blocking method, a display method, a device, and a mixed reality terminal of a mixed reality space. Background Art

[0002] Mixed reality is a mixture of the physical and digital worlds, opening up natural and intuitive 3D interactions between people, computers, and the environment. Currently, when displaying mixed reality scenes in mixed reality spaces, mixed reality scenes usually contain multiple virtual objects. When there are loopholes in the modeling of multiple virtual objects, model penetration will occur, resulting in poor realism when experiencing mixed reality spaces.

[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure propose a spatial occlusion method for a mixed reality space, a mixed reality space display method, an electronic device, a mixed reality terminal, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a spatial occlusion method for a mixed reality space, the method comprising: in response to detecting an opening operation of a spatial occlusion editing control displayed in a scene editing page acting on a target mixed reality space, displaying a preset occlusion element list in the above-mentioned scene editing page, wherein each preset occlusion element is displayed in the above-mentioned preset occlusion element list; in response to detecting a drag operation acting on any preset occlusion element in the above-mentioned preset occlusion element list, and the above-mentioned drag operation is released in a map element corresponding to the above-mentioned target mixed reality space in the above-mentioned scene editing page, displaying the above-mentioned any preset occlusion element at the released position; and blocking the space according to the corresponding any preset occlusion element. The configuration information corresponding to at least one configuration operation of the element, updates and displays any of the above-mentioned preset occlusion elements in the above-mentioned scene editing page, and uses the updated any of the above-mentioned preset occlusion elements as the occlusion element, wherein the above-mentioned occlusion element is the updated and displayed any of the above-mentioned preset occlusion elements, and the above-mentioned occlusion element is used to block at least a partial area of ​​at least one scene element, and the above-mentioned at least one scene element is each scene element set behind the above-mentioned occlusion element in the above-mentioned target mixed reality space within the field of view angle range; the above-mentioned occlusion element has adjustable parameters of field of view transparency and sound field transparency to set the shielding ability of the occlusion element in the mixed reality space for the scene elements relative to the user's field of view and sound field.

[0007] Optionally, the method further includes: in response to detecting an opening operation acting on the above-mentioned spatial occlusion editing control, performing transparency processing on each scene element in the above-mentioned target mixed reality space.

[0008] Optionally, a map element opening control is also displayed in the scene editing page; and the method further comprises: in response to detecting a closing operation acting on the map element opening control, performing transparency processing on the map element.

[0009] Optionally, a spatial occlusion enable control is also displayed in the above-mentioned scene editing page; and the method also includes: in response to detecting an enable operation acting on the above-mentioned spatial occlusion enable control, the occluding element, or the above-mentioned map element and the occluding element are subjected to transparency restoration processing.

[0010] Optionally, the method also includes: in response to detecting an opening operation acting on the above-mentioned map element opening control, performing transparency restoration processing on the above-mentioned map element; in response to detecting a closing operation acting on the above-mentioned spatial occlusion opening control, performing transparency processing on the occluding element, or the above-mentioned map element and the above-mentioned occluding element.

[0011] Optionally, the scene editing page also displays an occlusion element information bar, in which the occlusion element information of each occlusion element configured in the target mixed reality space is displayed; and the method further includes: in response to detecting a delete operation acting on any occlusion element information in the occlusion element information bar, deleting the occlusion element corresponding to any occlusion element information in the target mixed reality space; in response to detecting a hide operation acting on any occlusion element information in the occlusion element information bar, hiding the occlusion element corresponding to any occlusion element information in the target mixed reality space.

[0012] Optionally, the at least one configuration operation includes at least one of the following: a position adjustment operation, a size adjustment operation, a scale adjustment operation, and an angle adjustment operation; and according to the configuration information corresponding to at least one configuration operation corresponding to any of the above-mentioned preset blocking elements, updating and displaying any of the above-mentioned preset blocking elements in the above-mentioned scene editing page, including: in response to detecting a position adjustment operation acting on any of the above-mentioned preset blocking elements, displaying the above-mentioned preset blocking elements after position adjustment in the above-mentioned scene editing page according to the position information corresponding to the above-mentioned position adjustment operation; in response to detecting a size adjustment operation acting on any of the above-mentioned preset blocking elements, displaying the above-mentioned preset blocking elements after size adjustment in the above-mentioned scene editing page according to the size information corresponding to the above-mentioned size adjustment operation; in response to detecting a scale adjustment operation acting on any of the above-mentioned preset blocking elements, displaying the above-mentioned preset blocking elements after scale adjustment in the above-mentioned scene editing page according to the scale information corresponding to the above-mentioned scale adjustment operation; in response to detecting an angle adjustment operation acting on any of the above-mentioned preset blocking elements, displaying the above-mentioned preset blocking elements after scale adjustment in the above-mentioned scene editing page according to the angle information corresponding to the above-mentioned angle adjustment operation.

[0013] Optionally, the method further includes: in response to detecting an exit operation acting on the above-mentioned spatial occlusion editing control, saving various configuration information of various occlusion elements configured in the above-mentioned target mixed reality space.

[0014] In a second aspect, some embodiments of the present disclosure provide a mixed reality space display method, which is applied to a mixed reality end, wherein the mixed reality end includes a head-mounted display device, and the method includes: displaying a mixed reality scene of a target mixed reality space in the head-mounted display device, wherein the target mixed reality space includes at least one occlusion element, and each occlusion element is configured by the method as described in any implementation of the first aspect; in response to determining that the occlusion state corresponding to the mixed reality scene is an on state, determining each occlusion element within a field of view angle range as a target occlusion element set according to current posture information of the head-mounted display device; and according to the values ​​of field of view transparency and sound field transparency parameters of the occlusion elements, for each target occlusion element in the target occlusion element set, shielding the field of view and sound field of at least a partial area of ​​each scene element behind the target occlusion element within the field of view angle range.

[0015] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0016] In a fourth aspect, some embodiments of the present disclosure provide a mixed reality terminal, comprising: a head-mounted display device for allowing a user to browse mixed reality scenes and scene elements; the mixed reality terminal is configured to execute the method described in any implementation manner of the second aspect above.

[0017] In a fifth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation of the first aspect or the second aspect is implemented.

[0018] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the spatial occlusion method of the mixed reality space of some embodiments of the present disclosure, the penetration caused by the modeling loophole can be avoided, and the sense of reality when experiencing the mixed reality space is improved. Specifically, the reason for the poor sense of reality when experiencing the mixed reality space is that when there are loopholes in the modeling of multiple virtual objects, the penetration will occur, resulting in poor sense of reality when experiencing the mixed reality space. Based on this, the spatial occlusion method of the mixed reality space of some embodiments of the present disclosure, first, in response to detecting the opening operation of the spatial occlusion editing control displayed in the scene editing page acting on the target mixed reality space, a preset occlusion element list is displayed in the above scene editing page. Among them, each preset occlusion element is displayed in the above preset occlusion element list. Thus, when the user determines to edit the occlusion element in the mixed reality space, the preset occlusion element list can be displayed for the user to select the preset occlusion element from it. Then, in response to detecting a drag operation acting on any preset occlusion element in the above preset occlusion element list, and the above drag operation is released in the map element corresponding to the above target mixed reality space in the above scene editing page, the above any preset occlusion element is displayed at the release position. Thus, when the user drags the preset occlusion element into the map element, the preset occlusion element can be displayed at the release position of the drag. Finally, according to the configuration information corresponding to at least one configuration operation corresponding to any of the above preset occlusion elements, the above any preset occlusion element is updated and displayed in the above scene editing page, and the updated any of the above preset occlusion elements is used as an occlusion element. Among them, the above occlusion element is the above any preset occlusion element that is updated and displayed. The above occlusion element is used to block at least a partial area of ​​at least one scene element. The above at least one scene element is each scene element arranged behind the above occlusion element in the above target mixed reality space under the field of view angle range. The above occlusion element has adjustable parameters of field of view transparency and sound field transparency to set the shielding ability of the occlusion element in the mixed reality space for the scene element relative to the user's field of view and sound field. Thus, the user can customize the dragged preset occlusion element. Also, because the occlusion elements configured in the mixed reality space can be used to occlude at least part of the scene elements set behind the above-mentioned occlusion elements within the field of view angle range, it is possible to avoid model penetration due to modeling loopholes, thereby improving the sense of reality when experiencing the mixed reality space. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0020] Figure 1is a flow chart of some embodiments of a spatial occlusion method for a mixed reality space according to the present disclosure;

[0021] Figure 2 is a schematic diagram of an application scenario of a spatial occlusion method of a mixed reality space according to some embodiments of the present disclosure;

[0022] Figure 3 is a flow chart of some embodiments of the mixed reality space display method according to the present disclosure;

[0023] Figure 4 is a schematic diagram of an application scenario of a mixed reality space display method according to some embodiments of the present disclosure;

[0024] Figure 5 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the spatial occlusion method of the mixed reality space disclosed in the present invention;

[0025] Figure 6 It is a structural schematic diagram of a mixed reality terminal suitable for implementing some embodiments of the mixed reality space display method disclosed in the present invention. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0027] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0028] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0029] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0030] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0031] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0032] Figure 1 The process 100 of some embodiments of the spatial occlusion method of the mixed reality space according to the present disclosure is shown. The spatial occlusion method of the mixed reality space includes the following steps:

[0033] Step 101, in response to detecting an opening operation of a spatial occlusion editing control displayed in a scene editing page acting on a target mixed reality space, a preset occlusion element list is displayed in the scene editing page.

[0034] In some embodiments, the execution subject (e.g., computing device) of the spatial occlusion method of the mixed reality space may respond to detecting the opening operation of the spatial occlusion editing control displayed in the scene editing page acting on the target mixed reality space, and display a preset occlusion element list in the above scene editing page. Among them, the above target mixed reality space may be the mixed reality space currently edited by the user. The above scene editing page may be a graphical user interface when editing the above target mixed reality space. The above spatial occlusion editing control may be a control for switching the editing state of the occlusion element. The above spatial occlusion editing control may have an editing state and an unedited state. The above opening operation may be an operation of switching the above spatial occlusion editing control from an unedited state to an editing state. For example, the above opening operation may be a selection operation of the spatial occlusion editing control acting on the unedited state. Here, the selection operation may include but is not limited to at least one of the following: click, hover, slide. The above preset occlusion element list displays each preset occlusion element. The above preset occlusion element list may be a list of pre-configured occlusion elements of different styles that can be selected by the user. For example, the above preset occlusion element list may include preset occlusion elements of different shapes. The execution subject can be a browser or an application.

[0035] In practice, the execution entity may display the preset occlusion element list in the left column of the scene editing page. Optionally, the spatial occlusion editing control may be displayed in the mixed reality scene display area of ​​the scene editing page. The mixed reality scene display area may be an area for editing and displaying the mixed reality scene of the target mixed reality space.

[0036] Step 102, in response to detecting a drag operation acting on any preset occluding element in the preset occluding element list, and the drag operation is released in the map element corresponding to the target mixed reality space in the scene editing page, any preset occluding element is displayed at the released position.

[0037] In some embodiments, the execution subject may respond to detecting a drag operation acting on any preset blocking element in the preset blocking element list, and the drag operation is released in the map element corresponding to the target mixed reality space in the scene editing page, and display the any preset blocking element at the release position. Among them, the drag operation may be an operation of first selecting a preset blocking element and then dragging the selected preset blocking element to another position. The map element may be a white map film in the target mixed reality space. The release position may be a position where the dragging of the selected preset blocking element is canceled. For example, the drag operation may be a moving operation of pressing and holding the left mouse button. The release position may be the position of the mouse pointer in the map element when the left mouse button is released. In practice, the execution subject may display the any preset blocking element fixedly at the release position.

[0038] Step 103: update and display any preset occlusion element in the scene editing page according to the configuration information corresponding to at least one configuration operation corresponding to any preset occlusion element, and use the updated any preset occlusion element as the occlusion element.

[0039] In some embodiments, the execution subject may update and display any preset occlusion element in the scene editing page according to the configuration information corresponding to at least one configuration operation corresponding to any preset occlusion element, and use the updated preset occlusion element as an occlusion element. Among them, the at least one configuration operation may be various configuration operations for the basic attributes of any preset occlusion element. The at least one configuration operation may include but is not limited to a shape configuration operation. The shape configuration operation may be an operation to modify the shape of any preset occlusion element. For example, the shape configuration operation may be an operation to modify any preset occlusion element of a quadrilateral to any preset occlusion element of a circle. The configuration information may be various attribute information of any preset occlusion element after configuration. The configuration information may include but is not limited to the contour coordinates of any preset occlusion element. The occlusion element is any preset occlusion element updated and displayed. The occlusion element is used to cover at least a partial area of ​​at least one scene element. The at least one scene element is each scene element set behind the occlusion element in the target mixed reality space within the field of view angle range. Specifically, it can be understood that when the occlusion element is configured in the front view angle, the at least one scene element may include the various scene elements in the target mixed reality space that are set behind the occlusion element and are completely invisible due to the occlusion of the occlusion element. When the occlusion element is configured in the side view angle, the at least one scene element may include the various scene elements in the target mixed reality space that are set behind the occlusion element and are partially invisible due to the occlusion of the occlusion element. The occlusion element may have adjustable parameters of field transparency and sound field transparency to set the shielding ability of the occlusion element in the mixed reality space for the scene element relative to the user's field of view and sound field. For example, if there is a virtual TV behind the occlusion element, if the sound field is not blocked, the user cannot see the TV, but can hear the TV sound. It should be noted that the field transparency is generally 0, that is, complete occlusion, and the sound field transparency can be adjusted from 0 to 100%.

[0040] As an example, the configured occlusion element can be Figure 2 shown. Figure 2 In the embodiment of the present invention, an occlusion element 201 is displayed in a scene editing page 202. The scene editing page 202 includes a mixed reality scene display area 203. A spatial occlusion editing control 204 is displayed in the mixed reality scene display area 203. The occlusion element 201 can be set between a scene element 205 and a scene element 206. The occlusion element 201 can be used to replace the scene element 205 to occlude the scene element 206. Figure 2 In the example, scene element 205 may be a wall. Scene element 206 may be a cabinet. It should be noted that Figure 2The occluding element 201 displayed in the figure may be the occluding element before adjustment, and the user may adjust the height of the occluding element 201 to be the same as the height of the scene element 205 .

[0041] Optionally, the at least one configuration operation includes at least one of the following: a position adjustment operation, a size adjustment operation, a ratio adjustment operation, and an angle adjustment operation.

[0042] In some optional implementations of some embodiments, the execution subject may update and display any of the preset occlusion elements in the scene editing page according to the configuration information corresponding to at least one configuration operation corresponding to any of the preset occlusion elements through the following steps:

[0043] In the first step, in response to detecting a position adjustment operation acting on any of the above-mentioned preset occluding elements, the above-mentioned preset occluding elements after position adjustment are displayed in the above-mentioned scene editing page according to the position information corresponding to the above-mentioned position adjustment operation. The above-mentioned position information may be the position coordinates of the above-mentioned preset occluding elements after adjustment in the above-mentioned target mixed reality space. For example, the position coordinates may be the center coordinates of the above-mentioned preset occluding elements after adjustment.

[0044] In the second step, in response to detecting a resizing operation on any of the preset shielding elements, the resized preset shielding elements are displayed in the scene editing page according to the size information corresponding to the resizing operation. The size information may be the length and width of the resized preset shielding elements. The size information may also include the height.

[0045] In the third step, in response to detecting a scale adjustment operation on any of the preset occluding elements, the scale-adjusted preset occluding elements are displayed in the scene editing page according to the scale information corresponding to the scale adjustment operation. The scale information may be the adjusted aspect ratio of any of the preset occluding elements. The scale information may also include a height ratio.

[0046] In the fourth step, in response to detecting an angle adjustment operation on any of the preset occluding elements, the preset occluding elements after proportional adjustment are displayed in the scene editing page according to the angle information corresponding to the angle adjustment operation. The angle information may be the angle of the preset occluding elements in the target mixed reality space after adjustment. Thus, the user can adjust the position, size, proportion and angle of the occluding elements.

[0047] Optionally, the above-mentioned execution subject may also perform transparency processing on each scene element in the above-mentioned target mixed reality space in response to detecting an opening operation acting on the above-mentioned spatial occlusion editing control. Among them, the above-mentioned each scene element may be each spatial element configured in the above-mentioned target mixed reality space. For example, each scene element may represent a different virtual object. In practice, the above-mentioned execution subject may update the transparency of each scene element to a preset transparency. For example, the preset transparency may be 50%, so that each scene element may be semi-transparently processed. Thus, when the user edits the occlusion element, the each scene element in the target mixed reality space may have a certain transparency, thereby facilitating the editing of the occlusion element.

[0048] Optionally, a map element enable control is also displayed in the above-mentioned scene editing page. The above-mentioned map element enable control can be a control for switching the display state of the map element. The above-mentioned map element enable control can correspond to a map element full display state and a map element transparent display state. The map element in the full display state is opaque. The map element in the transparent display state is displayed with a preset transparency. Optionally, the map element enable control can be displayed in the right column of the above-mentioned scene editing page.

[0049] Optionally, the execution subject may perform transparency processing on the map element in response to detecting a close operation on the map element opening control. The close operation may be an operation of switching the map element opening control from a full display state of the map element to a transparent display state of the map element. For example, the close operation may be a click operation. In practice, the execution subject may display the map element with a preset transparency. It should be noted that the initial display state of the map element may be a full display state of the map element. Thus, the user may switch the display state of the map element through the map element opening control.

[0050] Optionally, a spatial occlusion enable control is also displayed in the above-mentioned scene editing page. Among them, the above-mentioned spatial occlusion enable control can be a control for switching the display status of the occlusion elements, or the above-mentioned map elements and occlusion elements. The above-mentioned spatial occlusion enable control can correspond to a full display state and a transparent display state. The switched occlusion elements, or the above-mentioned map elements and occlusion elements in the full display state are opaque. The switched occlusion elements, or the above-mentioned map elements and occlusion elements in the transparent display state are displayed with a preset transparency. Optionally, the spatial occlusion enable control can be displayed in the right column of the above-mentioned scene editing page.

[0051] Optionally, the above-mentioned execution subject may restore the transparency of the occluding element, or the above-mentioned map element, in response to detecting an opening operation acting on the above-mentioned spatial occlusion opening control. The above-mentioned opening operation may be an operation of switching the above-mentioned spatial occlusion opening control from a transparent display state to a full display state. For example, the opening operation may be a click operation. In practice, the above-mentioned execution subject may restore the transparency of the occluding element, or the above-mentioned map element and the occluding element. Thus, the user may switch the display state of the occluding element, or the above-mentioned map element and the occluding element through the spatial occlusion opening control.

[0052] Optionally, the above-mentioned execution subject may perform transparency restoration processing on the above-mentioned map element in response to detecting an opening operation acting on the above-mentioned map element opening control. Here, the opening operation may be an operation of switching the above-mentioned map element opening control from a transparent display state of the map element to a full display state of the map element. Secondly, in response to detecting a closing operation acting on the above-mentioned spatial occlusion opening control, transparency processing may be performed on the occluding element, or the above-mentioned map element and the above-mentioned occluding element. Here, the closing operation may be an operation of switching the above-mentioned spatial occlusion opening control from a full display state to a transparent display state. In practice, the above-mentioned execution subject may display the occluding element, or the above-mentioned map element and the above-mentioned occluding element with a preset transparency. For example, the preset transparency may be 50%.

[0053] It can be understood that the occlusion elements that need to switch the display state may be all the occlusion elements configured in the above-mentioned target mixed reality scene.

[0054] Optionally, the scene editing page also displays an occlusion element information bar. The occlusion element information bar displays the occlusion element information of each occlusion element configured in the target mixed reality space. The occlusion element information may be attribute information of the occlusion element. For example, the occlusion element information may include but is not limited to at least one of the following: the name, thumbnail, and scale of the occlusion element.

[0055] Optionally, the execution subject may delete the occlusion element corresponding to any occlusion element information in the target mixed reality space in response to detecting a delete operation acting on any occlusion element information in the occlusion element information column. The delete operation may be an operation of deleting the selected occlusion element information and the corresponding occlusion element. For example, the delete operation may be an operation of clicking a delete operation item in the operation item popped up by right-clicking the occlusion element information. For another example, the delete operation may be an operation of selecting a delete control corresponding to the occlusion element information. Here, the specific setting of the delete operation is not limited. Secondly, in response to detecting a hide operation acting on any occlusion element information in the occlusion element information column, the occlusion element corresponding to any occlusion element information may be hidden in the target mixed reality space. The hide operation may be a selection operation of a hide control acting on the corresponding occlusion element information. Specifically, the hide control may correspond to a hide state and a cancel hide state. The hide operation may be an operation of switching the hide control from the cancel hide state to the hide state. For example, the hide operation may be a click operation. Thus, the user may delete or hide the previously configured occlusion element.

[0056] Optionally, the execution subject may save various configuration information of various occlusion elements configured in the target mixed reality space in response to detecting an exit operation acting on the spatial occlusion editing control. The exit operation may be an operation of exiting the editing of the occlusion element. Specifically, the exit operation may be an operation of switching the spatial occlusion editing control from an editing state to an unedited state.

[0057] Optionally, the execution subject may also clear cached configuration information of each occlusion element configured in the target mixed reality space in response to detecting a program exception, so that the configuration information of the occlusion element edited this time may not be saved in an abnormal situation.

[0058] It should be noted that occlusion elements can also be used to divide the space in the target mixed reality space, that is, the target mixed reality space can be divided into various sub-spaces.

[0059] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the spatial occlusion method of the mixed reality space of some embodiments of the present disclosure, the penetration caused by the modeling loophole can be avoided, and the sense of reality when experiencing the mixed reality space is improved. Specifically, the reason for the poor sense of reality when experiencing the mixed reality space is that when there are loopholes in the modeling of multiple virtual objects, the penetration will occur, resulting in poor sense of reality when experiencing the mixed reality space. Based on this, the spatial occlusion method of the mixed reality space of some embodiments of the present disclosure, first, in response to detecting the opening operation of the spatial occlusion editing control displayed in the scene editing page acting on the target mixed reality space, a preset occlusion element list is displayed in the above scene editing page. Among them, each preset occlusion element is displayed in the above preset occlusion element list. Thus, when the user determines to edit the occlusion element in the mixed reality space, the preset occlusion element list can be displayed for the user to select the preset occlusion element from it. Then, in response to detecting a drag operation acting on any preset occlusion element in the above preset occlusion element list, and the above drag operation is released in the map element corresponding to the above target mixed reality space in the above scene editing page, the above any preset occlusion element is displayed at the release position. Thus, when the user drags the preset occlusion element into the map element, the preset occlusion element can be displayed at the release position of the drag. Finally, according to the configuration information corresponding to at least one configuration operation corresponding to any of the above preset occlusion elements, the above any preset occlusion element is updated and displayed in the above scene editing page, and the updated any of the above preset occlusion elements is used as an occlusion element. Among them, the above occlusion element is the above any preset occlusion element that is updated and displayed. The above occlusion element is used to block at least a partial area of ​​at least one scene element. The above at least one scene element is each scene element set behind the above occlusion element in the above target mixed reality space under the field of view angle range. The above occlusion element may have adjustable parameters of field of view transparency and sound field transparency to set the shielding ability of the occlusion element in the mixed reality space for the scene element relative to the user's field of view and sound field. Thus, the user can customize the dragged preset occlusion element. Also, because the occlusion elements configured in the mixed reality space can be used to occlude at least part of the scene elements set behind the above-mentioned occlusion elements within the field of view angle range, it is possible to avoid model penetration due to modeling loopholes, thereby improving the sense of reality when experiencing the mixed reality space.

[0060] Further references Figure 3 , which shows a process 300 of some embodiments of a mixed reality space display method. The process 300 of the mixed reality space display method, applied to a mixed reality terminal, includes the following steps:

[0061] Step 301: Display a mixed reality scene of a target mixed reality space in a head-mounted display device.

[0062] In some embodiments, the execution subject of the mixed reality space display method (e.g., a mixed reality terminal) can display the mixed reality scene of the target mixed reality space in the above-mentioned head-mounted display device. Among them, the above-mentioned mixed reality terminal may include a head-mounted display device. The above-mentioned head-mounted display device may be MR glasses. Optionally, the above-mentioned mixed reality terminal may also include a mobile device. The above-mentioned mobile device can be used to provide computing power and / or electricity for the head-mounted display device. For example, the above-mentioned mobile device may include but is not limited to one of the following: a mobile phone, a tablet computer, and a smart terminal. The above-mentioned mixed reality scene may be the above-mentioned target mixed reality space after loading. Various scene elements may be displayed in the above-mentioned mixed reality scene. The above-mentioned target mixed reality space may contain at least one occlusion element. Each occlusion element is obtained by Figure 1 The steps in the corresponding embodiments are configured.

[0063] Step 302, in response to determining that the occlusion state of the corresponding mixed reality scene is the on state, each occlusion element within the field of view angle is determined as a target occlusion element set according to the current posture information of the head mounted display device.

[0064] In some embodiments, the execution subject may determine each occlusion element within the field of view angle range as a target occlusion element set according to the current posture information of the head-mounted display device in response to determining that the occlusion state corresponding to the mixed reality scene is the on state. Among them, the occlusion state may be a state that characterizes whether the occlusion effect of the occlusion element is activated. In the on state, the occlusion element can produce an occlusion effect. In the off state, the occlusion element does not produce an occlusion effect. The posture information may be multi-degree-of-freedom position and attitude information. The posture information may be 3DOF or 6DOF posture data. The field of view angle range may be an area within the field of view angle corresponding to the posture information in the mixed reality scene.

[0065] Step 303, according to the values ​​of the field of view transparency and sound field transparency parameters of the above-mentioned occluding elements, for each target occluding element in the target occluding element set, shield the field of view and sound field of at least part of the area of ​​each scene element behind the target occluding element within the field of view angle range.

[0066] In some embodiments, according to the values ​​of the field of view transparency and sound field transparency parameters of the above-mentioned occlusion element, for each target occlusion element in the above-mentioned target occlusion element set, the above-mentioned execution subject may shield the field of view and sound field of at least part of the area of ​​each scene element behind the above-mentioned target occlusion element within the above-mentioned field of view angle range. Here, the occlusion logic between the target occlusion element and the scene element can refer to Figure 1 The description of step 103 in the corresponding embodiments.

[0067] As an example, the mixed reality scene displayed in the head mounted display device can refer to Figure 4 . Figure 4 In the example, the occlusion state is turned on, and the occlusion element can be set between scene elements 401 and 402. The occlusion element is not actually displayed in the mixed reality scene. Figure 4 Under the corresponding viewing angle, the blocking element may block a portion of the scene element 402 , so that when the user looks from the scene element 401 to the scene element 402 , the user cannot see the blocked portion of the scene element 402 .

[0068] Figure 3 The process 300 of the mixed reality space display method in some corresponding embodiments embodies the steps of displaying the mixed reality space. Therefore, the schemes described in these embodiments can enable a user wearing a head-mounted display device to view mixed reality scene elements with correct occlusion relationships and listen to the sounds emitted by mixed reality scene elements with occlusion relationships.

[0069] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device 500 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0070] like Figure 5 As shown, the electronic device 500 may include a processing device 501 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0071] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0072] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0073] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0074] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0075] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: in response to detecting the opening operation of the spatial occlusion editing control displayed in the scene editing page acting on the target mixed reality space, a preset occlusion element list is displayed in the above-mentioned scene editing page, wherein each preset occlusion element is displayed in the above-mentioned preset occlusion element list; in response to detecting a drag operation acting on any preset occlusion element in the above-mentioned preset occlusion element list, and the above-mentioned drag operation is released in the map element corresponding to the above-mentioned target mixed reality space in the above-mentioned scene editing page, the above-mentioned any preset occlusion element is displayed at the released position; according to the corresponding any preset occlusion element Configuration information corresponding to at least one configuration operation of a preset occlusion element, updating and displaying any of the above-mentioned preset occlusion elements in the above-mentioned scene editing page, and using the updated any of the above-mentioned preset occlusion elements as an occlusion element, wherein the above-mentioned occlusion element is the updated and displayed any of the above-mentioned preset occlusion elements, and the above-mentioned occlusion element is used to obstruct at least a partial area of ​​at least one scene element, and the above-mentioned at least one scene element is each scene element arranged behind the above-mentioned occlusion element in the above-mentioned target mixed reality space within the field of view angle range; the above-mentioned occlusion element may have adjustable parameters of field of view transparency and sound field transparency to set the shielding ability of the occlusion element in the mixed reality space for the scene elements relative to the user's field of view and sound field.

[0076] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0079] Figure 6 The mixed reality terminal 600 is shown as being suitable for implementing some embodiments of the mixed reality space display method disclosed herein. The mixed reality terminal includes a head mounted display device 601. The head mounted display device 601 is used for allowing a user to browse a mixed reality scene and scene elements. The mixed reality terminal is configured to execute Figure 3 The steps in the corresponding embodiments.

[0080] Optionally, the mixed reality end may also include a mobile device 602. The mobile device 602 may be, but is not limited to, a mobile phone, a tablet computer, or a smart terminal. The mobile device 602 is connected to the head-mounted display device 601. Here, the connection between the mobile device 602 and the head-mounted display device 601 may be a wired connection or a wireless connection. In some embodiments, both the head-mounted display device 601 and the mobile device 602 may include a processor. In some optional implementations of some embodiments, the head-mounted display device 601 may be an external display device for the mobile device 602 and does not have data processing capabilities. The mobile device 602 may provide power support for the head-mounted display device 601. The head-mounted display device 601 may also be configured with a sound playback device. For example, the sound playback device may be a bone conduction sound playback device.

[0081] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the mixed reality terminal of some embodiments of the present disclosure, users can watch the mixed reality scene.

[0082] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A spatial occlusion method for a mixed reality space, comprising: In response to detecting an opening operation of a spatial occlusion editing control displayed in a scene editing page acting on a target mixed reality space, displaying a preset occlusion element list in the scene editing page, wherein each preset occlusion element is displayed in the preset occlusion element list; In response to detecting a drag operation on any preset occluding element in the preset occluding element list, and the drag operation being released in the map element corresponding to the target mixed reality space in the scene editing page, displaying the any preset occluding element at the released position; According to the configuration information corresponding to at least one configuration operation corresponding to the arbitrary preset occlusion element, the arbitrary preset occlusion element is updated and displayed in the scene editing page, and the updated arbitrary preset occlusion element is used as the occlusion element, wherein the occlusion element is the updated and displayed arbitrary preset occlusion element, and the occlusion element is used to occlude at least a partial area of ​​at least one scene element, and the at least one scene element is each scene element arranged behind the occlusion element in the target mixed reality space within the field of view angle range; The occlusion element has adjustable parameters of field of view transparency and sound field transparency to set the shielding ability of the occlusion element for scene elements relative to the field of view and sound field of the user in the mixed reality space.

2. The method according to claim 1, wherein: The method further comprises: In response to detecting an opening operation acting on the spatial occlusion editing control, transparency processing is performed on each scene element in the target mixed reality space.

3. The method according to claim 1, wherein: The scene editing page also displays a map element opening control; as well as The method further comprises: In response to detecting a closing operation acting on the map element opening control, transparency processing is performed on the map element.

4. The method according to claim 3, wherein: The scene editing page also displays a spatial occlusion enable control; as well as The method further comprises: In response to detecting an opening operation acting on the spatial occlusion opening control, transparency restoration processing is performed on the occluding element, or the map element and the occluding element.

5. The method according to claim 4, wherein: The method further comprises: In response to detecting an opening operation acting on the map element opening control, performing transparency restoration processing on the map element; In response to detecting a closing operation acting on the spatial occlusion opening control, transparency processing is performed on the occluding element, or the map element and the occluding element.

6. The method according to claim 1, wherein: The scene editing page also displays an occlusion element information bar, in which the occlusion element information of each occlusion element configured in the target mixed reality space is displayed; as well as The method further comprises: In response to detecting a deletion operation acting on any occlusion element information in the occlusion element information column, deleting the occlusion element corresponding to the any occlusion element information in the target mixed reality space; In response to detecting a hiding operation acting on any occluding element information in the occluding element information column, hiding the occluding element corresponding to the any occluding element information in the target mixed reality space.

7. The method according to claim 1, wherein: The at least one configuration operation includes at least one of the following: a position adjustment operation, a size adjustment operation, a ratio adjustment operation, and an angle adjustment operation; as well as The updating and displaying of the arbitrary preset occlusion element in the scene editing page according to the configuration information corresponding to at least one configuration operation corresponding to the arbitrary preset occlusion element includes: In response to detecting a position adjustment operation acting on the arbitrary preset blocking element, displaying the arbitrary preset blocking element after position adjustment in the scene editing page according to position information corresponding to the position adjustment operation; In response to detecting a resizing operation acting on the arbitrary preset shielding element, displaying the arbitrary preset shielding element after resizing in the scene editing page according to size information corresponding to the resizing operation; In response to detecting a scale adjustment operation acting on the arbitrary preset occluding element, displaying the arbitrary preset occluding element after scale adjustment in the scene editing page according to scale information corresponding to the scale adjustment operation; In response to detecting an angle adjustment operation acting on the arbitrary preset blocking element, the arbitrary preset blocking element after proportional adjustment is displayed in the scene editing page according to angle information corresponding to the angle adjustment operation.

8. The method according to claim 1, wherein: The method further comprises: In response to detecting an exit operation acting on the spatial occlusion editing control, various configuration information of various occlusion elements configured in the target mixed reality space is saved.

9. A mixed reality space display method, applied to a mixed reality terminal, the mixed reality terminal including a head-mounted display device, the method comprising: Displaying a mixed reality scene of a target mixed reality space in the head mounted display device, wherein the target mixed reality space includes at least one occlusion element, and each occlusion element is configured by the method according to any one of claims 1 to 8; In response to determining that the occlusion state corresponding to the mixed reality scene is an on state, determining each occlusion element within the field of view angle range as a target occlusion element set according to the current posture information of the head mounted display device; According to the values ​​of the field of view transparency and sound field transparency parameters of the occluding element, for each target occluding element in the target occluding element set, the field of view and sound field of at least a partial area of ​​each scene element behind the target occluding element within the field of view angle range are shielded.

10. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A mixed reality terminal, comprising: A head-mounted display device for allowing users to browse mixed reality scenes and scene elements; The mixed reality terminal is configured to execute the method as claimed in claim 9.

Images