Projection method and related equipment for extended reality equipment

Abstract

This application discloses a projection method and related equipment for an extended reality device, relating to the field of extended reality technology, and aims to solve the problem that the projected content of a virtual scene occludes objects in a real scene image that the user is paying attention to or needs to pay attention to. The projection method for an extended reality device provided in this application includes: capturing an image of a real scene; identifying a target object from the image of the real scene; determining a target area based on the target object, wherein the target area is the area covered by the target object in the image of the real scene; in response to a projection command, capturing the projection content of the virtual scene; and projecting the projection content of the virtual scene or a portion of the projection content outside the target area.

Description

Technical Field

[0001] The present application relates to the field of extended reality technology, and in particular to a projection method for an extended reality device and related equipment. Prior Art

[0002] Extended Reality (XR) combines real and virtual scenes to create a virtual environment that allows for human-computer interaction. Currently, when XR devices receive projected content from a virtual scene, they overlay it onto the real-world image, obscuring portions of the real-world image. When the projected virtual content obscures objects in the real-world image that the user is currently focusing on or needs to focus on, it degrades the user experience and may even endanger their personal safety.

[0003] For example, when a user uses an extended reality device while driving a vehicle, if the projected content of the virtual scene obscures vehicles and / or pedestrians on the road in the image of the real scene, the risk of traffic accidents will increase, thereby endangering the user's personal safety. Summary of the Invention

[0004] In view of this, the present application provides a projection method for an extended reality device and related equipment, aiming to solve the problem of the projected content of a virtual scene obscuring objects that the user is currently paying attention to or needs to pay attention to in the image of the real scene.

[0005] In a first aspect, the present application provides a projection method for an extended reality device, comprising: capturing an image of a real scene; identifying a target object from the image of the real scene; determining a target area based on the target object, the target area being the area covered by the target object in the image of the real scene; capturing projection content of a virtual scene in response to a projection instruction; and projecting the projection content or a portion of the projection content outside the target area.

[0006] The projection method of the extended reality device of this embodiment identifies a target object from an image of a real scene, determines a target area based on the target object, and then projects the projection content of the virtual scene outside the target area. This prevents the projection content of the virtual scene from obscuring an object in the image of the real scene that the user is currently focusing on or needs to focus on, thereby improving the user experience and reducing safety risks for the user when using the extended reality device.

[0007] A second aspect of the present application provides an extended reality device comprising a processor, memory, a projection device, and a camera. The extended reality device, via the processor, runs a computer program or code stored in the memory to: capture an image of a real scene using the camera; identify a target object from the image of the real scene using the processor, and determine a target area based on the target object, where the target area is the area covered by the target object in the image of the real scene; capture projection content of a virtual scene using the processor in response to a projection instruction; and project the projection content or a portion of the projection content outside the target area using the projection device.

[0008] The third party of this application provides a computer-readable storage medium for storing a computer program or code. When the computer program or code is executed by a processor, the projection method of the extended reality device provided in the first aspect of this application is implemented.

[0009] It can be understood that the specific implementation methods and beneficial effects of the extended reality device provided in the second aspect of this application and the computer-readable storage medium provided by the third party are roughly the same as the specific implementation methods and beneficial effects of the projection method of the extended reality device provided in the first aspect of this application, and will not be repeated here. Simple diagram description

[0010] FIG1 is a schematic diagram of the structure of the extended reality device provided in this application. FIG2 a is a schematic diagram of a real scene in Scenario 1. FIG. FIG2 b is a schematic diagram of the real scene superimposed on the virtual scene in scene 1. FIG. FIG3 a is a schematic diagram of a real scene in scenario 2. FIG. FIG3 b is a schematic diagram of a real scene superimposed on a virtual scene in scene 2. FIG. FIG4 is a flow chart of the projection method of the extended reality device provided in this application. FIG5 is a flowchart of sub-steps of step S402 shown in FIG4 . FIG6 is a flowchart of sub-steps of step S405 shown in FIG4 . FIG7 a is a schematic diagram of a real scene in scenario 3. FIG. FIG7 b is a schematic diagram of identifying a target object from an image of a real scene in scenario 3. FIG. FIG7 c is a schematic diagram of determining a target area based on a target object in scene 3. FIG. FIG7 d is a schematic diagram of projecting content outside the target area in scene 3. FIG. Implementation Method

[0011] It should be noted that in the embodiments of this application, "at least one" refers to one or more than one, and "more than one" refers to two or more than two. "And / or" describes the relationship between related items, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The terms "first," "second," "third," "fourth," and so on (if any) in the specification, claims, and drawings of this application are used to distinguish similar items, not to describe a specific order or sequence.

[0012] It should also be noted that the methods disclosed in the embodiments of the present application or the methods shown in the flowcharts include one or more steps for implementing the methods. Without departing from the scope of the claims, the execution order of the multiple steps can be interchanged with each other, and some of the steps can also be deleted.

[0013] The following is a brief description of the extended reality device provided by this application.

[0014] FIG1 is a schematic structural diagram of an extended reality device 100 provided in this application.

[0015] As shown in FIG1 , an extended reality device 100 includes a processor 110, a memory 120, a projection device 130, and a camera 140. Processor 110 is electrically connected to memory 120, projection device 130, and camera 140. Processor 110 can execute a computer program or code stored in memory 120 to implement the projection method of the embodiments of the present application.

[0016] Processor 110 may include one or more processing units. For example, processor 110 may include, but is not limited to, an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video transcoder, a digital signal processor (DSP), a baseband processor, a neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0017] Processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that processor 110 has just used or is looping. If processor 110 needs to use the instruction or data again, it can directly call it from this memory.

[0018] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include, but are not limited to, an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, a Universal Serial Bus (USB) interface, and the like.

[0019] It should be understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the extended reality device 100. In other embodiments, the extended reality device 100 may also employ different interface connection methods from those in the above embodiments, or a combination of multiple interface connection methods.

[0020] The memory 120 may include an external memory interface and internal memory. The external memory interface can be used to connect an external memory card, such as a Micro SD card, to increase the storage capacity of the extended reality device 100. The external memory card communicates with the processor 110 via the external memory interface to implement data storage. The internal memory can be used to store computer executable code, which includes instructions. The internal memory may include a program storage area and a data storage area. The program storage area may store an operating system and applications required for at least one function (such as a camera function, a projection function, or an image playback function). The data storage area may store data created during the use of the extended reality device 100 (such as image data, projection data, etc.). Furthermore, the internal memory may include high-speed random access memory and non-volatile memory, such as at least one disk memory device, a flash memory device, or Universal Flash Storage (UFS). The processor 110 executes various functional applications and data processing of the extended reality device 100 by running instructions stored in the internal memory and / or instructions stored in the memory provided in the processor 110, such as implementing the projection method of the embodiment of the present application.

[0021] The projection device 130 is used to project images. Specifically, the projection device 130 can capture an image of a virtual scene from the processor 110 or the memory 120 or an external device (not shown) and then project the image of the virtual scene.

[0022] Among them, external devices may include smartphones, tablet computers, personal computers (PCs), e-book readers, workstations, servers, personal digital assistants (PDAs), portable multimedia players (PMPs), etc.

[0023] It can be understood that the extended reality device 100 can be connected to and communicate with an external device via a wireless communication module, thereby capturing an image of a virtual scene from the external device via the projection device 130.

[0024] The camera 140 is used to capture images. Specifically, the XR device 100 can capture images of real scenes using the camera 140. In some embodiments, the XR device 100 may include one or more cameras 140.

[0025] It should be understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the extended reality device 100. In other embodiments, the extended reality device 100 may include more or fewer components than illustrated, or may combine or separate certain components, or employ a different component arrangement. The illustrated components may be implemented as hardware, software, or a combination of both.

[0026] It is understood that the extended reality device 100 may include an augmented reality (AR) device, a virtual reality (VR) device, or a mixed reality (MR) device. For example, the extended reality device 100 may include, but is not limited to, an AR / VR / MR head-mounted display device, AR / VR / MR glasses, an AR / VR / MR helmet, etc.

[0027] The following uses scenario 1 and scenario 2 as examples to illustrate the application scenarios of this application. Scenario 1

[0028] FIG2a and FIG2b are schematic diagrams of scenario 1. ...

[0029] Specifically, Figure 2a is a schematic diagram of a real scene. The augmented reality device 100 can capture an image of the real scene using the camera 140. As shown in Figure 2a, the real scene includes the character that the user is focusing on.

[0030] Figure 2b is a schematic diagram of a real scene superimposed on a virtual scene. The augmented reality device 100 can use the projection device 130 to capture the projection content of the virtual scene from the processor 110, memory 120, or an external device, and then project the projection content of the virtual scene through the projection device 130. As shown in Figure 2b, the projected content of the virtual scene includes a message prompt box, which obscures the character image that the user is currently focusing on. Scenario 2

[0031] FIG3a and FIG3b are schematic diagrams of scenario 2. ...

[0032] Specifically, Figure 3a is a schematic diagram of a real scene. The extended reality device 100 can capture an image of the real scene using the camera 140. As shown in Figure 3a, the real scene includes vehicles and pedestrians on the road.

[0033] Figure 3b is a schematic diagram of a real scene superimposed on a virtual scene. The augmented reality device 100 can use the projection device 130 to capture the projection content of the virtual scene from the processor 110, memory 120, or an external device, and then project the projection content of the virtual scene through the projection device 130. As shown in Figure 3b, the projected content of the virtual scene includes an incoming call notification box, which obscures the vehicle on the road that the user needs to pay attention to.

[0034] As can be seen from Scenes 1 and 2, when the extended reality device 100 receives projected content from a virtual scene, it overlays the projected content of the virtual scene on the image of the real scene, causing a portion of the real scene image to be obscured by the projected content of the virtual scene. When the projected content of the virtual scene obscures an object in the real scene image that the user is currently focusing on or needs to focus on, it degrades the user's experience and may even endanger the user's personal safety.

[0035] Based on this, the present application provides a projection method and related equipment for an extended reality device. By identifying a target object from an image of a real scene, determining a target area based on the target object, and then projecting the projection content of a virtual scene outside the target area, the projection content of the virtual scene can be prevented from obscuring an object that the user is currently paying attention to or needs to pay attention to in the image of the real scene, thereby improving the user experience and reducing the safety risks of the user when using the extended reality device.

[0036] The projection method of the extended reality device provided in this application is described below.

[0037] FIG4 is a flow chart of the projection method of the extended reality device provided in this application.

[0038] It is understood that the projection method can be applied to the extended reality device 100. As shown in FIG4 , the projection method may include the following steps:

[0039] S401, capturing an image of a real scene.

[0040] In this embodiment, the extended reality device 100 can capture an image of the real scene by photographing the real scene via the camera 140.

[0041] S402, identifying a target object from an image of a real scene.

[0042] The target object may include an object that the user is currently paying attention to or needs to pay attention to. For example, referring to the schematic diagrams of real scenes shown in Figures 2a and 3a, the target object in the real scene shown in Figure 2a is a person that the user is currently paying attention to, while the target object in the real scene shown in Figure 3a is a vehicle and pedestrians on the road.

[0043] It is understood that the number of targets can be one or more.

[0044] In some embodiments, the extended reality device 100 may use a feature matching method to identify a target object from an image of a real scene.

[0045] Specifically, the extended reality device 100 can first identify target features from the image of the real scene, and then compare the target features with the features of the preset scene type to determine the preset scene type corresponding to the target features. The preset scene may include one or more targets; then, according to the preset scene type, the target object database is queried. The target object database stores information on targets corresponding to multiple preset scene types, thereby determining the target corresponding to the preset scene type.

[0046] It is understood that the preset scene types can be set as needed. For example, the preset scene types may include indoor scene types and outdoor scene types, where indoor scene types may include restaurant scene types and conference room scene types, and outdoor scene types may include road scene types and park scene types.

[0047] S403: Determine the target area according to the target object.

[0048] Among them, the target area is the area covered by the target object in the image of the real scene.

[0049] In some embodiments, the extended reality device 100 may use a connected region analysis method to determine the target area based on the outline of the target object.

[0050] It can be understood that a connected region refers to a region in an image that is composed of primitive points with the same primitive value and adjacent positions, and connected region analysis refers to marking independent connected regions in an image.

[0051] Specifically, the extended reality device 100 can first convert the image of the real scene into a binary image, then perform connected region analysis on the binary image, mark the outline of the target object in the binary image, and thus determine the target area based on the outline of the target object.

[0052] S404, in response to the projection instruction, capturing the projection content of the virtual scene.

[0053] The projection instruction is used to instruct the extended reality device 100 to project the content of the virtual scene. The projected content may include a two-dimensional or three-dimensional image.

[0054] In some embodiments, the extended reality device 100 may include a touch module and / or a voice module. The touch module may include a button or a touch device. A user may trigger a projection command by pressing a projection button or by clicking or long-pressing a touch device. The voice module may receive a user's voice signal (e.g., "project") and trigger the projection command based on the voice signal.

[0055] In other embodiments, the XR device 100 may receive projection instructions from an external device. For example, the external device may include a console, and the user may first trigger the projection instruction by touching a projection control on the console of the external device. The external device then sends the projection instruction to the XR device 100.

[0056] In this embodiment, the extended reality device 100 responds to the projection instruction and can capture the projection content of the virtual scene from the processor 110 or the memory 120 or the external device through the projection device 130.

[0057] Specifically, an external device sends projection content of a virtual scene to XR device 100. XR device 100 may first receive the projection content via processor 110 or projection device 130. After XR device 100 receives the projection content via processor 110, processor 110 may send the projection content to memory 120 or projection device 130. After processor 110 sends the projection content to memory 120, XR device 100 may store the projection content in memory 120 and then retrieve the projection content from memory 120 via projection device 130.

[0058] S405: Project the projection content or a portion of the projection content outside the target area.

[0059] In this embodiment, after capturing the projection content of the virtual scene, the extended reality device 100 projects the projection content or a portion of the projection content outside the target area, thereby preventing the projection content of the virtual scene from obscuring the target object in the target area.

[0060] FIG5 is a flowchart of sub-steps of step S402 shown in FIG4 .

[0061] As shown in FIG5 , identifying a target object from an image of a real scene may specifically include the following steps:

[0062] S501, identifying target features from an image of a real scene.

[0063] Among them, the target features may include various objects in the image of the real scene, such as the human figure, cup, coaster, table, etc. shown in Figure 2a.

[0064] It's understood that target features are related to scene types. For example, in a restaurant scene, target features might include people, tables, chairs, and dishes. Another example is a road scene, where target features might include vehicles, pedestrians, traffic lights, and zebra crossings. Another example is a conference room scene, where target features might include people, whiteboards or projection screens, tables, notepads, or laptops.

[0065] In some embodiments, the extended reality device 100 can perform feature detection on an image of a real scene to determine target features.

[0066] S502: Determine whether the type of the real scene is a preset scene type based on the target feature.

[0067] In some embodiments, the extended reality device 100 may perform feature matching on target features in an image of a real scene based on features in a preset scene type, thereby determining whether the type of the real scene is the preset scene type.

[0068] In this embodiment, if the type of the actual scene is a preset scene type, steps S503 to S504 are executed; if not, steps S505 to S506 are executed.

[0069] S503: Query the target object database according to the preset scene type.

[0070] The object database is used to store information about objects corresponding to preset scene types. The object information may include the object's name, shape, and color.

[0071] S504: Retrieve an object corresponding to a preset scene type from an object database.

[0072] In this embodiment, if the type of the real scene is a preset scene type, the extended reality device 100 queries the target object database according to the preset scene type and extracts the target object corresponding to the preset scene type from the target object database.

[0073] S505: Identify the target object in the image of the real scene according to the preset rules.

[0074] Among them, the preset rules are used to set the target object from the target features of the real scene.

[0075] For example, referring to FIG2a again, the target features of the real scene may include a human figure, a dining table, chairs, dishes, etc. The preset rule may be to set the human figure as the target object.

[0076] S506: Store the information of the target object corresponding to the type of the real scene into the target object database.

[0077] In this embodiment, if the type of the real scene is not a preset scene type, the extended reality device 100 identifies the target object in the image of the real scene according to the preset rules, and stores the information of the target object corresponding to the type of the real scene in the target object database.

[0078] It is understandable that in other embodiments, the extended reality device 100 may not execute step S506.

[0079] FIG6 is a flowchart of sub-steps of step S405 shown in FIG4 .

[0080] As shown in FIG6 , projecting the projection content or a portion of the projection content outside the target area may specifically include the following steps:

[0081] S601: Determine whether there is any overlapping area between the target area and the projection area.

[0082] In some embodiments, the XR device 100 may employ a connected region analysis method to first determine the image display area based on the contours of the image of the real scene, then determine the target area within the image display area based on the contours of the target object in the image of the real scene, and finally determine the projection area based on the contours of the projected content. The image display area is the area covered by the image of the real scene, and the projection area is the area covered by the projected content.

[0083] It will be appreciated that the position and size of the image display area are determined by processor 110 and can be changed by adjusting the image display parameters of processor 110. The position and size of the projection area are determined by processor 110 and / or projection device 130 and can be changed by adjusting the parameters of projection device 130. The projection area is smaller than or equal to the size of the image display area.

[0084] In this embodiment, if the target area and the projection area have overlapping areas, step S602 is executed; if not, step S603 is executed.

[0085] S602: Project the projection content or a portion of the projection content in a projection area outside the overlapping area.

[0086] In some embodiments, the target area and the projection area overlap, and the extended reality device 100 projects the projection content into the projection area outside the overlapping area. The processor 110 and / or the projection device 130 adjusts the projection position or size of the projection content, and then projects the projection content into the projection area outside the overlapping area via the projection device 130, thereby preventing the projection content from obscuring the target object or a portion of the target object.

[0087] In other embodiments, the target area and the projection area overlap, and the extended reality device 100 projects a portion of the projection content onto the projection area outside the overlapped area. The processor 110 and / or the projection device 130 readjusts the outline of the projection content based on the outline of the overlapped area, and then projects the content onto the projection area outside the overlapped area via the projection device 130, thereby preventing the projection content from obscuring the target object or a portion of the target object.

[0088] S603: Projecting the projection content in the projection area.

[0089] In this embodiment, if the target area and the projection area do not overlap, indicating that the projected content will not obstruct the target object, the extended reality device 100 projects the projection content in the projection area.

[0090] The following takes scenario 3 as an example to specifically illustrate the projection method of the extended reality device provided by this application. Scenario 3

[0091] Please refer to Figures 7a to 7d, where Figure 7a is a schematic diagram of a real scene, Figure 7b is a schematic diagram of identifying a target object from an image of a real scene, Figure 7c is a schematic diagram of determining a target area based on the target object, and Figure 7d is a schematic diagram of projecting content outside the target area.

[0092] First, the extended reality device 100 can capture an image of a real scene using the camera 140. As shown in FIG7a , the real scene is a conference room, and the image includes the meeting host, participants, a whiteboard, a cup, a pen, and a table.

[0093] The extended reality device 100 can then use the processor 110 to identify a target object from the image of the real scene. Specifically, the processor 110 can first identify target features from the image of the real scene, then compare the target features with features of a preset scene type to determine the preset scene type corresponding to the target features. The preset scene type may include one or more targets. Next, based on the preset scene type, the device 100 can query a target database that stores information about targets corresponding to multiple preset scene types to determine the target object corresponding to the preset scene type. As shown in Figure 7b, target features 701 in the image of the real scene include a meeting host, a participant, a whiteboard, a cup, and a pen. The preset scene type corresponding to target feature 701 is a conference room scene type, and the target objects 702 corresponding to the preset scene type are the meeting host and the whiteboard.

[0094] Next, the XR device 100 can use the processor 110 to determine the target area based on the target object. Specifically, the XR device 100 can first convert the image of the real scene into a binary image through the processor 110, then perform connected component analysis on the binary image to mark the outline of the target object in the binary image, thereby determining the target area based on the outline of the target object. As shown in Figure 7c, the target area 703 includes the area covered by the meeting host and the area covered by the whiteboard.

[0095] Finally, the XR device 100 can use the projection device 130 to capture the projected content of the virtual scene from the processor 110, memory 120, or an external device. The projection device 130 then projects the projected content of the virtual scene, or a portion of the projected content, outside the target area. As shown in Figure 7d, the projected content 704 of the virtual scene includes an icon frame, an incoming call notification frame, and a message notification frame. The projected content 704 of the virtual scene is displayed outside the target area 703 and does not obscure the target object 702 within the target area 703.

[0096] The embodiment of the present application further provides a computer-readable storage medium for storing a computer program or code. When the computer program or code is executed by a processor, the projection method of the embodiment of the present application is implemented.

[0097] It is understood that computer-readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0098] The embodiments of the present application are described in detail above with reference to the accompanying drawings. However, the present application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by persons skilled in the art without departing from the purpose of the present application.

[0099] 100: Extended Reality Devices 110: Processor 120: Memory 130: Projection device 140: Camera 701: Target Features 702: Target 703: Target Area 704: Projection content S401-S405, S501-S506, S601-S603: Steps

Claims

1. A projection method for an extended reality device, the projection method comprising: Capture images of real-world scenes; Identify target features from images of the real-world scene, the target features including various objects in the images of the real-world scene; Determine whether the type of the real scene is a preset scene type based on the target features; extract the target object corresponding to the preset scene type; determine the target area based on the target object, the target area being the area covered by the target object in the image of the real scene; respond to the projection command and extract the projection content of the virtual scene; Projecting the content or a portion thereof outside the target area.

2. The projection method as described in claim 1, wherein, The projection method further includes: determining whether the type of the real scene is the preset scene type based on the target features; if the type of the real scene is the preset scene type, querying a target object database based on the preset scene type, the target object database being used to store information about the target object corresponding to the preset scene type; and retrieving the target object corresponding to the preset scene type from the target object database.

3. The projection method as described in claim 2, wherein, After determining whether the type of the real scene is a preset scene type based on the target features, the projection method further includes: if the type of the real scene is not the preset scene type, then identifying the target object in the real scene according to preset rules; and storing the information of the target object corresponding to the type of the real scene in the target object database.

4. The projection method as described in claim 1, wherein, The step of projecting the projection content or a portion of the projection content outside the target area includes: determining whether the target area and the projection area overlap, wherein the projection area is the area covered by the projection content; if the target area and the projection area overlap, then projecting the projection content or a portion of the projection content into the projection area outside the overlapping area.

5. The projection method as described in claim 4, wherein, After determining whether the target area and the projection area overlap, the projection method further includes: if the target area and the projection area do not overlap, then projecting the projection content onto the projection area.

6. The projection method as described in claim 1, wherein, The process of capturing images of a real scene includes: capturing images of the real scene by taking pictures of the real scene with a camera.

7. An extended reality device, comprising a processor, memory, a projection device, and a camera, wherein the extended reality device, by means of the processor, executes a computer program or code stored in the memory to perform the following: capturing an image of a real scene by means of the camera; identifying target features from the image of the real scene by means of the processor, the target features including various objects in the image of the real scene; determining whether the type of the real scene is a preset scene type based on the target features; capturing a target object corresponding to the preset scene type; determining a target area based on the target object, the target area being the area covered by the target object in the image of the real scene; and, in response to a projection command, capturing projection content of a virtual scene by means of the processor; and projecting the projection content or a portion of the projection content outside the target area by means of the projection device.

8. The extended reality device as described in claim 7, wherein, The processor is further configured to determine whether the type of the real scene is the preset scene type based on the target features; if the type of the real scene is the preset scene type, the processor queries the target object database based on the preset scene type and retrieves the target object corresponding to the preset scene type from the target object database, the target object database being used to store the target object corresponding to the preset scene type; if the type of the real scene is not the preset scene type, the processor identifies the target object in the real scene according to preset rules and stores the target object corresponding to the type of the real scene in the target object database.

9. The extended reality device as described in claim 7, wherein, Projecting the projection content or a portion of the projection content outside the target area using the projection device includes: determining, by the processor, whether there is an overlapping area between the target area and the projection area, wherein the projection area is the area covered by the projection content; if there is an overlapping area between the target area and the projection area, then projecting the projection content or a portion of the projection content into the projection area outside the overlapping area using the projection device; if there is no overlapping area between the target area and the projection area, then projecting the projection content or a portion of the projection content into the projection area using the projection device.

10. A computer-readable storage medium for storing a computer program or code, wherein the improvement is that, when the computer program or code is executed by a processor, it implements the projection method as described in any one of claims 1 to 6.

Images