Safety boundary adjustment method, display device, display system and computer storage medium

Abstract

The present disclosure relates to the technical field of display, and provides a safety boundary adjustment method, a display device, a display system and a computer storage medium, capable of solving the existing problem of large potential safety hazards for users due to impact of external environments being not considered during the defining of safety boundaries. The adjustment method of the present disclosure comprises: obtaining an environmental image of the surroundings of a display device, the environmental image being acquired by an acquisition apparatus; on the basis of the environmental image at the current moment and a target safety boundary of the display device, determining whether an object is present in an area defined by the target safety boundary; and in response to an object is present in the area defined by the target safety boundary, adjusting the target safety boundary on the basis of position information of the object. User safety can be protected to the maximum extent.

Description

Methods for adjusting security boundaries, display devices, display systems, and computer storage media. Technical Field

[0001] This disclosure belongs to the field of display technology, specifically relating to a method for adjusting security boundaries, a display device, a display system, and a computer storage medium. Background Technology

[0002] VR (Virtual Reality) is a technology that uses computer technology to create and experience virtual worlds. It simulates human senses such as sight, hearing, and touch, allowing users to immerse themselves in a virtual environment and interact with and perceive objects within it. VR technology typically requires specialized equipment, such as VR headsets, gloves, and controllers, to provide an immersive experience. For example, with a VR headset, users wear it and view a virtual scene on a screen, interacting with the virtual environment through head tracking technology and controllers. VR technology has wide applications in gaming, education, training, healthcare, and entertainment. It can provide immersive learning and experiences, helping people better understand and master knowledge and skills, and also deliver more immersive entertainment experiences. As technology continues to develop, the cost of VR technology is gradually decreasing, and its application scenarios are becoming increasingly widespread. Simultaneously, VR technology is constantly evolving and improving, with advancements in resolution, latency, and interaction methods, bringing users a more realistic and comfortable experience.

[0003] When users wear VR to experience a virtual environment, their eyes cannot see the real world around them. To avoid potential bumps or collisions during interaction with the virtual environment, safety boundaries need to be defined to ensure user safety.

[0004] Summary of the Invention

[0005] The present invention aims to at least solve one of the technical problems existing in the prior art, and to provide a method for adjusting security boundaries, a display device, a display system, and a computer storage system.

[0006] Firstly, the technical solution adopted to solve the technical problem of the present invention is a method for adjusting a safety boundary, wherein the adjustment method is applied to a display device, and the adjustment method includes:

[0007] Determine the initial security boundaries of the display device based on the security mode type;

[0008] Acquire environmental images of the surrounding environment of the display device collected by the acquisition device, and determine the target safety boundary based on the acquired environmental images and the initial safety boundary;

[0009] Based on the target security boundary and the current environmental image, determine whether there is an object within the area defined by the target security boundary;

[0010] In response to the presence of an object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

[0011] In some embodiments, the adjustment method further includes: receiving a security mode type selected by the user; when the received security mode type selected by the user is in-place mode, the step of determining the initial security boundary of the display device based on the security mode type includes:

[0012] The initial position information of the display device is determined, and the boundary information of the area within a preset range of the reference point is obtained using the initial position information as a reference point to obtain the initial safety boundary.

[0013] In some embodiments, the adjustment method further includes: receiving a security mode type selected by the user; when the received security mode type selected by the user is a custom mode, the step of determining the initial security boundary of the display device based on the security mode type includes:

[0014] Receive user-defined security boundary information, acquire image information within the user-defined security boundary information, and determine the initial security boundary.

[0015] In some embodiments, determining whether an object exists within the area defined by the target security boundary based on the target security boundary and the current environmental image specifically includes:

[0016] Detect whether there is an object in the current environmental image;

[0017] In response to the presence of an object in the current environmental image, the depth information of the object is obtained to obtain the position information of the object relative to the target safety boundary;

[0018] Based on the location information of the object, determine whether there is an object within the area defined by the target security boundary.

[0019] In some embodiments, the environmental image includes multiple sub-environment images, and the environmental image around the display device acquired by the acquisition device specifically includes:

[0020] The acquisition device captures multiple sub-environment images of the surrounding environment of the display device, wherein some objects overlap between the multiple sub-environment images;

[0021] The environmental image is obtained by stitching together multiple sub-environment images.

[0022] In some embodiments, adjusting the target security boundary based on the location information of the object in response to the presence of an object within the area defined by the target security boundary specifically includes:

[0023] In response to the presence of an object within the area defined by the target security boundary, a first prompt message is generated to alert the user;

[0024] In response to the user not clearing objects within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the objects.

[0025] In some embodiments, the adjustment method further includes:

[0026] Obtain the first distance between the display device and the target security boundary;

[0027] In response to the display device being within the area defined by the target security boundary, and the first distance being less than or equal to a preset distance, a second prompt message is generated to prompt the user.

[0028] In some embodiments, the first prompt information includes display information and voice information; and / or,

[0029] The second prompt message includes display information and voice information.

[0030] In some embodiments, the adjustment method further includes:

[0031] In response to the display device being outside the area defined by the target security boundary, the virtual environment screen is switched to a perspective screen.

[0032] In a second aspect, embodiments of this disclosure also provide a display device, wherein the display device includes: a memory, a processor, and a security boundary setting program stored in the memory and executable on the processor, wherein when the security boundary setting program is executed by the processor, it implements the steps of the security boundary setting method as described in any one of the first aspects above.

[0033] Thirdly, embodiments of this disclosure also provide a display system, which includes a processor and a data acquisition device, the data acquisition device being used to acquire environmental images around the display device;

[0034] The processor is configured to perform the following steps:

[0035] Determine the initial security boundaries of the display device based on the security mode type;

[0036] Acquire environmental images of the display device collected by the acquisition device, and determine the target safety boundary based on the acquired environmental images and the initial safety boundary;

[0037] Based on the target security boundary and the current environmental image, determine whether there is an object within the area defined by the target security boundary;

[0038] In response to the presence of an object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

[0039] In some embodiments, the environmental image includes multiple sub-environment images, and the acquisition device includes multiple acquisition components disposed around the display device.

[0040] Each of the acquisition components is used to acquire the sub-environment images around its respective display device, wherein there is partial overlap of scenery among the multiple sub-environment images;

[0041] The processor is further configured to stitch together multiple sub-environment images to obtain the environment image.

[0042] In some embodiments, the acquisition device includes a projector and a receiver;

[0043] The projector is used to project a light pattern onto the surface of the object to obtain a projected pattern;

[0044] The receiver is configured to receive the light pattern and the projection pattern, and obtain the depth information of the object based on the change of the projection pattern relative to the light pattern.

[0045] In some embodiments, the acquisition device includes a projector and a receiver;

[0046] The projector is used to emit light signals and cause the light signals to be reflected on the surface of the object;

[0047] The receiver is used to receive the reflected signal of the light signal reflected from the surface of the object;

[0048] The receiver is also used to calculate the time of flight of the light signal and the distance between the projector and the object to obtain the depth information of the object.

[0049] In some embodiments, the processor is also configured to receive a security mode type selected by the user;

[0050] When the user selects the in-place security mode, the processor is specifically configured as follows:

[0051] The initial position information of the display device is determined, and the boundary information of the area within a preset range of the reference point is obtained using the initial position information as a reference point to obtain the initial safety boundary.

[0052] In some embodiments, the processor is also configured to receive a security mode type selected by the user;

[0053] When the user selects a custom security mode, the processor is specifically configured as follows:

[0054] Receive user-defined security boundary information, acquire image information within the user-defined security boundary information, and determine the initial security boundary.

[0055] In some embodiments, the processor is specifically configured to perform the following steps:

[0056] In response to the presence of an object within the area defined by the target security boundary, a first prompt message is generated to alert the user;

[0057] In response to the user not clearing objects within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the objects.

[0058] In some embodiments, the processor is further configured to perform the following steps:

[0059] Obtain the first distance between the display device and the target security boundary;

[0060] In response to the display device being within the area defined by the target security boundary, and the first distance being less than or equal to a preset distance, a second prompt message is generated to prompt the user.

[0061] In some embodiments, the display system further includes a handle device for setting initial safety boundaries at initial moments.

[0062] Fourthly, embodiments of this disclosure also provide a computer storage medium, wherein the computer storage medium stores a security boundary setting program, and the security boundary adjustment program, when executed by a processor, implements the steps of the security boundary adjustment method as described in any one of the first aspects above. Attached Figure Description

[0063] Figure 1 is a flowchart of a method for adjusting a security boundary provided in an embodiment of this disclosure;

[0064] Figure 2 is a flowchart of yet another adjustment method provided in an embodiment of this disclosure;

[0065] Figures 3a and 3b are schematic diagrams of the initial security boundaries under two security modes, respectively;

[0066] Figures 4a and 4b are schematic diagrams showing the current security boundary before and after two adjustments;

[0067] Figures 5a-5c are schematic diagrams of three types of prompt information;

[0068] Figure 6 is a structural block diagram of a display system 600 provided in an embodiment of this disclosure. Detailed Implementation

[0069] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0070] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0071] In the relevant technical solutions, the influence of objects in the environment is not considered when defining the security boundary, which poses a significant security risk to users.

[0072] In stationary mode, when defining safety boundaries, the display system requires the user to manually check for objects within a 360° radius. If objects are present, it must ensure they have been cleared to prevent the user's arms, legs, or head from touching them. This clearly does not meet the user's actual needs. Furthermore, if the user has not completely cleared the area, the display system will ignore the objects and still define the safety boundaries automatically, posing a significant safety hazard to the user.

[0073] In custom mode, when the system defines a security boundary, if an object exists within the defined boundary, it will prompt the user to clear the object. If the user does not clear the object, the system will ignore the object and still define the boundary directly, which poses a significant security risk to the user.

[0074] Furthermore, after defining the safety boundary, the display system only determines whether the user (headset position) approaches or crosses the safety boundary during the user experience (in custom modes, it can also determine gestures (handheld controller position)). If the user crosses the safety boundary, the display system alerts the user. However, existing display systems do not consider the possibility of moving objects in the environment entering the safety boundary, such as robot vacuums, cats, dogs, etc. This poses a significant safety hazard to the user.

[0075] In view of the above-mentioned problems, this disclosure provides a method for adjusting the security boundary, which can be applied to display devices.

[0076] Figure 1 is a flowchart of a security boundary adjustment method provided in an embodiment of this disclosure. Figures 4a-4b are schematic diagrams before and after adjusting the current security boundary, respectively. As shown in Figure 1, the adjustment method includes the following steps:

[0077] S100. Determine the initial security boundary of the display device based on the security mode type.

[0078] S101. Acquire environmental images of the display device by the acquisition device, and determine the target safety boundary based on the acquired environmental images and the initial safety boundary.

[0079] S103 determines whether an object exists within the area defined by the target security boundary based on the target security boundary and the current environmental image. If an object exists within the area defined by the target security boundary, then S104 is executed; otherwise, S105 is executed.

[0080] S104. In response to the presence of an object within the area defined by the target security boundary, adjust the target security boundary based on the object's location information.

[0081] S105. If no object exists within the area defined by the target security boundary, then the target security boundary is not processed.

[0082] Specifically, the display device can be a VR device, such as a VR headset, VR gloves, etc., and of course, it can also be other display devices. When users wear the display device to experience a virtual environment, their eyes cannot see the surrounding real world. To avoid potential bumps or collisions during interaction with the virtual environment, a safety solution is needed to ensure user safety. In the adjustment method provided in this disclosure, at the initial moment after the display device is started and before the application is experienced, different methods are typically used to define the initial safety boundary based on the user's selected safety mode type to ensure user safety. The safety mode types include two modes: stationary mode and custom mode. Stationary mode refers to setting the initial safety boundary based on the initial position of the display device at the initial moment; custom mode refers to setting the initial safety boundary based on the user's instructions. The user's instructions can be based on the user's own will or on certain rules; this disclosure does not impose any restrictions on this.

[0083] The environmental image surrounding the display device refers to an image that surrounds the display device 360° and is within a preset range. Objects may exist within the area defined by the initial safety boundary, and active objects may enter the area defined by the initial safety boundary after it is set, thereby affecting user safety. This embodiment of the disclosure uses an acquisition device to obtain an environmental image of the area surrounding the display device to determine whether there are objects affecting user safety within the area defined by the initial safety boundary. If objects affecting user safety exist within the area defined by the initial safety boundary, the initial safety boundary is adjusted to obtain a target safety boundary, thereby removing the objects affecting user safety from the target safety boundary; conversely, if no objects affecting user safety exist within the area defined by the initial safety boundary, the initial safety boundary is used as the target safety boundary.

[0084] It should be noted that the object mentioned in the embodiments of this disclosure refers to any obstacle that affects the safety of the user, other than the user of the display device. The obstacle may include at least one of the following: moving objects such as a robot vacuum cleaner, a cat, or a dog; and at least one of the following: fixed objects such as a table or a chair.

[0085] Optionally, the acquisition device can be a camera or other imaging components. Taking a camera as an example, the camera can be integrated in front of the display device. In this case, the user needs to simply look around in a 360° range so that the acquisition device can capture a complete environmental image of the area around the display device. Alternatively, a ring of cameras can be integrated into the display device to acquire sub-images from various angles around the display device. This way, the user does not need to turn their head to observe, and a 360° environmental image of the area around the display device can still be captured. The acquisition device can also exist independently of the display device.

[0086] The area defined by the target safety boundary of the display device refers to the range of movement that the display device can make while wearing it, i.e., the safety zone of the display device. As shown in Figures 4a and 4b, optionally, the area defined by the target safety boundary can be a regular area or an irregular area, which can be determined according to the position of the object in the specific scene.

[0087] In addition, since there may be moving objects in the external environment, these moving objects may enter the safe zone defined by the target safety boundary at any time. These objects include robot vacuum cleaners, cats, dogs, and even people, which may cause unsafe factors to exist in the originally safe zone and affect the safety of users.

[0088] In the adjustment method provided in this embodiment, the display device acquires a 360° environmental image of its surroundings and detects whether there is an object within the area defined by the target safety boundary based on the environmental image. When the display device detects that there is an object within the area defined by the target safety boundary, it automatically adjusts the target safety boundary according to the specific location information of the object, so as to automatically remove the object from the area defined by the target safety boundary and maximize the safety of the user.

[0089] Figure 2 is a flowchart of another adjustment method provided in an embodiment of this disclosure. Figures 3a and 3b are schematic diagrams of the initial security boundary under two security modes, wherein Figure 3a is a schematic diagram of the initial security boundary in the in-situ mode; and Figure 3b is a schematic diagram of the initial security boundary in the automatic mode.

[0090] As shown in Figures 1 and 2, in some embodiments, the adjustment method includes not only steps S100, S101, S103, S104 and S105, but also: S106, receiving the security mode type selected by the user.

[0091] In some embodiments, when the security mode type selected by the user is the in-place mode, step S100, the step of determining the initial security boundary of the display device based on the security mode type, specifically includes: S1061, determining the initial position information of the display device, and using the initial position information as a reference point, obtaining the boundary information of the area within a preset range of the reference point, and obtaining the initial security boundary.

[0092] Specifically, the display device's location information is also the user's location information. When the security mode type is in-place mode, at the initial moment, the display device uses its initial position as the center and automatically generates an initial security boundary based on boundary information. Optionally, the area defined by the initial security boundary can be a regular area. For example, if the area defined by the initial security boundary is a circular area, the boundary information can be the radius of the circular area; if the area defined by the initial security boundary is a rectangular area, the boundary information can be the side length information of the rectangular area. Optionally, the area defined by the initial security boundary can also be an irregular area, in which case the boundary information can be coordinate information with the initial position of the irregular security boundary as the origin, etc.

[0093] As shown in Figure 3a, taking a circular area defined by the initial safety boundary as an example, the specific process of determining the initial safety boundary in the in-place mode is as follows: Using the user's current location as the center, a first preset value is determined based on the boundary information of the area within the preset range of the reference point. A circular area is then automatically delineated with this first preset value as the radius, serving as the safety area. The boundary of this circular area is the initial safety boundary. Optionally, the first preset value can be determined according to the specific model of the display device. Taking the Pico4 display device as an example, the safety boundary supports three sizes: 1.7m, 2.5m, and 3.5m. That is, the first preset value can be set to 1.7m, 2.5m, or 3.5m. Optionally, the first preset value can also be determined based on the external environment of the display device. For example, if the external environment of the current display device is relatively open, the first preset value can be set larger; conversely, if the external environment of the current display device is relatively complex, the first preset value can be set smaller to ensure user safety. This disclosure does not limit the determination of the first preset value.

[0094] In this embodiment, the display device acquires an image of the external environment. After defining an initial safety boundary, the display device automatically determines whether an object exists within the area defined by the initial safety boundary based on the external environment image. If an object is detected within the area defined by the initial safety boundary, the display device automatically adjusts the initial safety boundary based on the object's specific location information and uses the adjusted initial safety boundary as the target safety boundary to automatically remove the object from the area defined by the target safety boundary. In the prior art, when the display device selects the stationary mode, the user needs to manually check whether there are objects in the surrounding 360° range when automatically defining the initial safety boundary. If objects exist, the user needs to clear them to ensure that objects within the safety area have been cleared to prevent the user's arms, legs, or head from touching any objects. The adjustment method in the prior art obviously does not meet the user's actual needs. Compared with the prior art, for the stationary mode, the adjustment method provided by this embodiment, from the user's perspective, changes the need to actively identify and determine objects to passively receiving reminders, which is more in line with actual usage needs.

[0095] In some embodiments, when the security mode type selected by the user is a custom mode, step S100, determining the initial security boundary of the display device based on the security mode type, specifically includes: S1062, receiving the security boundary information set by the user, obtaining the image information within the security boundary information set by the user, and determining the initial security boundary.

[0096] Specifically, as shown in Figure 3b, in custom mode, when defining the initial safety boundary, the user needs to manually rotate the device once using the handle. The display device then receives the user-defined safety boundary information and defines the initial safety boundary. Preferably, the display device can also acquire image information within the user-defined safety boundary information to determine the initial safety boundary. For example, some display devices also support automatically defining a custom initial safety boundary. Taking the Meta Quest display device as an example, when the user wears the device and rotates it once, the device will collect the depth information of the environment in real time. Using the depth information, objects in the environment can be identified, and a custom safety boundary can be automatically defined based on the user-defined safety boundary information and the object information in the environment. The user then confirms this, for example, by using the handle to adjust the safety boundary and complete the confirmation. Finally, the initial safety boundary of the display device at the initial moment is defined.

[0097] In this embodiment, in custom mode, the display device acquires an external environmental image. Based on this image, the display device automatically determines whether an object exists within the safe area corresponding to the user-defined safety boundary information. If the display device detects an object within the current safe area, it automatically adjusts the safety boundary corresponding to the user-defined safety boundary information based on the object's specific location information, thereby automatically removing the object from the safe area. The adjusted safety boundary is then used as the initial safety boundary. In the prior art, when the display device selects custom mode, it automatically defines the initial safety boundary after receiving the user-defined safety boundary information. If an object is detected within the initial safety boundary, the user needs to remove it. If the user does not remove the object, the initial safety boundary is still defined directly. This results in an object existing within the safe area defined by the initial safety boundary, posing a significant safety hazard to the user. Compared to the prior art, this embodiment can automatically adjust the safety boundary corresponding to the user-defined safety boundary information when an object is detected within the safety boundary, obtaining an initial safety boundary to automatically remove the object from the safe area, thus maximizing user safety.

[0098] As shown in Figures 1 and 2, in some embodiments, step S103, determining whether an object exists within the area defined by the target security boundary based on the current environmental image and the target security boundary of the display device, specifically includes the following steps:

[0099] S1031. Detect whether there is an object in the current environmental image. If no object is found in the current environmental image, proceed to step S105. Do not process the target safety boundary. If an object is found in the current environmental image, proceed to step S1032. Determine whether there is an object within the area defined by the target safety boundary.

[0100] Specifically, the area where the environmental image is located is generally greater than or equal to the area defined by the target security boundary; that is, the area where the environmental image is located includes the area defined by the target security boundary. After the display device acquires the environmental image, it first determines whether there is an object in the environmental image. If there is no object in the environmental image, then there is definitely no object within the area defined by the target security boundary. Conversely, if there is an object in the environmental image, it is necessary to further determine whether the object is located within the area defined by the target security boundary based on the object's specific location.

[0101] In this embodiment of the disclosure, by determining whether there is an object in the environmental image, when it is determined that there is an active object in the environmental image, the display device can detect in real time whether the object in the environment has crossed the target safety boundary, so that when the active object crosses the target safety boundary, the target safety boundary can be adjusted in a timely manner to ensure the user's safety to the greatest extent.

[0102] It is understandable that the display device can also directly acquire the external image of the area defined by the target security boundary and directly determine whether there is an object in the external image of the area defined by the target security boundary.

[0103] In some embodiments, step S1031, detecting whether there is an object in the current environmental image, specifically includes: recognizing the current environmental image based on a preset image recognition algorithm, and determining whether there is an object in the current environmental image based on the recognition result.

[0104] Optionally, the preset image recognition algorithm can be any one of object detection algorithm, object segmentation algorithm, semantic segmentation algorithm, and panoramic segmentation algorithm. Based on this preset image recognition algorithm, it is possible to identify the specific type of object in the environmental image, such as whether the object is a moving object like a cat or dog, or a fixed object like a pillar.

[0105] Object detection refers to identifying and locating targets of interest, such as people, vehicles, and animals, in images or videos. Object detection typically returns the bounding box and category label of the target. Object segmentation refers to separating the target from the background in an image or video, i.e., separating the target from the background. Object segmentation typically returns a mask or contour of the target. Semantic segmentation assigns each pixel in an image or video to a predefined category, such as roads, buildings, and trees. Semantic segmentation does not distinguish between different target instances but divides the entire image into different semantic regions. Panoptic segmentation is a more advanced form of semantic segmentation. It not only segments the image into different semantic regions but also attempts to distinguish between target instances of the same category. Panoptic segmentation typically returns a mask or contour of each target instance. Object detection is the simplest image recognition algorithm, while semantic segmentation is a more complex one, but the recognition results are relatively more accurate.

[0106] Once the display device detects an object, it can further determine the object's specific location information based on the object's depth information. In some embodiments, step S1032 may specifically include: in response to the presence of an object in the current environmental image, acquiring the object's depth information to obtain the location information of the obstacle area relative to the target safety boundary; and determining whether an object exists within the domain defined by the safety boundary at the previous time based on the object's location information.

[0107] Optionally, a camera can be integrated into the display device to obtain the depth information.

[0108] Specifically, since precise location of objects is required, scene depth can be acquired using either active or passive cameras when integrating them. Active cameras directly acquire depth data using structured light or TOF (Time of Flight) depth cameras. Structured light cameras typically consist of a projector and a receiver. The projector projects a light pattern onto the object's surface, and the receiver captures the deformation of this pattern. By analyzing these deformations, the object's depth can be determined. TOF cameras also consist of a projector and a receiver. When a light source emits a beam of light, the light signal reflects off the object's surface and is then received by the receiver. By measuring the time of flight of the light signal, the distance between the light source and the object is calculated. This type of camera provides a complete scene depth map in a single image capture, requires no scanning devices, has a fast imaging speed, low computational load, and is widely used. Passive cameras use binocular cameras to calculate depth data. The principle is to compare the parallax of the same object captured by the left and right cameras, and calculate the distance between the object and the cameras based on the parallax.

[0109] For a camera system with a 90° field of view, four cameras are required if an active camera is used, and eight cameras are required if a passive camera is used.

[0110] In some embodiments, the environmental image includes multiple sub-environmental images. The acquisition of the environmental image around the display device acquired by the acquisition device in step S101 specifically includes: acquiring multiple sub-environmental images acquired by the acquisition device, wherein there is partial overlap of scenery between the multiple sub-environmental images; and stitching the multiple sub-environmental images together to obtain the environmental image.

[0111] Specifically, if a display device integrates a ring of cameras, each camera captures a sub-image of its corresponding area, and multiple sub-images need to be stitched together. Image stitching refers to combining two or more images with partially overlapping scenes to generate a single image with a wider field of view or a 360° panoramic view. This image processing technique provides more information to support various subsequent processing steps and has wide applications in the field of machine vision, such as motion detection and tracking, augmented reality, image jitter removal, resolution enhancement, and video compression.

[0112] In some embodiments, multiple sub-environment images are stitched together to obtain an environment image, specifically including: stitching multiple sub-environment images together based on a first preset algorithm to obtain an environment image.

[0113] In some embodiments, multiple sub-environment images are stitched together based on a first preset algorithm to obtain an environment image. Specifically, this includes: performing image preprocessing on the multiple sub-environment images; performing feature point detection and matching on the processed multiple sub-images respectively; solving the homography matrix and performing perspective transformation; performing image postprocessing; and stitching the multiple sub-images together to obtain a complete environment image.

[0114] Specifically, image preprocessing mainly refers to image enhancement and denoising, used to remove noise from images and enhance the details and texture features of images, thereby improving the accuracy of feature detection; feature point detection and matching refers to extracting key feature points from each attached image, such as corner points, edges, and textures, and determining the correspondence between images; solving the homography matrix and performing perspective transformation refers to solving the mapping relationship between the image to be registered and the reference image by using the image coordinates of the matched feature point pairs, with the aim of transforming the perspective of the image to be registered to the perspective of the reference image, ensuring correct stitching between images, and generating a synthetic image with spatial consistency and a natural appearance; image postprocessing refers to using image fusion technology to eliminate any seams that may exist between images after stitching, thereby improving the quality of image synthesis.

[0115] In this embodiment, multiple sub-environment images are acquired separately, and then fused together based on a first preset algorithm to obtain a complete environment image. This eliminates the need for user cooperation in acquiring environment images, reduces the difficulty of acquiring environment images, improves the efficiency of setting security boundaries, and maximizes user security.

[0116] In some embodiments, as shown in FIG2, step S104, in response to the presence of an object within the area defined by the target security boundary, adjusts the target security boundary based on the location information of the object, specifically including: in response to the presence of an object within the area defined by the target security boundary, generating a first prompt message to prompt the user; in response to the user not clearing the object within the area defined by the target security boundary, adjusting the target security boundary based on the location information of the object.

[0117] Specifically, as shown in Figure 2, if the display device determines that an object exists within the area defined by the target safety boundary, it executes step S107 to generate a first prompt message to remind the user to clear the object. The display device further executes step S108 to determine whether the user has cleared the object within the area defined by the target safety boundary; if the user pays attention to the reminder and clears the object, step S105 is executed; if the user ignores the reminder and insists on not clearing the object within the area defined by the target safety boundary, step S104 is executed: the display device automatically adjusts the target safety boundary, automatically removing the object from the area defined by the target safety boundary, thus maximizing the user's safety.

[0118] Understandably, S108 determines whether the user has cleared objects within the target security boundary area by acquiring the current environmental image again through the display device and performing image recognition on the current environmental image to determine whether objects exist within the area defined by the target security boundary, thus determining whether the user has cleared the objects within the target security boundary area. Of course, the display device can also make this determination in other ways. For example, if the user has cleared the objects, the user could send a command to the display device to inform it, and the display device could then make its own determination based on the received command, and so on.

[0119] Figures 5a-5c are schematic diagrams of three types of prompt information.

[0120] In some embodiments, the first prompt information includes display information and voice information. Optionally, the display information includes any one of displaying line information, displaying grid information, or displaying information in a small window. For example, the display device detects in real time whether an object in the environment has crossed the target safety boundary. If an object enters the safety area, it will alert the user in a small window, as shown in Figure 5c. Of course, it can also alert the user by displaying lines or grids, as shown in Figures 5a and 5b, or by emitting an alarm sound; this disclosure does not limit this.

[0121] In some embodiments, the adjustment method includes not only steps S100, S101, S103, S104, and S105, but also: obtaining a first distance h1 between the display device and the target security boundary; and generating a second prompt message to prompt the user in response to the display device being within the area defined by the target security boundary and the first distance h1 being less than or equal to a preset distance. In some embodiments, the adjustment method includes not only steps S100, S101, S103, S104, and S105, but also: switching the virtual environment screen to a perspective screen in response to the display device being outside the area defined by the target security boundary.

[0122] In some embodiments, the second prompt information includes display information and voice information, wherein the display information includes any one of displaying line information, displaying grid information, or displaying a small window.

[0123] Specifically, as shown in Figures 4a and 4b, the position of the display device is also the user's own position, and the preset distance can be determined according to the specific experience scenario. After the safety boundary is defined, the display device will detect the user's own position in real time. If it detects that the user (headset position) or gesture (handheld position) is close to the boundary and within a certain distance, it will provide a second prompt message such as lines or grids. If the user receives the second prompt message and moves away from the safety boundary, the second prompt message will disappear. If the user ignores the second prompt message and insists on going beyond the target safety boundary, the perspective view will switch to display the surrounding real environment to ensure the user's safety. In addition, the display device automatically pauses the application.

[0124] In some embodiments, the first prompt message and the second prompt message are different. This configuration ensures that the user can react differently to different prompt messages, thereby guaranteeing the user's safety.

[0125] In some embodiments, as shown in Figures 4a and 4b, the minimum distance h2 between the adjusted target security boundary and the object ranges from 10cm to 30cm. This ensures that user safety is maximized without excessively restricting the area defined by the target security boundary.

[0126] The adjustment method provided in this embodiment of the present disclosure allows the display device to acquire an environmental image of a preset 360° range around it. Based on this environmental image, when the display device detects an object within the area defined by the target safety boundary, it automatically adjusts the target safety boundary according to the specific location information of the object, automatically removing the object from the area defined by the target safety boundary, so as to maximize the safety of the user.

[0127] Secondly, based on the same inventive concept, this disclosure also provides a display device, wherein the display device includes: a memory, a processor, and a security boundary adjustment program stored in the memory and executable on the processor, wherein when the security boundary adjustment program is executed by the processor, it implements the steps of the security boundary adjustment method as described in any of the above embodiments.

[0128] Thirdly, based on the same invention, this disclosure also provides a display system. Figure 6 is a structural block diagram of a display system 600 provided in an embodiment of this disclosure. As shown in Figure 6, the display system 600 includes a processor 601 and a data acquisition device 602.

[0129] The processor 601 is configured to perform the following steps: determining the initial security boundary of the display device based on the security mode type; acquiring an environmental image of the area surrounding the display device collected by the acquisition device 602; determining a target security boundary based on the acquired environmental image and the initial security boundary; determining whether an object exists within the area defined by the target security boundary based on the target security boundary and the current environmental image; and adjusting the target security boundary based on the object's location information in response to the presence of an object within the area defined by the target security boundary. In some embodiments, the processor 601 is specifically configured to generate a first prompt message to prompt the user in response to the presence of an object within the area defined by the target security boundary; and to adjust the target security boundary based on the object's location information in response to the user not clearing the object within the area defined by the target security boundary.

[0130] The acquisition device 602 is used to acquire environmental images around the display device and send them to the processor 601. Optionally, the acquisition device 602 can be integrated into the display device, for example, it can be integrated in front of the display device or around the display device. Optionally, the acquisition device 602 can also exist independently and then send the acquired environmental images around the display device to the processor 601 for further processing.

[0131] In some embodiments, the environmental image includes multiple sub-environment images. The acquisition device 602 includes multiple acquisition components disposed around the display device. Each acquisition component is used to acquire its corresponding sub-environment image around the display device. There is partial overlap between the multiple sub-environment images. The processor 601 is also used to stitch the multiple sub-environment images together to obtain an environmental image.

[0132] Specifically, the acquisition device 602 can be a camera or other imaging components. Taking a camera as an example, the camera can be integrated in front of the display device. In this case, the user needs to simply look at the surrounding 360° range so that the acquisition device 602 can capture a complete environmental image. Alternatively, a ring of cameras can be integrated into the display device to acquire sub-environmental images of the display device from various angles. This setup eliminates the need for the user to turn their head to observe, and can still capture a 360° environmental image of the display device.

[0133] In some embodiments, the acquisition device 602 includes a projector and a receiver; the projector is used to project a light pattern onto the surface of an object to obtain a projected pattern; the receiver is used to receive the light pattern and the projected pattern, and obtain the depth information of the object based on the change of the projected pattern relative to the light pattern.

[0134] Specifically, the acquisition device 602 can be a structured light camera, which typically consists of a projector and a receiver. The projector projects a light pattern onto the surface of an object, and the receiver captures the deformation of the pattern. By analyzing these deformations, the depth information of the object can be obtained.

[0135] In some embodiments, the acquisition device 602 includes a projector and a receiver; the projector is used to emit light signals and cause the light signals to be reflected on the surface of an object; the receiver is used to receive the reflected signals of the light signals reflected on the surface of the object; the receiver is also used to calculate the time of flight of the light signals and calculate the distance between the projector and the object to obtain the depth information of the object.

[0136] Specifically, the acquisition device 602 can be a TOF (Time of Flight) depth camera. A TOF camera also consists of a projector and a receiver. When a light source emits a beam of light, the light signal is reflected off the surface of an object and then received by the receiver. By measuring the time of flight of the light signal, the distance between the light source and the object is calculated. This type of camera provides a complete scene depth map through a single imaging operation. It has no scanning device, fast imaging speed, low computational load, and wide application.

[0137] It is understandable that the acquisition device 602 can be any camera other than the structured light camera or TOF camera mentioned above. For example, a binocular camera can also be used to calculate depth data. The principle is to compare the parallax of the same object captured by the left and right cameras and calculate the distance between the object and the camera based on the parallax.

[0138] In some embodiments, the processor 601 is further configured to receive the security mode type selected by the user; when the security mode type selected by the user is the in-place mode, the processor 601 is specifically configured to: determine the initial position information of the display device, and use the initial position information as a reference point to obtain the boundary information of the area within a preset range of the reference point, thereby obtaining the initial security boundary.

[0139] In some embodiments, the processor 601 is further configured to receive a security mode type selected by the user; when the received security mode type selected by the user is a custom mode, the processor 601 is specifically configured to: receive security boundary information set by the user, obtain image information within the security boundary information set by the user, and determine the initial security boundary.

[0140] In some embodiments, the processor 601 is further configured to obtain a first distance between the display device and the target security boundary; and in response to the display device being within the area defined by the target security boundary and the first distance being less than or equal to a preset distance, generate a second prompt message to prompt the user.

[0141] In some embodiments, the display system 600 includes not only a processor 601 and a sampling device 602, but also a handle device 603 for setting initial security boundaries.

[0142] Specifically, in stationary mode, the processor 601 determines its initial position information. Using this initial position information as a reference point, it obtains the boundary information of the area within a preset range of the reference point. The handle device 603 then uses this initial position information as a reference point and sets an initial safety boundary based on the boundary information. The initial position information of the display device is also the user's position information. For example, the handle device 603 uses the user's current position as the center and obtains the radius information based on the boundary information to automatically delineate a circular area as the initial safety boundary.

[0143] In custom mode, the controller device 603 can rotate one full circle according to the user's instructions to define the initial safety boundary. Optionally, in custom mode, the processor 601 receives the user-defined safety boundary information and acquires image information within the user-defined safety boundary information. The controller device 603 then sets the initial safety boundary based on the image information within the user-defined safety boundary information. For example, when the display device is a VR headset, the acquisition device 602 is integrated into the display device. When the user wears the display device and rotates one full circle, the acquisition device 602 acquires the depth information of the environment surrounding the display device. The processor 601 uses the depth information to identify objects in the environment and automatically generates custom initial safety boundary information based on the objects. Then, the controller device 603 sets the initial safety boundary based on the initial safety boundary information. Other details in the display system 600 provided in this embodiment are similar to the details in the embodiments of the above-described safety boundary adjustment method, and will not be repeated here.

[0144] The display system 600 provided in this embodiment integrates a sampling device 602 on the display device. The sampling device 602 can collect environmental images around the display device. The processor 601 detects whether there is an object within the area defined by the target safety boundary based on the environmental image. In response to the presence of an object within the area defined by the target safety boundary, the processor adjusts the target safety boundary to remove the object from the safety area, thereby ensuring the safety of the user.

[0145] Based on the same inventive concept, this disclosure also provides a computer storage medium, wherein the computer storage medium stores a security boundary adjustment program, and when the security boundary setting program is executed by a processor, it implements the steps of the security boundary adjustment method as described in any of the above embodiments.

[0146] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for adjusting a safety boundary, wherein, The adjustment method is applied to a display device, and the adjustment method includes: Determine the initial security boundaries of the display device based on the security mode type; Acquire environmental images of the area surrounding the display device collected by the acquisition device, and determine the target safety boundary based on the acquired environmental images and the initial safety boundary; Based on the target security boundary and the current environmental image, determine whether there is an object within the area defined by the target security boundary; In response to the presence of the object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

2. The adjustment method according to claim 1, wherein, The adjustment method further includes: receiving the security mode type selected by the user; when the security mode type selected by the user is in-place mode, the step of determining the initial security boundary of the display device based on the security mode type includes: The initial position information of the display device is determined, and the boundary information of the area within a preset range of the reference point is obtained using the initial position information as a reference point to obtain the initial safety boundary.

3. The adjustment method according to claim 1, wherein, The adjustment method further includes: receiving the security mode type selected by the user; when the security mode type selected by the user is a custom mode, the step of determining the initial security boundary of the display device based on the security mode type includes: Receive user-defined security boundary information, acquire image information within the user-defined security boundary information, and determine the initial security boundary.

4. The adjustment method according to claim 1, wherein, The step of determining whether the object exists within the area defined by the target security boundary based on the target security boundary and the current environmental image specifically includes: Detect whether the object exists in the current environmental image; In response to the presence of the object in the current environmental image, the depth information of the object is obtained to obtain the position information of the object relative to the target safety boundary; Based on the location information of the object, determine whether the object exists within the area defined by the target security boundary.

5. The adjustment method according to claim 1, wherein, The environmental image includes multiple sub-environment images. The environmental image around the display device acquired by the acquisition device specifically includes: The acquisition device acquires multiple sub-environment images of the area surrounding the display device, wherein some objects overlap between the multiple sub-environment images; The environmental image is obtained by stitching together multiple sub-environment images.

6. The adjustment method according to claim 1, wherein, The step of adjusting the target security boundary based on the object's location information in response to the presence of the object within the area defined by the target security boundary specifically includes: In response to the presence of the object within the area defined by the target security boundary, a first prompt message is generated to notify the user; In response to the user not clearing the object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

7. The adjustment method according to claim 6, wherein, The adjustment method further includes: Obtain a first distance between the display device and the target security boundary; In response to the display device being within the area defined by the target security boundary, and the first distance being less than or equal to a preset distance, a second prompt message is generated to prompt the user.

8. The adjustment method according to claim 7, wherein, The first prompt information includes display information and voice information; and / or, The second prompt message includes display information and voice information.

9. The adjustment method according to any one of claims 1-8, wherein, The adjustment method further includes: In response to the display device being outside the area defined by the target security boundary, the virtual environment screen is switched to a perspective screen.

10. A display device, wherein, The display device includes: a memory, a processor, and a security boundary adjustment program stored in the memory and executable on the processor, wherein the security boundary adjustment program, when executed by the processor, implements the steps of the security boundary adjustment method as described in any one of claims 1 to 9.

11. A display system, wherein, It includes a processor and a data acquisition device, the data acquisition device being used to acquire environmental images around the display device; The processor is configured to perform the following steps: The initial security boundary of the display device is determined based on the security mode type; Acquire the environmental image around the display device collected by the acquisition device, and determine the target safety boundary based on the acquired environmental image and the initial safety boundary; Based on the target security boundary and the current environmental image, determine whether there is an object within the area defined by the target security boundary; In response to the presence of the object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

12. The display system according to claim 11, wherein, The environmental image includes multiple sub-environment images, and the acquisition device includes multiple acquisition components disposed around the display device. Each of the acquisition components is used to acquire the sub-environment image around its respective display device, wherein there is partial overlap of scenery among the multiple sub-environment images; The processor is further configured to stitch together multiple sub-environment images to obtain the environment image.

13. The display system according to claim 11, wherein, The data acquisition device includes a projector and a receiver; The projector is used to project a light pattern onto the surface of the object to obtain a projected pattern; The receiver is configured to receive the light pattern and the projection pattern, and according to the projection pattern... The depth information of the object is obtained by considering the changes in the light pattern.

14. The display system according to claim 11, wherein, The data acquisition device includes a projector and a receiver; The projector is used to emit light signals and cause the light signals to be reflected on the surface of the object; The receiver is used to receive the reflected signal of the light signal reflected from the surface of the object; The receiver is also used to calculate the time of flight of the light signal and the distance between the projector and the object to obtain the depth information of the object.

15. The display system according to claim 11, wherein, The processor is also configured to receive the security mode type selected by the user; When the security mode type selected by the user is in-place mode, the processor is specifically configured as follows: The initial position information of the display device is determined, and the boundary information of the area within a preset range of the reference point is obtained using the initial position information as a reference point to obtain the initial safety boundary.

16. The display system according to claim 11, wherein, The processor is also configured to receive the security mode type selected by the user; When the security mode type selected by the user is a custom mode, the processor is specifically configured as follows: Receive user-defined security boundary information, acquire image information within the user-defined security boundary information, and determine the initial security boundary.

17. The display system according to claim 11, wherein, The processor is specifically configured to perform the following steps: In response to the presence of the object within the area defined by the target security boundary, a first prompt message is generated to notify the user; In response to the user not clearing the object within the area defined by the target security boundary, the target security boundary is adjusted based on the location information of the object.

18. The display system according to claim 11, wherein, The processor is also configured to perform the following steps: Obtain a first distance between the display device and the target security boundary; In response to the display device being within the area defined by the target security boundary, and the first distance being less than or equal to a preset distance, a second prompt message is generated to prompt the user.

19. The display system according to claim 11, wherein, The display system also includes a handle device for setting the initial safety boundary at the initial moment.

20. A computer storage medium, wherein, The computer storage medium stores a security boundary adjustment program, which, when executed by a processor, implements the steps of the security boundary adjustment method as described in any one of claims 1 to 9.

Images