Image display method and apparatus, device, storage medium, and program product

Abstract

The present application provides an image display method and apparatus, a device, a storage medium, and a program product, applied to an intelligent terminal. The method comprises: in response to detecting that a user wears an intelligent terminal, displaying a virtual object, and determining whether a corresponding real position of the virtual object in a real environment falls within a set area; if yes, detecting real objects around the user to determine an occluded object, and detecting whether the relative distance from the occluded object to the real position is not greater than a safety threshold; and if yes, adjusting the display transparency of the virtual object according to the relative distance. According to the present application, when a user wears an intelligent terminal, the distance between an occluded object, among real objects, occluded by a virtual object within a set area and the virtual object is monitored at any time, and transparency processing is performed on the virtual object to which the distance is not greater than a safety threshold, so that the occluded object is presented in front of the user. Therefore, the user can perceive the occluded object in advance and make a response as soon as possible, thereby improving user experience and avoiding injury.

Description

Image display method, device, equipment, storage medium and program product

[0001] The present application claims priority to the Chinese patent application No. 202310218487X, filed on March 8, 2023, entitled "Image display method, device, equipment, storage medium and program product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of extended reality technology, in particular to an image display method, device, equipment, storage medium and program product. BACKGROUND

[0003] At present, the extended reality (XR) technology has gradually entered people's life and various related technical fields. The corresponding augmented reality (AR) and virtual reality (VR) devices block the user's vision, construct a virtual stereoscopic vision environment, and let people participate in the environment to achieve an immersive experience.

[0004] However, since the current extended reality technology blocks the real vision, the user cannot avoid some safety hazards in the real environment when wearing the extended reality device to move, which is easy to cause collision, reduces the user experience, and even causes personal injury.

[0005] SUMMARY

[0006] In order to achieve the above purpose, the present application provides an image display method applied to a smart terminal, comprising:

[0007] In response to detecting that a user wears the smart terminal, a virtual object is displayed, and it is determined whether a real position corresponding to the virtual object in a real environment falls within a set region;

[0008] If yes, a real object around the user is detected, a blocked object in the real object blocked by the virtual object is determined, and it is detected whether a relative distance from the blocked object to the real position is not greater than a safety threshold;

[0009] If yes, the display transparency of the virtual object is adjusted according to the relative distance.

[0010] In some embodiments, the smart terminal is provided with an image sensor;

[0011] The display of the virtual object comprises:

[0012] The environment image around the user is obtained through the image sensor;

[0013] The environment image is displayed, and the virtual object is inserted into the environment image.

[0014] In some embodiments, the image sensor comprises a lidar;

[0015] The acquiring of an image of the environment surrounding the user by the image sensor includes:

[0016] The laser radar is used to perform laser scanning on the user's surrounding environment, and a point cloud image is generated according to the scanning result, and the point cloud image is used as the environment image.

[0017] In some embodiments, detecting whether the relative distance between the obscured object and the actual position is not greater than a safety threshold includes:

[0018] determining, by the image sensor, a first positional relationship between the obscured object and the user;

[0019] The relative distance from the obscured object to the real position is determined based on the first position relationship and the second position relationship between the user and the virtual object, and it is determined whether the relative distance is not greater than the safety threshold.

[0020] In some embodiments, the smart terminal is provided with a thermal sensor;

[0021] The detecting whether the relative distance between the blocked object and the actual position is not greater than a safety threshold includes:

[0022] monitoring the thermal radiation of the shielded object by the thermal sensor, and determining a first positional relationship between the shielded object and the user according to changes in the thermal radiation;

[0023] The relative distance from the obscured object to the real position is determined based on the first position relationship and the second position relationship between the user and the virtual object, and whether the relative distance is not greater than the safety threshold is determined.

[0024] In some embodiments, adjusting the display transparency of the virtual object according to the relative distance includes:

[0025] As the relative distance decreases, the display transparency is increased in proportion;

[0026] When the relative distance is 0, the display transparency is 100%.

[0027] In some embodiments, after adjusting the display transparency of the virtual object according to the relative distance, the method further includes:

[0028] perform object recognition on the occluded object with the relative distance not greater than the safety threshold, and generate text prompt information according to a recognition result.

[0029] In some embodiments, the generating of the text prompt information according to the recognition result comprises:

[0030] determining position information of the occluded object with the relative distance not greater than the safety threshold, and adjusting a display position of the text prompt information according to the position information so that the text prompt information is located beside the corresponding occluded object.

[0031] In some embodiments, after the adjusting of the display transparency of the virtual object according to the relative distance, the method further comprises:

[0032] detecting light intensity of a real environment around the user;

[0033] in response to the light intensity being less than a preset threshold, performing different display on the occluded object with the relative distance not greater than the safety threshold.

[0034] In some embodiments, the different display at least comprises: contouring processing according to an outline of the occluded object, brightness enhancement processing on the occluded object, or displaying a prompt mark beside the occluded object.

[0035] Based on the same idea, the present application further provides an image display device applied to a smart terminal, comprising:

[0036] a determining module configured to, in response to detecting that a user wears the smart terminal, display a virtual object, and determine whether a real position corresponding to the virtual object in a real environment falls within a set region;

[0037] a detecting module configured to, if the real position falls within the set region, detect real objects around the user, and determine an occluded object occluded by the virtual object among the real objects, and detect whether a relative distance from the occluded object to the real position is not greater than a safety threshold;

[0038] a display module configured to, if the relative distance is not greater than the safety threshold, adjust display transparency of the virtual object according to the relative distance.

[0039] Based on the same idea, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of the above embodiments when executing the program.

[0040] Based on the same idea, the present application also provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to implement the method according to any one of the preceding items.

[0041] Based on the same idea, the present application also provides a computer program product comprising computer program instructions for causing a computer to implement the method according to any one of the preceding items when the computer program instructions are run on the computer. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0043] Fig. 1 is a flow diagram of an image display method provided by an embodiment of the present application;

[0044] Fig. 2 is a scene diagram when the occluded object is far away from the virtual object according to an embodiment of the present application;

[0045] Fig. 3 is a top view diagram when the occluded object is far away from the virtual object according to an embodiment of the present application;

[0046] Fig. 4 is a scene diagram when the occluded object is close to the virtual object according to an embodiment of the present application;

[0047] Fig. 5 is a top view diagram when the occluded object is close to the virtual object according to an embodiment of the present application;

[0048] Fig. 6 is a scene diagram when the occluded object overlaps the virtual object according to an embodiment of the present application;

[0049] Fig. 7 is a top view diagram when the occluded object overlaps the virtual object according to an embodiment of the present application;

[0050] Fig. 8 is a structural diagram of an image display device according to an embodiment of the present application;

[0051] Fig. 9 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the embodiments and the drawings.

[0053] It can be understood that, before using the technical solutions disclosed in the embodiments of the present application, the type of personal information involved in the present application, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0054] For example, in response to receiving an active request of a user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium, etc. performing the operation of the technical solutions of the present application according to the prompt information.

[0055] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending prompt information to the user may, for example, be a pop-up window manner, and the prompt information may be presented in the form of text in the pop-up window. In addition, the pop-up window may also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0056] It can be understood that the above notification and obtaining of user authorization process is only illustrative, and does not limit the implementation manner of the present application, and other manners meeting relevant laws and regulations can also be applied to the implementation manner of the present application.

[0057] It can be understood that the data involved in the present technical solutions (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0058] In order to make the purpose, technical solutions and advantages of the present specification more clear and explicit, the present specification is further described in detail below in combination with specific embodiments and with reference to the drawings.

[0059] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood as the usual meaning understood by those skilled in the art to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Include" or "contain" and similar words mean that the elements, objects or method steps listed before the word cover the elements, objects or method steps listed after the word and their equivalents, without excluding other elements, objects or method steps. "Connected" or "connected" and similar words are not limited to physical or mechanical connection, but can include electrical connection, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationship, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0060] As described in the background section, Extended Reality (XR) refers to combining reality with virtuality through a computer to create a virtual environment that can be interacted with by a human. It is also a general term for a variety of technologies such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). By integrating the visual interaction technologies of the three, it brings the "immersion" of seamless conversion between the virtual world and the real world to the experimenter. In XR technology, users can interact with a virtual digital world, providing users with a more immersive experience. VR and AR technologies that allow users to perceive the digital world in an extraordinary way are also becoming increasingly popular. When users use XR devices, they usually wear XR glasses and other devices, allowing users to see the virtual environment and virtual scene generated by the system. This scene can be superimposed on the real scene or only show the virtual scene. However, this can hinder the user's perception of things in the real environment, which may pose some safety hazards to the user. For example, past pedestrians, pets running around the house, and some smart home appliances (sweeping machines, mobile devices, etc.) may collide with the user. This reduces the user experience and even causes personal injury.

[0061] Therefore, the present application provides an image display method, device, equipment, storage medium and program product, which can solve the problem that the user cannot see some safety hazards in the real environment when wearing an extended reality device, so that the user can be aware in advance and respond early to improve the user experience and avoid injury.

[0062] The image display method, device, equipment, storage medium and program product provided by the present application are applied to a smart terminal and include the following steps: in response to detecting that a user wears a smart terminal, displaying a virtual object and determining whether a real location corresponding to the virtual object in a real environment falls within a set region; if so, detecting real objects around the user and determining a blocked object that is blocked by the virtual object among the real objects; detecting whether a relative distance from the blocked object to the real location is not greater than a safety threshold; and if so, adjusting the display transparency of the virtual object according to the relative distance. The present application monitors the distance between the blocked object that is blocked by the virtual object in the set region among the real objects and the virtual object at all times when the user wears the smart terminal. For the virtual object whose distance is not greater than the safety threshold, the present application performs transparent processing on the virtual object, so as to present the blocked object to the user, allowing the user to be aware in advance and respond early to improve the user experience and avoid injury.

[0063] In combination with the actual situation, the embodiment of the present application provides an image display scheme. The present application monitors the distance between the occluded object occluded by the virtual object in the set region in the real object and the virtual object at all times when the user wears the smart terminal. For the virtual object with a distance not greater than a safety threshold, the virtual object is transparently processed, so as to present the occluded object in front of the user, so that the user can be aware in advance and make a response early, improve the user experience, and avoid injury.

[0064] As shown in FIG. 1, a flowchart of an image display method according to the present application is shown. The method is applied to a smart terminal, and specifically includes:

[0065] In step 101, in response to detecting that the user wears the smart terminal, a virtual object is displayed, and it is determined whether the corresponding real position of the virtual object in the real environment falls within a set region.

[0066] In some embodiments, the smart terminal refers to a modern intelligent electronic device capable of certain numerical calculation, logical operation, having storage memory function, and being able to execute corresponding operations according to the user's instructions, such as XR glasses, XR head-mounted devices, mobile phones, iPads, etc. Then, the smart terminal can perform image acquisition operations by setting image sensors, etc. By acquiring external images, these images can be presented on the smart terminal, such as presenting the real environment, or presenting the image of the real object, etc. Among them, the image sensor is a device that can receive external optical signals and can process these optical signals into an image picture, such as a camera and other devices. At the same time, the image sensor can also contain a variety of optical receiving devices, such as ordinary cameras, infrared cameras, laser cameras, etc., which can present different pictures according to different needs.

[0067] After detecting that the user wears the smart terminal, the virtual object can be displayed on the smart terminal for the user to watch. These virtual objects can be some tables, illustrations or virtual objects, characters set by the user, etc. The smart terminal can display these virtual objects at any position in the current display scene. The current display scene can be a completely virtual virtual scene, or a real scene reflecting the surrounding environment of the current user, etc.

[0068] Afterwards, a safety area can be determined for the user in the real environment, i.e., a setting area, the range of which can be adjusted arbitrarily according to the setting of the user, for example, a circle (or sphere) with a radius of 10 meters centered at the user in the real environment can be set as the safety area. The setting of the safety area can be understood as the protection range of the user, and it is considered that only when a thing in the real environment is close to or has entered the safety area, it can affect the user. Afterwards, it can be determined one by one whether the corresponding position of each virtual object displayed is within the safety area. In order to make the virtual object more hierarchical or correspond to the things in the scene (virtual or real) during the display of the virtual object by the intelligent terminal, a corresponding position and distance are generally set for the virtual object. The point corresponding to the virtual object in the real environment can be considered as the real position thereof.

[0069] Step 102, if yes, detecting the real objects around the user, determining the occluded objects occluded by the virtual object among the real objects, and detecting whether the relative distance from the occluded objects to the real position is not greater than a safety threshold.

[0070] In some embodiments, after detecting that the position of a virtual object is within the safety area, the positions of the things in the real environment where the user is located can be detected, so as to determine which of the things are occluded objects occluded by the virtual object according to the relative positional relationship. The occluded objects can be real objects that are partially occluded or totally occluded. The distance between the occluded objects and the point corresponding to the virtual object (the real position) can be calculated. The things in the real environment are real objects, which can be moving things such as walking people, cats, dogs, vehicles, and cleaning robots, or stationary things such as trees and chairs. Since the user wearing the intelligent device can also be in a moving state, the relative positional relationship and occlusion relationship of the things can also change.

[0071] Afterwards, a safety threshold of the relative distance can be set in advance to determine which occluded objects need to be concerned and which are temporarily not needed to be concerned. For example, the safety threshold is set to 5 meters. When the distance between any occluded object and the real position corresponding to the virtual object is less than or equal to 5 meters, it is considered that the occluded object needs to be concerned, and the user needs to be reminded.

[0072] Step 103, if yes, adjusting the display transparency of the virtual object according to the relative distance.

[0073] In some embodiments, after determining that an occluded object needs to be focused on, in order to remind the user, the virtual object involved (i.e. the virtual object that occludes the occluded object) needs to be transparentized. Since the distance between the occluded object and the virtual object is not greater than the safety threshold, it can be considered that the virtual object hinders the user from seeing the occluded object, thereby possibly causing adverse effects to the user. In order to display the occluded object, the occluder (the virtual object) needs to be transparentized, which can adjust the transparency according to the distance between the two, the closer the distance, the higher the transparency of the virtual object, the farther the distance, the lower the transparency of the virtual object. The transparency is the degree of transparency of a thing, which is generally from 0% to 100%, 0% when the thing is completely opaque, and 100% when the thing is completely transparent. Through the transparent processing of the virtual object, the occluded object is exposed, and the user can see the occluded object, thereby making the user aware.

[0074] In some embodiments, as shown in FIGS. 2-7, where A is a user wearing a smart terminal, B is an occluded object in a real object, and C is a virtual object. As shown in FIGS. 2 and 3, when the distance between the occluded object B and the virtual object C is 5 meters, the distance between the two is monitored, and the transparency of the virtual object C is 0% at this time. As shown in FIGS. 4 and 5, when the distance between the occluded object B and the virtual object C is 2.5 meters, the transparency of the virtual object C can be 50% at this time. As shown in FIGS. 6 and 7, when the distance between the occluded object B and the virtual object C is 0 meters, the transparency of the virtual object C can be 100% at this time.

[0075] As can be seen from the above, the image display method provided by the present application is applied to a smart terminal, which includes: in response to detecting that a user wears a smart terminal, displaying a virtual object, determining whether a real position corresponding to the virtual object in a real environment falls within a set region; if so, detecting real objects around the user, determining an occluded object in the real objects that is occluded by the virtual object, detecting whether a relative distance from the occluded object to the real position is not greater than a safety threshold; if so, adjusting the display transparency of the virtual object according to the relative distance. The present application monitors the distance between the occluded object occluded by the virtual object in the set region in the real object and the virtual object at all times when the user wears the smart terminal, transparentizes the virtual object whose distance is not greater than the safety threshold, presents the occluded object to the user, enables the user to be aware in advance, makes a response early, improves the user experience, and avoids injury.

[0076] In some embodiments, the smart terminal is provided with an image sensor; and the displaying of the virtual object includes: acquiring an environment image around the user through the image sensor; and displaying the environment image and inserting the virtual object in the environment image.

[0077] In some embodiments, the smart terminal such as the extended reality device can use a video perspective method to display the scene, that is, the scene in reality can be directly or indirectly viewed through the XR glasses device. That is, it can obtain the environmental image around the user through the image sensor; then, while displaying the environmental image, a virtual object is inserted into it, realizing the display of the scene and things of the extended reality.

[0078] In some embodiments, the image sensor includes a laser radar; and the obtaining the environmental image around the user by the image sensor includes: performing laser scanning on the environment around the user by the laser radar, generating a point cloud image according to the scanning result, and taking the point cloud image as the environmental image.

[0079] The image sensor can be a laser radar, that is, Laser Detecting and Ranging (LiDAR). Similar to the working principle of radar, the laser radar performs image imaging by emitting and receiving laser beams. In this embodiment, after the laser radar completes the scanning of the surrounding environment by emitting and receiving laser beams, it determines the distance by measuring the time difference and phase difference of the received laser signals, measures the angle by horizontal rotation scanning, and establishes a two-dimensional polar coordinate system according to the two parameters, and then obtains the height information in three dimensions by acquiring signals at different pitch angles. High-frequency laser can obtain a large number (about 1.5 million) of position point information (called point cloud) in one second, and three-dimensional modeling is performed according to these information. The image finally completed by three-dimensional modeling is the point cloud image.

[0080] In some embodiments, the detecting whether the relative distance from the occluded object to the real position is not greater than a safety threshold value includes: determining, by the image sensor, a first positional relationship between the occluded object and the user; and determining the relative distance from the occluded object to the real position according to the first positional relationship and a second positional relationship between the user and the virtual object, and determining whether the relative distance is not greater than the safety threshold value.

[0081] In some embodiments, since the user wears the smart terminal, it is easier to determine the positional relationship between the occluded object and the user, such as the distance, the angle, etc., i.e., the first positional relationship is easier to determine. Then, since the virtual object is generally generated by the smart terminal, the relationship between its position and the user is generally fixed, set, or easy to calculate, i.e., the second positional relationship is easier to determine. Then, after obtaining the first positional relationship and the second positional relationship, the triangular relationship formed by the three points is determined, and then the relative distance from the occluded object to the corresponding real position of the virtual object in the real environment can be calculated using the first positional relationship and the second positional relationship. Finally, the size between the safety threshold is determined.

[0082] In some embodiments, the smart terminal is provided with a thermal sensor; the detection of whether the relative distance from the occluded object to the real position is not greater than the safety threshold value includes: monitoring the thermal radiation of the occluded object by the thermal sensor, determining the first positional relationship between the occluded object and the user according to the change of the thermal radiation; determining the relative distance from the occluded object to the real position according to the first positional relationship and the second positional relationship between the user and the virtual object, and determining whether the relative distance is not greater than the safety threshold value.

[0083] In some embodiments, similar to the previous embodiment, in order to facilitate the detection of the temperature of the occluded object, a thermal sensor can be used for temperature monitoring. The thermal sensor, such as a thermal imager, converts the invisible infrared energy emitted by the object into a visible thermal image. Different colors on the thermal image represent different temperatures of the measured object. By viewing the thermal image, the overall temperature distribution of the measured target can be observed, and the heating condition of the target can be studied, so that the next step of work can be judged. The working principle of modern thermal imagers is to use photoelectric devices to detect and measure radiation and establish a correlation between radiation and surface temperature. Then, the pixels of the thermal image can be used to encode the thermal intensity value, which represents the relative amount of thermal energy received by the thermal imager from the object in the environment. The amount of thermal energy received by the thermal imager from the object can vary depending on the distance between the thermal imager and the object. In this way, the first positional relationship can also be quickly determined by the thermal sensor, and the second positional relationship is relatively easy to obtain or set in advance, so that the relative distance is calculated according to the first positional relationship and the second positional relationship, and then compared with the safety threshold. The thermal sensor can be used in scenes with poor light or complex structure.

[0084] In some embodiments, the adjusting the display transparency of the virtual object according to the relative distance comprises: proportionally increasing the display transparency according to a decrease of the relative distance; and when the relative distance is 0, the display transparency is 100%.

[0085] In some embodiments, in order to gradually remind the user as the occluded object approaches, the display transparency can be gradually increased according to a decrease of the relative distance, and the user can gradually see the occluded object, which is conducive to increasing the attention level.

[0086] In some embodiments, after the adjusting the display transparency of the virtual object according to the relative distance, the method further comprises: performing object recognition on the occluded object whose relative distance is not greater than the safety threshold, and generating text prompt information according to the recognition result.

[0087] In some embodiments, in order to further improve the prompting effect, the occluded object can be recognized to determine what specific object is approaching the user. After the recognition is completed, corresponding text prompt information is generated according to the recognition result. For example, if the occluded object is identified as a car, text prompts of a car approaching are generated, which are combined with the gradually faded virtual object and the gradually clear occluded object to remind the user.

[0088] In some embodiments, the generating text prompt information according to the recognition result comprises: determining position information of the occluded object whose relative distance is not greater than the safety threshold, and adjusting a display position of the text prompt information according to the position information, so that the text prompt information is located beside the corresponding occluded object.

[0089] In some embodiments, in order to further improve the prompting effect of the text information, the text prompt information can be displayed beside the occluded object. Since the text prompt information is virtual and the occluded object is real, the display position of the text prompt information generally needs to be determined first. At this time, the position information of the occluded object is determined to determine the display position of the text prompt information, so that the text prompt information is displayed beside the occluded object.

[0090] In some embodiments, after the adjusting the display transparency of the virtual object according to the relative distance, the method further comprises: detecting light intensity of a real environment around the user; and in response to the light intensity being less than a preset threshold, performing different display on the occluded object whose relative distance is not greater than the safety threshold.

[0091] In some embodiments, in order to obviously prompt the user in a relatively dark scene and further improve the prompting effect, the occluded object can be displayed differently. The light intensity reflects the brightness of the real environment, indicating whether the user is currently in a bright environment or a relatively dark environment. In an environment with small light intensity, each thing is difficult to be perceived by the user due to the dark light. Therefore, the outline of the occluded object can be outlined, which can be outlined with a relatively bright color, so as to make it stand out in the scene to remind the user; the image of the occluded object can also be highlighted to improve its brightness, so as to make it stand out in the scene to remind the user; a prompt symbol or prompt identifier can also be generated beside the image of the occluded object to remind the user. That is, the different display at least includes: outline processing according to the outline of the occluded object, brightness improvement processing on the occluded object, or displaying a prompt identifier beside the occluded object.

[0092] It should be noted that the method of the embodiments of the present application can be executed by a single device, such as a computer or a server. The method of the embodiments of the present application can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present application, and the multiple devices can interact with each other to complete the method.

[0093] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than described above and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0094] Based on the same concept, the present application also provides an image display device corresponding to any of the above-mentioned embodiment methods.

[0095] Referring to FIG. 8, the image display device applied to a smart terminal includes:

[0096] The determining module 210 is configured to, in response to detecting that the user wears the smart terminal, display a virtual object, and determine whether a real position corresponding to the virtual object in a real environment falls within a set region.

[0097] The detection module 220 is configured to, if the real position falls within the set region, detect real objects around the user, determine a blocked object in the real objects that is blocked by the virtual object, and detect whether a relative distance from the blocked object to the real position is not greater than a safety threshold.

[0098] The display module 230 is configured to, if the relative distance is not greater than the safety threshold, adjust a display transparency of the virtual object according to the relative distance.

[0099] For the convenience of description, the above apparatus is described in various modules in terms of functions. Of course, the functions of the modules can be implemented in one or more software and / or hardware in the implementation of the embodiments of the present application.

[0100] The apparatus of the above embodiments is used to implement the corresponding image display method in the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

[0101] In some embodiments, the intelligent terminal is provided with an image sensor.

[0102] The determination module 210 is further configured to:

[0103] The image sensor is used to acquire an environment image around the user.

[0104] The environment image is displayed, and the virtual object is inserted into the environment image.

[0105] In some embodiments, the image sensor includes a laser radar.

[0106] The determination module 210 is further configured to:

[0107] The laser radar is used to perform laser scanning on the environment around the user, and a point cloud image is generated according to a scanning result, and the point cloud image is used as the environment image.

[0108] In some embodiments, the detection module 220 is further configured to:

[0109] The image sensor is used to determine a first positional relationship between the blocked object and the user.

[0110] The first positional relationship and a second positional relationship between the user and the virtual object are used to determine the relative distance from the blocked object to the real position, and it is determined whether the relative distance is not greater than the safety threshold.

[0111] In some embodiments, the intelligent terminal is provided with a thermal sensor.

[0112] The detection module 220 is also configured to:

[0113] monitor the heat radiation of the occluded object through the heat-sensitive sensor, and determine a first positional relationship between the occluded object and the user according to a change in the heat radiation;

[0114] determine the relative distance from the occluded object to the real position according to the first positional relationship and a second positional relationship between the user and the virtual object, and determine whether the relative distance is not greater than the safety threshold.

[0115] In some embodiments, the display module 230 is further configured to:

[0116] proportionally increase the display transparency according to a decrease in the relative distance;

[0117] when the relative distance is 0, the display transparency is 100%.

[0118] In some embodiments, the display module 230 is further configured to:

[0119] perform object recognition on the occluded object whose relative distance is not greater than the safety threshold, and generate text prompt information according to a recognition result.

[0120] In some embodiments, the display module 230 is further configured to:

[0121] determine position information of the occluded object whose relative distance is not greater than the safety threshold, adjust a display position of the text prompt information according to the position information, and display the text prompt information beside the corresponding occluded object.

[0122] In some embodiments, the display module 230 is further configured to:

[0123] detect a light intensity of a real environment around the user;

[0124] in response to the light intensity being less than a preset threshold, perform different display on the occluded object whose relative distance is not greater than the safety threshold.

[0125] In some embodiments, the different display at least includes contouring processing according to an outline of the occluded object, brightness improvement processing on the occluded object, or displaying a prompt mark beside the occluded object.

[0126] Based on the same concept, the present application also provides an electronic device corresponding to the method of any of the above embodiments, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image display method of any of the above embodiments.

[0127] FIG. 9 shows a more specific schematic diagram of the hardware structure of an electronic device according to the present embodiment. The device can comprise a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication within the device.

[0128] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the present embodiment.

[0129] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided by the present embodiment are implemented by software or firmware, the related program codes are stored in the memory 1020 and executed by the processor 1010.

[0130] The input / output interface 1030 is configured to connect to an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0131] The communication interface 1040 is configured to connect to a communication module (not shown in the figure) to realize the communication interaction between the present device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0132] Bus 1050 includes a path for transferring information between the various components (e.g., processor 1010, memory 1020, input / output interface 1030, and communication interface 1040) of the device.

[0133] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the embodiments of the present specification, and does not have to contain all the components shown in the figure.

[0134] The electronic device of the above embodiment is used to implement the corresponding image display method in any of the preceding embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0135] Based on the same concept, corresponding to any of the above embodiment methods, the present application also provides a non-transitory computer readable storage medium, which stores computer instructions for causing the computer to execute the image display method according to any one of the above embodiments.

[0136] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0137] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the image display method according to any one of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0138] Based on the same concept, corresponding to the image display method described in any of the above embodiments, the present application also provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the image display method described. Corresponding to the execution subject of each step in each embodiment of the image display method, the processor performing the corresponding step can belong to the corresponding execution subject.

[0139] The computer program product of the above embodiments is used to cause the computer and / or the processor to perform the image display method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0140] It should be understood by those of ordinary skill in the art that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including claims) of the present application is limited to these examples; the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0141] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the present application difficult to understand, the known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, devices can be shown in block diagram form in order to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (i.e. these details should be well within the understanding of those skilled in the art). Where specific details (e.g. circuits) are set forth in order to describe an exemplary embodiment of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be practiced without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than limiting.

[0142] Although the present application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g. dynamic RAM (DRAM)) can use the embodiments discussed.

[0143] Embodiments of the present application are intended to embrace all such alterations, modifications, and variations that fall within the broad scope of the appended claims. Accordingly, any one or more features of a given embodiment are intended to be illustrative only and not limiting of the scope of the application.

Claims

1. An image display method, characterized in that: Applied to smart terminals, including: In response to detecting that the user is wearing the smart terminal, displaying a virtual object, and determining whether a real position corresponding to the virtual object in the real environment falls within a set area; If so, detecting real objects around the user, determining an obscured object in the real objects that is obscured by the virtual object, and detecting whether a relative distance from the obscured object to the real position is not greater than a safety threshold; If so, the display transparency of the virtual object is adjusted according to the relative distance.

2. The method according to claim 1, characterized in that The smart terminal is provided with an image sensor; The displaying of the virtual object comprises: Acquire an image of the environment around the user through the image sensor; The environment image is displayed, and the virtual object is inserted into the environment image.

3. The method according to claim 2, characterized in that The image sensor includes a laser radar; The acquiring the image of the environment around the user by the image sensor includes: The laser radar is used to perform laser scanning on the user's surrounding environment, and a point cloud image is generated according to the scanning result, and the point cloud image is used as the environment image.

4. The method according to claim 2, characterized in that The detecting whether the relative distance between the blocked object and the actual position is not greater than a safety threshold includes: determining, by the image sensor, a first positional relationship between the obscured object and the user; The relative distance from the obscured object to the real position is determined based on the first position relationship and the second position relationship between the user and the virtual object, and it is determined whether the relative distance is not greater than the safety threshold.

5. The method according to claim 1, wherein The intelligent terminal is provided with a thermal sensor; The detecting whether the relative distance between the blocked object and the actual position is not greater than a safety threshold includes: monitoring the thermal radiation of the shielded object by the thermal sensor, and determining a first positional relationship between the shielded object and the user according to changes in the thermal radiation; The relative distance from the obscured object to the real position is determined based on the first position relationship and the second position relationship between the user and the virtual object, and whether the relative distance is not greater than the safety threshold is determined.

6. The method according to claim 1, characterized in that The adjusting the display transparency of the virtual object according to the relative distance includes: As the relative distance decreases, the display transparency is increased in proportion; When the relative distance is 0, the display transparency is 100%.

7. The method according to claim 1, characterized in that After adjusting the display transparency of the virtual object according to the relative distance, the method further includes: Object recognition is performed on the blocked object whose relative distance is not greater than the safety threshold, and text prompt information is generated according to the recognition result.

8. The method according to claim 7, characterized in that Generating text prompt information according to the recognition result includes: Determine the position information of the obscured object whose relative distance is not greater than a safety threshold, and adjust the display position of the text prompt information according to the position information so that the text prompt information is located next to the corresponding obscured object.

9. The method according to claim 1, characterized in that After adjusting the display transparency of the virtual object according to the relative distance, the method further includes: Detecting the light intensity of the real environment around the user; In response to the light intensity being less than a preset threshold, the blocked objects whose relative distance is not greater than a safety threshold are displayed distinctively.

10. The method according to claim 9, characterized in that The distinguishing display at least includes: performing a stroke process according to the outline of the obscured object, performing a brightness enhancement process on the obscured object, or displaying a prompt mark next to the obscured object.

11. An image display device, characterized in that: Applied to smart terminals, including: a determination module, configured to, in response to detecting that a user is wearing the smart terminal, display a virtual object and determine whether a corresponding real position of the virtual object in a real environment falls within a set area; a detection module, configured to detect real objects around the user if the real location falls within the set area, determine obscured objects in the real objects that are obscured by the virtual object, and detect whether a relative distance from the obscured objects to the real location is not greater than a safety threshold; The display module is configured to adjust the display transparency of the virtual object according to the relative distance if the relative distance is not greater than a safety threshold.

12. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 10 is implemented.

13. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to implement the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to perform the method according to any one of claims 1 to 10.

Images