Image display method and device and electronic equipment

Abstract

The invention provides an image display method. The image display method comprises the steps that a first image is determined, the first image represents a collected image of a target area, away from the user side, of the display equipment, and the target area is determined based on the user sight line and the frame contour of the display equipment; and determining a second image, wherein the second image represents a display image of the display equipment except the acquisition image. And fusing the display signals corresponding to the first image and the second image by using a display device and then displaying the fused display signals.

Description

Technical Field

[0001] The present disclosure relates to the field of image display technology, and in particular to an image display method, device and electronic device. Background Art

[0002] With the continuous development of electronic consumer products, various electronic devices have emerged one after another, among which transparent display screens are also one of the hot spots. However, a technical problem faced by transparent display screens is that the display screen and the transparent part compete for space or have different optimization directions, making it difficult to balance the display effect and light transmittance.

[0003] Currently, the main methods used to achieve transparent display effects are mixed reality and augmented reality. Among them, mixed reality is computationally complex and requires high computing power, while augmented reality has a complex optical imaging system, and both are very expensive. Summary of the invention

[0004] One aspect of the present disclosure provides an image display method, including: determining a first image, the first image representing a captured image of a target area of ​​a display device far from a user side, the target area being determined based on the user's line of sight and a frame outline of the display device. Determining a second image, the second image representing a display image of the display device other than the captured image. And using the display device to fuse display signals corresponding to the first image and the second image for display.

[0005] Optionally, determining the first image includes: determining an intersection line between a user's line of sight and at least a portion of a frame outline of a display device. Determining a target area according to an intersection point between the intersection line and an area of ​​the display device away from the user. Using a camera to capture an image of the target area, and intercepting the captured image to obtain the first image.

[0006] Optionally, using a camera to capture an image of the target area, and intercepting the captured image to obtain the first image includes: determining a focal plane where a target object far from the user side is located, and the target object is at least partially located in the target area. Determining an obscured portion of the focal plane where the target object is located according to the target area. Intercepting the captured image corresponding to the obscured portion to obtain the first image.

[0007] Optionally, the user line of sight represents a line between the user's eyes and any point in the target area, and the viewpoint corresponding to the user's line of sight is the midpoint of the line between the user's eyes or one of the end points of the line. The first image is adjusted based on the user's line of sight and / or the viewpoint. The distance between the viewpoint and the target object in the target area is the sum of the focal length between the camera and the target object and the projection distance between the viewpoint and the display device.

[0008] Optionally, when the target object in the target area includes a dynamic target, the method further includes: determining an image similarity threshold. Performing image detection on the first image. When the similarity of the first image of N consecutive frames is greater than or equal to the image similarity threshold, stopping updating the first image. When the similarity of the first image of N consecutive frames is less than the image similarity threshold, updating the first image in real time.

[0009] Optionally, when the target object in the target area is a static target, determining the first image includes: modeling the image captured by the camera to obtain a modeled image; and intercepting the modeled image according to the target area to obtain the first image.

[0010] Optionally, there are multiple cameras, and the types of cameras include wide-angle cameras and macro cameras. Each camera captures a different area in the target area, and the first image is a superposition of image signals corresponding to multiple different areas. The camera adjusts the first image based on the user's line of sight.

[0011] Optionally, the image display method further includes: determining a virtual visual focus of the user's eyes in the target area, and focusing the camera according to the virtual visual focus.

[0012] Another aspect of the present disclosure provides an image display device, including: a first determination module, used to determine a first image, the first image represents a captured image of a target area of ​​a display device far from a user side, the target area being determined based on the user's line of sight and a frame outline of the display device. A second determination module, used to determine a second image, the second image represents a display image of the display device other than the captured image. And a display module, used to fuse and display the display signals corresponding to the first image and the second image using the display device.

[0013] Another aspect of the present disclosure provides an electronic device, including: a display device, configured to fuse and display display signals corresponding to a first image and a second image; a camera, connected to the display device, configured to capture images of a target area of ​​the display device away from a user side; and a capture device, connected to the display device and disposed toward the user side, configured to capture a user's line of sight and / or a user's position. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0015] Figure 1A The application scenario diagram of the image display method according to the embodiment of the present disclosure is schematically shown; Figure 1B A display effect diagram corresponding to the image display method according to an embodiment of the present disclosure is schematically shown;

[0016] Figure 2A A flowchart of an image display method according to an embodiment of the present disclosure is schematically shown; Figure 2B The schematic diagram shows a principle diagram of an image display method according to an embodiment of the present disclosure;

[0017] Figure 3 A flowchart of a method for determining a first image according to an embodiment of the present disclosure is schematically shown;

[0018] Figure 4A A flowchart of a method for determining a first image according to another embodiment of the present disclosure is schematically shown; Figure 4B The schematic diagram shows a principle diagram of a viewpoint conversion method according to an embodiment of the present disclosure;

[0019] Figure 5 A flowchart of an image display method according to another embodiment of the present disclosure is schematically shown;

[0020] Figure 6 A flowchart of a method for determining a first image according to another embodiment of the present disclosure is schematically shown;

[0021] Figure 7 A flowchart of an image display method according to another embodiment of the present disclosure is schematically shown;

[0022] Figure 8 A display effect diagram corresponding to an image display method according to another embodiment of the present disclosure is schematically shown;

[0023] Fig. 9 The structure block diagram of the image display device according to the embodiment of the present disclosure is schematically shown;

[0024] Fig.10 A block diagram schematically shows an electronic device suitable for implementing an image display method according to an embodiment of the present disclosure;

[0025] Fig.11 A block diagram of an electronic device suitable for implementing an image display method according to another embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0027] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0030] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transmit instructions. For example, a computer-readable medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, device, or propagation medium. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tape or hard disk (HDD); optical storage devices, such as compact disk (CD-ROM); memory, such as random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0031] An embodiment of the present disclosure provides an image display method for achieving a transparent display effect. The image display method includes: determining a first image, the first image represents a captured image of a target area of ​​a display device away from a user side, and the target area is determined based on the user's line of sight and the border outline of the display device. Determine a second image, the second image represents a display image of the display device other than the captured image. And use a display device to fuse the display signals corresponding to the first image and the second image and then display them. By capturing an image of the target area corresponding to the user-opposite side of the display device, and fusing it with an image corresponding to the display content of the display device itself, a visual effect can be formed in which the line of sight can see the area behind the display device through the screen. The implementation method of this solution is relatively simple and has high real-time performance, which greatly reduces the cost of achieving a transparent display effect.

[0032] Figure 1A The application scenario diagram of the image display method according to the embodiment of the present disclosure is schematically shown. Figure 1B The display effect diagram corresponding to the image display method according to the embodiment of the present disclosure is schematically shown.

[0033] like Figure 1A As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0034] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 101, 102, 103, such as security applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples). For example, the terminal devices 101, 102, 103 are also equipped with one or more cameras on the back side of the screen (opposite to the user) for collecting images of the back target area.

[0035] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0036] The server 105 may be a server that provides various services, such as a background management server that provides support for websites browsed or applications logged in by users using terminal devices 101, 102, 103 (only as an example). The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices, so that the terminal devices 101, 102, 103 used by the users can achieve the following functions: Figure 1B The transparent display effect shown means that the user can see the target object 106 (such as a vase) placed behind the terminal device 101 through the screen of the terminal device 101 .

[0037] It should be noted that the image display method provided in the embodiment of the present disclosure can generally be executed by the server 105 or by the terminal device. Accordingly, the image display device provided in the embodiment of the present disclosure can generally be set in the server 105. The image display method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the image display device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0038] It should be understood that Figure 1A The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0039] The following will be based on Figure 1A The scenario described in Figure 2~ Figure 8 The image display method of the disclosed embodiment is described in detail.

[0040] Figure 2A The flowchart of the image display method according to the embodiment of the present disclosure is schematically shown. Figure 2B The principle diagram of the image display method according to the embodiment of the present disclosure is schematically shown.

[0041] According to the embodiments of the present disclosure, Figure 2A As shown, the image display method of this embodiment includes, for example, operations S210 to S230.

[0042] In operation S210, a first image is determined, where the first image represents a captured image of a target area of ​​the display device that is far away from a user side, and the target area is determined based on a user line of sight and a frame outline of the display device.

[0043] In this step, a first image is first determined, which is a captured image of a target area of ​​the display device away from the user side. The target area here is determined based on the user's line of sight and the frame outline of the display device, which means that it will change according to the user's viewing angle and the specific shape of the display device.

[0044] In operation S220, a second image is determined, the second image representing a display image of the display device excluding the acquired image.

[0045] Next, a second image is determined, which represents a regular image displayed by the display device other than the captured image, such as the content of a computer desktop or an interface of other application programs.

[0046] In operation S230, a display device is used to fuse display signals corresponding to the first image and the second image and then display them.

[0047] Finally, if Figure 2B As shown, the display device is used to fuse the display signals corresponding to the first image and the second image, and the fused image information is converted into a signal that can be displayed on the corresponding display (according to the display protocol). In this way, the user can simultaneously see the area behind the display device (presented by the first image) and the regular output content of the display device (presented by the second image) on the same screen.

[0048] Image fusion is a key step. Image fusion refers to the use of specific algorithms to combine two or more images into a new image. This process extracts the beneficial information in each channel and retains this information to the maximum extent, ultimately generating an image that has a more comprehensive and clear description of the scene. Image fusion technology is an important technology in the field of modern computer vision and image processing. It fuses multiple images from different sensors, different perspectives or different time points to generate an image with richer information and higher quality.

[0049] In some embodiments, before image fusion is performed, it is usually necessary to pre-process the image, including denoising, contrast enhancement, brightness adjustment, etc., to improve the image quality.

[0050] Image registration is an important step in image fusion, which ensures the spatial consistency of the two images. In this embodiment, since the first image and the second image come from different sources (one is an image captured by a camera, and the other is an output image of a display device), image registration is required to ensure that they can be correctly aligned when fused.

[0051] In the process of image fusion, multiple algorithms can be selected to achieve image fusion. Image fusion algorithms include pixel-level fusion and feature-level fusion, for example.

[0052] Fusion at the pixel level can retain more detailed information in the image, such as edges, textures, etc. This method is conducive to further analysis, processing and understanding of the image, and can expose potential target pixels. In this embodiment, pixel-level fusion can ensure that each pixel in the first image and the second image can be correctly fused into the final image.

[0053] Fusion at the feature level usually involves the extraction and fusion of image features. This method can reduce data redundancy and improve fusion efficiency, but may lose some detail information. In this embodiment, if it is necessary to fuse specific features in the image (such as edges, corners, etc.), the feature-level fusion method can be considered.

[0054] It is understandable that in addition to the above fusion algorithms, other algorithms can be selected according to specific needs, such as image fusion algorithms based on deep learning, etc. These algorithms can further improve the quality and efficiency of image fusion.

[0055] After image fusion is completed, image post-processing is usually required to further improve the image quality, such as removing artifacts generated during the fusion process and adjusting the brightness and contrast of the image.

[0056] In practical applications, the real-time performance of image fusion is also an important consideration. In order to ensure that users can see the fused image in real time, it is necessary to select efficient image fusion algorithms and hardware acceleration technology to shorten the processing time.

[0057] This embodiment uses an innovative image display method to achieve superimposed display of the conventional display image of the display device (such as computer desktop content) and the image of the blocked area of ​​the display device (i.e., the part of the human eye's line of sight blocked by the display device frame). This technology allows users to see the area behind the screen while watching the conventional content on the display device, thereby creating a visual effect of a transparent screen. This effect not only enhances the user's visual experience, but also provides a new development direction for future display technology.

[0058] Figure 3 A flow chart of a method for determining a first image according to an embodiment of the present disclosure is schematically shown.

[0059] According to the embodiments of the present disclosure, Figure 3 As shown, in addition to operations S210 to S230 described above with reference to FIG2 , the method of this embodiment further determines the first image through operations S311 to S313. For the sake of brevity, the description of operations S210 to S230 is omitted here, and the subsequent related method embodiments are similar and will not be repeated.

[0060] In operation S311 , an intersection line between a user's sight line and at least a portion of a frame outline of a display device is determined.

[0061] In this step, gaze tracking technology can be used to determine the direction of the user's gaze. Gaze tracking technology can accurately determine the direction of the user's gaze by analyzing the user's eye movements. Once the gaze direction is determined, its intersection with at least part of the border outline of the display device can be calculated. This usually involves geometric calculations and solving for the intersection of the gaze and the edge of the screen.

[0062] For example, first, make sure that the gaze tracking device (such as an eye tracker) has been properly calibrated and can accurately track the user's eye movements. And guide the user to do a simple gaze tracking adaptation test to ensure that the device can stably capture the user's gaze direction.

[0063] Then, the eye tracking device is started to capture the user's eye movement data in real time. Based on the captured eye movement data, the user's eye direction vector is calculated using an algorithm.

[0064] Finally, the precise size and frame shape information of the display device is obtained, and a three-dimensional model of the display device frame is established. The sight direction vector and the display device frame model are geometrically calculated to solve the intersection of the sight and the frame.

[0065] In operation S312, a target area is determined according to an intersection point between the intersection line and an area of ​​the display device far from the user side.

[0066] According to the intersection of the calculated intersection line and the area of ​​the display device away from the user, a target area can be determined. This area is the part of the user's sight blocked by the display device, but the user hopes to see through the transparent screen. The determination of the target area is crucial for subsequent image capture and fusion, because it determines which image information will be used to generate the first image.

[0067] For example, based on the intersection of the line of sight and the frame, the intersection located in the area of ​​the display device far from the user side can be screened out. Based on the screened intersection, combined with the size and shape of the display device, the boundary of the target area is defined.

[0068] In operation S313, a camera is used to capture an image of the target area, and the captured image is intercepted to obtain a first image.

[0069] After the target area is determined, a high-precision camera is used to shoot the target area. The camera should be placed in a position where it can clearly capture the image of the target area, and its shooting angle and focal length should be precisely adjusted to ensure the quality of the captured image. After the shooting is completed, the portion corresponding to the target area is intercepted from the captured image to obtain the first image.

[0070] Then, the second image is determined, and a display device is used to perform fusion processing on the display signals corresponding to the first image and the second image, and the fused image is displayed.

[0071] This embodiment uses a more accurate and detailed method to determine the first image, that is, the image of the area behind the display device blocked by the user's line of sight. This method enables the image of the blocked area to more accurately reflect the user's perspective, thereby achieving a more realistic transparent screen effect.

[0072] Figure 4A A flow chart of a method for determining a first image according to another embodiment of the present disclosure is schematically shown. Figure 4B The schematic diagram shows a principle diagram of a viewpoint conversion method according to an embodiment of the present disclosure.

[0073] According to the embodiments of the present disclosure, Figure 4A As shown, for example, by operating S4131 to S4133, a camera is used to capture an image of the target area, and the captured image is intercepted to obtain a first image.

[0074] In operation S4131, a focal plane where a target object far from the user side is located is determined, and the target object is at least partially located in a target area.

[0075] In this step, it is necessary to determine the target objects far from the user side, which are at least partially within the previously determined target area. At the same time, it is also necessary to determine the focal plane where these target objects are located. The focal plane can be determined by analyzing the shooting parameters of the camera (such as focal length, shooting distance, etc.) and the actual position of the target object.

[0076] In operation S4132, a blocked portion of a focal plane where the target object is located is determined according to the target area.

[0077] Based on the previously determined target area and the focal plane where the target object is located, the portion of the target object that is blocked by the display device on the focal plane can be further determined. This step is key to capturing the first image because it determines which image information will be used to generate the final first image. This usually involves accurate calculation of the spatial relationship between the target object, the display device, and the camera.

[0078] For example, Figure 4B As shown, firstly, a depth camera or other three-dimensional perception technology is used to determine the position of the target object in three-dimensional space, and obtain the size and shape information of the target object for subsequent calculation.

[0079] Then the precise size and frame shape information of the display is obtained, and a three-dimensional model of the display is built, including the frame and screen part.

[0080] Then, the linear distance between the target object and the display is calculated based on the data provided by the depth camera. The direction vector of the target object relative to the display is determined using the depth camera or eye tracking technology.

[0081] According to the distance between the target object and the display and the focal length of the camera, the focal plane where the target object is located can be calculated.

[0082] Finally, a geometric calculation is performed between the three-dimensional model of the display and the focal plane where the target object is located to obtain the range of the projection of the display frame on the focal plane. This range is the part of the target object that is blocked.

[0083] Considering the influence of the camera's viewing angle, it is necessary to determine the camera's position and shooting angle to ensure that the camera can capture the obscured part of the target object. Then, according to the camera's viewing angle and focal length, the captured image is corrected to eliminate the effects of perspective distortion.

[0084] In the corrected image, the corresponding image area is intercepted according to the range of the previously calculated blocked part, and the intercepted image is preprocessed, such as denoising, contrast enhancement, etc., to improve the image quality.

[0085] In operation S4133, the captured image corresponding to the blocked portion is intercepted to obtain a first image.

[0086] Start the camera to shoot the target object, and ensure that the camera lens is perpendicular to the focal plane where the target object is located to obtain the clearest image. During the shooting process, the camera should be placed in a position where the blocked part of the target object can be clearly captured. After the shooting is completed, the area corresponding to the blocked part is intercepted from the captured image to obtain the first image.

[0087] The second image represents a conventional image displayed by the display device except for the captured image. Once the first image and the second image are determined, they can be fused by the display device and the fused image can be displayed.

[0088] This embodiment uses a more precise and detailed method to determine and obtain the first image, that is, the image of the area behind the display device blocked by the user's line of sight. It not only uses the line of sight tracking technology to determine the target area, but also further considers the determination of the target object and its focal plane, as well as the precise interception of the blocked part. This method enables the first image to more accurately reflect the user's perspective and the actual scene behind the display device, thereby achieving a more realistic transparent screen effect.

[0089] According to an embodiment of the present disclosure, the user's line of sight represents the line between the user's eyes and any point in the target area, and the viewpoint corresponding to the user's line of sight is the midpoint of the line between the user's eyes or one of the end points of the line. The first image is adjusted based on the user's line of sight and / or the viewpoint. The distance between the viewpoint and the target object in the target area is the sum of the focal length between the camera and the target object and the projection distance between the viewpoint and the display device.

[0090] In image display technology, in order to provide a more realistic transparent screen effect, it is necessary to accurately capture the user's line of sight and adjust the display content accordingly. However, there are differences in the line of sight characteristics of different users, especially between normal binocular users and monocular users. Therefore, this embodiment proposes a viewing angle optimization method for these two user groups.

[0091] The user's line of sight is represented as the line between the user's eye and any point in the target area. However, it should be noted that for normal binocular users, the viewpoint corresponding to the user's line of sight should be the midpoint of the line between the user's eyes. For monocular users, since only one eye is involved in line of sight capture, the viewpoint is represented as the endpoint of the line.

[0092] The distance between the viewpoint and the target object in the target area is the sum of the focal length between the camera and the target object and the projection distance between the viewpoint and the display device. The calculation of this distance is crucial for subsequent image adjustment.

[0093] For a normal binocular user, the intersection of the binocular sight and the target area is calculated based on the captured binocular sight direction, and the blocked part of the target area is determined accordingly. Then, the first image is transformed in perspective based on the intersection position and distance information to determine the real scene from the user's perspective.

[0094] For monocular users, due to the particularity of monocular sight, the position and distance information of the viewpoint need to be additionally considered. After capturing the monocular sight, the first image is first scaled and offset according to the viewpoint position and the distance information of the target object. Then, the first image is cropped or filled in combination with the shape and size of the target area to ensure that the image content is consistent with the real scene from the user's perspective.

[0095] After the adjustment of the first image is completed, it is fused with the conventional display image (second image) of the display device. The fusion algorithm can be selected according to the actual application scenario, such as pixel-level fusion, feature-level fusion, etc.

[0096] Finally, the fused image is displayed on the display device to achieve a transparent screen effect.

[0097] Through the method of this embodiment, the viewing angle can be optimized for different user groups (normal binocular users and monocular users). By accurately capturing the user's line of sight and adjusting the display content accordingly, a more realistic transparent screen effect is achieved. This method not only takes into account the differences in the user's line of sight characteristics, but also combines the viewpoint position and distance information to optimize the image display.

[0098] In image display technology, when the target area contains dynamic targets, how to update and display the images of these dynamic targets in real time becomes a key issue. Especially in application scenarios such as transparent screens, real-time performance and accuracy are crucial. Therefore, this embodiment proposes a real-time image update and fusion display method for dynamic objects.

[0099] Figure 5 The flowchart of an image display method according to another embodiment of the present disclosure is schematically shown.

[0100] According to the embodiments of the present disclosure, Figure 5 As shown, in the case where the target object in the target area includes a dynamic target, the image display method of this embodiment, for example, further includes operations S510 to S540.

[0101] In operation S510, an image similarity threshold is determined.

[0102] First, the target area is determined and the dynamic targets in the area are identified. The dynamic targets can be pedestrians, vehicles, animals, and any other objects moving in the target area.

[0103] In order to determine whether the first image needs to be updated in real time, an image similarity threshold needs to be set. This threshold can be adjusted according to the actual application scenario and user needs. For example, in a scenario where high-precision real-time update is required, the threshold can be set lower; while in a scenario where real-time requirements are not high, the threshold can be set higher.

[0104] In operation S520, image detection is performed on the first image.

[0105] Perform image detection on the first image to obtain its key features and information. This can be achieved through image processing methods (such as edge detection, corner detection, etc.) or deep learning-based methods (such as convolutional neural networks, etc.). The purpose of image detection is to provide a basis for subsequent calculation of image similarity.

[0106] In operation S530, when the similarity between the N consecutive first image frames is greater than or equal to the image similarity threshold, the updating of the first image is stopped.

[0107] For N consecutive first images, the similarity between them is calculated. The similarity calculation method can adopt normalized cross correlation (NCC), cosine similarity, hash algorithm, etc. After the similarity is calculated, it is compared with the set image similarity threshold.

[0108] If the similarity of the first images of consecutive N frames is greater than or equal to the image similarity threshold, it means that the content of these images has not changed much, and it can be considered that the dynamic target is in a relatively stable state. At this time, in order to save computing resources and improve display efficiency, the first image can be stopped from being updated.

[0109] In operation S540, when the similarity between N consecutive first image frames is less than an image similarity threshold, the first image is updated in real time.

[0110] If the similarity of the first images of consecutive N frames is less than the image similarity threshold, it means that there is a significant change in the content between these images, and the dynamic target may be moving or undergoing other changes. At this time, the first image needs to be updated in real time to ensure the accuracy and real-time nature of the displayed content.

[0111] When the first image needs to be updated in real time, the target area is re-photographed by the camera, and a new first image is captured. Then, the new first image is fused with the conventional display image (second image) of the display device, and the fused image is displayed on the display device.

[0112] By using the method of this embodiment, real-time image update of dynamic objects can be achieved. When the dynamic object moves or undergoes other changes in the target area, these changes can be captured in time and the displayed content can be updated. At the same time, by setting the image similarity threshold, computing resources can be saved and display efficiency can be improved while ensuring the accuracy and real-time performance of the displayed content.

[0113] In image display technology, when the target object in the target area is a static target, real-time shooting may not be the most efficient or necessary method. In particular, in transparent screen application scenarios, if the position and shape of the static target will not change in a short period of time, it may be more appropriate to adopt a solution that replaces real-time shooting. Therefore, this embodiment proposes a modeling and interception solution for static targets to determine the first image.

[0114] Figure 6 A flow chart of a method for determining a first image according to yet another embodiment of the present disclosure is schematically shown.

[0115] According to the embodiments of the present disclosure, Figure 6 As shown, in the case where the target object in the target area is a static target, the first image is determined, for example, through operations S611 to S612.

[0116] In operation S611, the image captured by the camera is modeled to obtain a modeled image.

[0117] First, the target area is determined and the static targets in the area are identified. Static targets can be, for example, buildings, trees, sculptures, and any other objects with fixed positions and unchanged shapes in the target area.

[0118] Use a camera to shoot the target area and obtain multiple images from different angles and positions. Then, use 3D modeling technology (such as structured light scanning, stereo vision, etc.) to process these images and generate a 3D model of the target area and the static objects inside it. This 3D model will serve as the basis for the subsequent capture of the first image.

[0119] In operation S612, the modeling image is intercepted according to the target area to obtain a first image.

[0120] After obtaining the 3D model, the 3D model is cropped or sliced ​​according to the previously determined target area boundary to obtain a modeling image corresponding to the target area. This modeling image will be used as the first image for subsequent image fusion and display.

[0121] The obtained first image (modeling image) is fused with the conventional display image (second image) of the display device. Finally, the fused image is displayed on the display device to achieve a transparent screen effect.

[0122] Through the method of this embodiment, a solution that replaces real-time shooting can be used to determine the first image for a static target. This method not only avoids the waste of computing resources and storage space that may be caused by real-time shooting, but also can obtain a more accurate and realistic first image through three-dimensional modeling technology. At the same time, since the position and shape of the static target will not change in a short time, this method can ensure the stability and consistency of the first image.

[0123] In image display technology, in order to provide a richer and more detailed visual experience, it is often necessary to capture and display multiple different areas within a target area. At the same time, adjusting the display content according to the user's line of sight can further enhance the user experience. Therefore, this embodiment proposes an image display method based on multiple types of cameras (including wide-angle cameras and macro cameras) and user line of sight adjustment.

[0124] According to an embodiment of the present disclosure, there are multiple cameras, and the types of cameras include wide-angle cameras and macro cameras. Each camera captures different areas in the target area, and the first image is a superposition of image signals corresponding to multiple different areas. The camera adjusts the first image based on the user's line of sight.

[0125] For example, multiple cameras are configured, including a wide-angle camera and a macro camera. The wide-angle camera is used to capture a broad field of view within the target area, while the macro camera is used to capture small or important details within the target area.

[0126] Determine the target area based on actual needs and ensure that each camera can cover different areas within the target area.

[0127] Each camera is activated to capture different areas within the target area. The wide-angle camera captures images of a wide field of view, while the macro camera focuses on specific details.

[0128] The image signals captured by each camera are digitized and transmitted to the image processing system.

[0129] In the image processing system, the image signals captured by each camera are superimposed and processed. Since the wide-angle camera and the macro camera capture different areas, their image signals need to be accurately aligned and fused.

[0130] Through image superposition processing, a complete image including multiple different areas in the target area is generated, that is, the first image.

[0131] Use eye tracking technology to capture the user's gaze direction. This can be achieved using an eye tracking sensor installed on the display device.

[0132] The first image is adjusted according to the direction of the user's sight. For example, when the user's sight is focused on a specific area in the target area, the image details of the area can be magnified; when the user's sight moves, the display content of the first image can be adjusted accordingly.

[0133] The adjusted first image is fused with the conventional display image (second image) of the display device. The fusion algorithm can be selected according to the actual application scenario to ensure that the fused image is natural and coherent.

[0134] Finally, the fused image is displayed on a display device for users to watch.

[0135] Through the method of this embodiment, it is possible to capture and display multiple different areas within the target area, and adjust the displayed content in real time according to the user's line of sight. This method not only improves the richness and detail of image display, but also enhances the interactivity and immersion of user experience.

[0136] In a transparent screen application scenario, a user may need to switch the visual focus between the content on the display screen and the real object behind the screen. For example, in a transparent screen application, a user may want to read information from the screen first and then shift their sight to the real object behind the screen. In order to achieve this natural visual focus switching, this embodiment proposes an image fusion display method based on user sight focus adjustment.

[0137] Figure 7 The flowchart of an image display method according to another embodiment of the present disclosure is schematically shown.

[0138] According to the embodiments of the present disclosure, Figure 7 As shown, the image display method of this embodiment, for example, also includes operations S710 to S720.

[0139] In operation S710, a virtual visual focus of a user's eyes at a target area is determined.

[0140] The target area is determined based on the actual application scenario. The area is determined based on the display screen and the real objects behind it. Eye tracking technology is used to capture the user's line of sight in real time and determine the user's current line of sight focus.

[0141] Based on the direction and depth information of the user's line of sight (which may be estimated by an additional depth sensor or algorithm), the user's virtual visual focus in the target area is determined. This focus can be a point, line, or area, indicating the current focus position of the user's line of sight. The virtual visual focus is the point where the user's line of sight may focus on the object behind the screen, which can be obtained by calculating the relative position of the user's eyes and the object behind the screen.

[0142] In operation S720, the camera is focused according to the virtual visual focus.

[0143] The camera is focused according to the determined virtual visual focus. The focusing process can be achieved by adjusting the focal length or lens position of the camera to ensure that the image of the object behind the screen is clear at the virtual visual focus. After the focusing is completed, the camera captures the image of the object behind the screen and transmits it to the image processing system.

[0144] In the image processing system, the captured image of the object behind the screen is fused with the conventional display image of the display device. The fusion algorithm can be selected according to the actual application scenario to ensure that the fused image is natural and coherent.

[0145] The fused image is displayed on the display device. Since the camera captures the image of the object behind the screen, the user can see a clear image of the object behind the screen through the screen and achieve focused viewing.

[0146] For example, the captured image of the object behind the screen can also be processed and adjusted accordingly according to the user's interactive operations, such as zooming in, zooming out, rotating, etc., to meet the user's viewing needs.

[0147] Through the method of this embodiment, the user can shift his sight from the content on the screen to the objects behind the screen, and achieve clear and focused viewing of these objects. This method not only improves the interactivity and immersion of the displayed content, but also provides users with a richer visual experience. At the same time, since the camera captures the image of the object behind the screen, it can avoid interference and occlusion of the screen content (such as increasing the transparency of the screen content), allowing users to focus more on the objects behind the screen.

[0148] During the image display process, when the object or the user's line of sight changes too quickly, the display device may not be able to capture and update the image in time, resulting in display anomalies such as image blur, tearing or delay. To avoid this, this embodiment proposes an image display method based on image processing time and display device frame rate, which determines a time threshold and stops updating the image when the object or line of sight change time is less than the threshold, thereby ensuring the stability and accuracy of the display effect.

[0149] According to an embodiment of the present disclosure, the image display method further includes: determining a time threshold according to at least one of the image processing time and the frame rate of the display device. When the change time of at least one of the user's eye position and the dynamic target is less than the time threshold, stopping updating the second image.

[0150] First, determine the image processing time of the display device, that is, the time required for the device to receive image data, complete image processing and display the image. This time usually depends on the hardware performance of the device and the efficiency of the image processing algorithm.

[0151] Secondly, determine the frame rate of the display device, that is, the number of image frames that the device can display per second. The higher the frame rate, the more times the device updates the image per second, and the smoother the display effect.

[0152] According to the image processing time and the display device frame rate, calculate a suitable time threshold. This time threshold should be greater than or equal to the image processing time to ensure that there is no tearing or delay during the image update process. At the same time, the time threshold should also take into account the user's visual perception and the device's response time to ensure that the display effect is natural and accurate.

[0153] Use sensors or image processing algorithms to monitor changes in objects or user gazes in real time. This can be achieved through devices such as cameras mounted on display devices, infrared sensors, or eye trackers.

[0154] Calculate the time it takes for an object or line of sight to change, that is, the time it takes for an object to move or for a line of sight to shift. This time can be obtained by comparing the image differences between consecutive frames or the changes in sensor signals.

[0155] The time when the object or sight changes is compared with the time threshold. If the change time is less than the time threshold, it means that the object or sight changes too quickly and the display device may not be able to capture and update the image in time.

[0156] In this case, the updating of the currently displayed image is stopped, that is, the previous frame image is kept unchanged, and the image updating is resumed until the time when the object or the sight line changes is greater than or equal to the time threshold.

[0157] When the time of the object or sight line change is greater than or equal to the time threshold, the image update and display are resumed. At this time, the display device can capture the latest state of the object or sight line and update the display image to reflect these changes.

[0158] The method of this embodiment can effectively avoid display anomalies caused by objects or sight lines changing too quickly. When the object or sight line changes too quickly, the display device will stop updating the image, thereby avoiding the occurrence of image tearing, delay or blur. This method not only improves the stability and accuracy of the display effect, but also enhances the user's visual experience.

[0159] According to an embodiment of the present disclosure, the screen material of the display device may be an opaque material or a transparent material. When the screen material of the display device is an opaque material, the target display image displayed in the non-transparent screen is the obscured image. The target display image may be just the obscured image, or it may be a fusion image of the image captured by the camera and the normal display image. When the screen material of the display device is a transparent material, the target display image displayed in the transparent screen is the image of the object outside the screen. Because there is no obscured image, the target display image may be a fusion image of the image captured by the camera and the regular display image, or it may be a fusion image of the image of the object outside the screen captured by the camera and the regular display image. It appears to be overlapping to the observer, and the display image will always coincide with the object as the viewpoint does not have a ghosting phenomenon.

[0160] For example, there is an opaque display device (such as a common LCD display) placed on the user's desk, with some important documents or decorations behind it. The user hopes that while using the display device, he or she can also conveniently view the items behind it without frequently turning his or her head or moving the screen.

[0161] Since the screen is opaque, a camera is needed to capture the area behind the display device (i.e., the obscured image). This camera can be mounted on the top or side of the back of the display device to ensure that the area of ​​interest to the user is captured.

[0162] At the same time, the display device continues to display its regular image content, such as various applications and files on the computer desktop.

[0163] The first image (the obscured image) captured by the camera is fused with the second image (the regular display image) of the display device. This fusion process can be completed in the internal processor of the display device or through an external computing device. Ultimately, the user will see an image on the screen that combines the object behind and the regular display content.

[0164] Users can clearly see the objects behind the screen while using the display device, as if the screen has become transparent. This effect greatly improves the user's work efficiency and comfort.

[0165] Figure 8 A display effect diagram corresponding to an image display method according to another embodiment of the present disclosure is schematically shown.

[0166] For example, Figure 8 As shown, there is a transparent display device (such as a transparent OLED screen), which is also placed on the user's desk. Since the screen itself is transparent, the user can directly see the objects behind the screen (such as a vase). However, the user still hopes to have a more immersive experience when using the display device, that is, to integrate the content on the screen with the objects behind it in some form.

[0167] Since the screen is transparent, the camera can be used directly to capture objects behind the screen (i.e., images of objects outside the screen). This camera can be installed at or near the edge of the display device to ensure that the area of ​​interest to the user can be captured.

[0168] The display device continues to display its regular image content.

[0169] The first image captured by the camera (the image of the object outside the screen) is fused with the second image of the display device (the conventional display image). This fusion process can be more complicated because it needs to take into account the characteristics of the transparent screen and the angle and position of the user's line of sight. In the end, the user will see a highly fused image without ghosting on the screen.

[0170] Users can enjoy a more immersive visual experience while using transparent display devices. The content on the screen is perfectly integrated with the objects behind it, as if they were a whole. This effect not only improves the user's visual enjoyment, but also enhances the interactivity between the user and the display device.

[0171] Through these two examples, we can see that the image display method can adapt to screens of different materials and provide users with a richer visual experience. Whether it is an opaque screen or a transparent screen, this method can achieve the visual effect of screen transparency or fusion display through clever image acquisition and fusion technology.

[0172] Based on the above method, the present disclosure also provides an image display device. Fig. 9 The image display device is described in detail.

[0173] Fig. 9 The structure block diagram of the image display device according to the embodiment of the present disclosure is schematically shown.

[0174] like Fig. 9 As shown, the image display device 900 of this embodiment includes, for example, a first determination module 910, a second determination module 920 and a display module 930. The device 900 can execute the above-mentioned reference FIG. Figure 7 The method described is used to achieve a transparent display effect of the screen.

[0175] Specifically, the first determination module 910 is used to determine a first image, the first image representing a captured image of a target area of ​​the display device away from the user side, and the target area is determined based on the user's line of sight and the frame outline of the display device. In one embodiment, the first determination module 910 can be used to perform the operation S210 described above, which will not be repeated here.

[0176] The second determination module 920 is used to determine a second image, where the second image represents a display image of the display device other than the captured image. In one embodiment, the second determination module 920 can be used to perform the operation S220 described above, which will not be described in detail herein.

[0177] The display module 930 is used to use a display device to fuse the display signals corresponding to the first image and the second image for display. In one embodiment, the display module 930 can be used to perform the operation S230 described above, which will not be described in detail here.

[0178] It is understandable that the first determination module 910, the second determination module 920 and the display module 930 can be combined in one module, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first determination module 910, the second determination module 920 and the display module 930 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuit, or in a proper combination of software, hardware and firmware. Alternatively, at least one of the first determination module 910, the second determination module 920 and the display module 930 can be at least partially implemented as a computer program module, and when the program is run by a computer, the function of the corresponding module can be executed.

[0179] Fig.10 A block diagram of an electronic device suitable for implementing an image display method according to an embodiment of the present disclosure is schematically shown.

[0180] Another aspect of the present disclosure provides an electronic device 1000, for example, including: a display device 1010, for displaying after fusing the display signals corresponding to the first image and the second image. A camera 1020, connected to the display device 1010, for collecting images of a target area of ​​the display device 1010 away from the user side. A collection device 1030, connected to the display device 1010 and disposed toward the user side, for collecting the user's line of sight and / or the user's position.

[0181] The display device 1010 may be, for example, a high-definition, high-refresh rate liquid crystal display (or a transparent OLED screen or other advanced display device). It is used to fuse the display signals corresponding to the first image (the target area image behind the display device) and the second image (the conventional display image of the display device) and then display them. Through advanced image processing algorithms, the perfect fusion of the two images is achieved, providing users with a clear visual experience.

[0182] The camera 1020 is installed, for example, on the top or side of the display device to ensure that the target area image of the display device away from the user side can be captured. It is used to capture images of the area behind the display device and transmit the captured images to the display device for fusion display. The camera has high resolution and wide angle characteristics to ensure that all areas of interest to the user can be captured.

[0183] The acquisition device 1030 may be, for example, an infrared sensor or an eye tracking sensor (set toward the user side). It is used to collect user line of sight and / or user position information. This acquisition device can track the user's line of sight direction and position changes in real time, and transmit this information to the display device or the processor inside the device. By analyzing this information, the device can automatically adjust the display content and layout to maintain a stable transparent display effect.

[0184] Under opaque screen materials, the image behind the display device is captured by the camera and merged with the regular display image to achieve a transparent screen visual effect. Users can clearly see the objects or environment behind the screen while using the display device.

[0185] With a transparent screen material, users can directly see the target object behind the screen through the screen, and at the same time, the image behind the screen is captured by the camera and fused with the regular display image. In addition, the target object that the user directly sees through the screen and the target object in the fused image also overlap without ghosting, providing users with a good visual experience.

[0186] By collecting information about the user's line of sight and location in real time, the device can automatically adjust the display content and layout.

[0187] For example, when the user's line of sight moves to the other side of the screen, the device can automatically adjust the target display area to maintain a stable transparent display effect. The user's line of sight changes can include changes in the line of sight of both eyes, changes in the line of sight of one eye, changes from both eyes to one eye, etc. The device can automatically track them and adjust the position of the target object in the fused image accordingly.

[0188] For another example, when the user's position changes, such as the distance of the body changes, the head or body moves left or right, etc., the device can also automatically track and adjust the position of the target object in the fused image accordingly.

[0189] Users can highly customize the device according to their needs and preferences, for example, choosing which areas of the image to display, adjusting the transparency of the image, and setting trigger conditions for automatically adjusting the displayed content.

[0190] Devices can use energy-saving technologies, such as dynamic backlight adjustment, intelligent sleep, etc., to reduce energy consumption and extend service life.

[0191] The intelligent transparent fusion display device is suitable for various scenarios where it is necessary to view the display content and the surrounding environment at the same time. For example, in the office, users can conveniently view documents or decorations on the desktop while using the computer; at home, users can pay attention to the dynamics of family members or the scenery outside the window while watching TV programs.

[0192] Fig.11 A block diagram of an electronic device suitable for implementing an image display method according to another embodiment of the present disclosure is schematically shown.

[0193] like Fig.11 As shown, the electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include an onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0194] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1101, ROM 1102 and RAM 1103 are connected to each other through bus 1104. Processor 1101 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0195] According to an embodiment of the present disclosure, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 may further include one or more of the following components connected to the I / O interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.

[0196] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the image display method according to the embodiment of the present disclosure is implemented.

[0197] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus, or a device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 1102 and / or RAM 1103 described above and / or one or more memories other than ROM 1102 and RAM 1103.

[0198] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the image display method provided by the embodiment of the present disclosure.

[0199] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 1101. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0200] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0201] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1101, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0202] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0203] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0204] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0205] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An image display method, comprising: Determine a first image, where the first image represents a captured image of a target area of ​​the display device far from a user side, where the target area is determined based on a user line of sight and a border outline of the display device; Determine a second image, where the second image represents a display image of the display device other than the acquired image; as well as The display device is used to fuse the display signals corresponding to the first image and the second image for display.

2. The image display method according to claim 1, wherein: Determining the first image includes: Determine an intersection line between a user's line of sight and at least a portion of a frame outline of the display device; Determining the target area according to an intersection point between the intersection line and an area of ​​the display device far from a user side; A camera is used to capture an image of the target area, and the captured image is intercepted to obtain the first image.

3. The image display method according to claim 2, wherein: The step of using a camera to capture an image of the target area and intercepting the captured image to obtain the first image comprises: Determine a focal plane where a target object far from a user side is located, wherein the target object is at least partially located in the target area; Determine the blocked portion of the focal plane where the target object is located according to the target area; The captured image corresponding to the blocked portion is intercepted to obtain the first image.

4. The image display method according to claim 2, wherein: The user line of sight represents the line between the user's eyes and any point in the target area, and the viewpoint corresponding to the user's line of sight is the midpoint of the line between the user's two eyes or one of the endpoints of the line; adjusting the first image based on the user line of sight and / or the viewpoint; The distance between the viewpoint and the target object in the target area is the sum of the focal length between the camera and the target object and the projection distance between the viewpoint and the display device.

5. The image display method according to claim 1, wherein: In the case where the target object in the target area includes a dynamic target, the method further includes: Determine an image similarity threshold; performing image detection on the first image; When the similarity of the first image of N consecutive frames is greater than or equal to the image similarity threshold, stop updating the first image; When the similarity of N consecutive frames of the first image is less than the image similarity threshold, the first image is updated in real time.

6. The image display method according to claim 2, wherein: When the target object in the target area is a static target, determining the first image includes: Modeling the image captured by the camera to obtain a modeled image; The modeling image is intercepted according to the target area to obtain the first image.

7. The image display method according to claim 2, wherein: There are multiple cameras, and the types of the cameras include wide-angle cameras and macro cameras; Each of the cameras captures a different area within the target area, and the first image is a superposition of image signals corresponding to a plurality of the different areas; The camera adjusts the first image based on the user's line of sight.

8. The image display method according to claim 4, further comprising: Determine the virtual visual focus of the user's eyes in the target area; The camera is focused according to the virtual visual focus.

9. An image display device, comprising: A first determination module is used to determine a first image, where the first image represents a captured image of a target area of ​​the display device far from a user side, where the target area is determined based on a user line of sight and a frame outline of the display device; A second determining module, configured to determine a second image, where the second image represents a display image of the display device other than the acquired image; as well as The display module is used to use the display device to fuse the display signals corresponding to the first image and the second image for display.

10. An electronic device, comprising: A display device, used for fusing the display signals corresponding to the first image and the second image for display; A camera connected to the display device and used to capture an image of a target area of ​​the display device away from a user side; The collection device is connected to the display device and arranged toward the user side, and is used to collect the user's line of sight and / or the user's position.

Images