Image display method and device, augmented reality equipment and readable storage medium

Abstract

The invention provides an image display method and device, augmented reality equipment and a readable storage medium, and the method comprises the steps: carrying out the image collection of a real-time visual angle environment, obtaining a first real-time image containing a figure image, and enabling the figure image to have a shielded part; obtaining character feature data corresponding to the character image; restoring the shielded part in the figure image based on the figure feature data to obtain a complete figure image and a second real-time image including the complete figure image; and displaying the second real-time image. Under the condition that the person in the real-time image is sheltered, the corresponding person feature data can be acquired to restore the sheltered part and display the complete image of the person, and the split feeling caused by sheltering is avoided, so that a user can select to display the complete image or the sheltered image based on the actual use requirement, and the user experience is improved. And user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of extended reality, and in particular to an image display method, an apparatus, an extended reality device and a readable storage medium. Background Art

[0002] When using XR (Extended Reality), the surrounding environment can be captured and displayed; however, other characters in the environment may be obscured. For example, if the character is also wearing an extended reality device, the character's face is obscured. In this case, the user cannot directly view the character's facial posture, resulting in a perceptual split and affecting the user experience. Summary of the invention

[0003] The main purpose of the present invention is to propose an image display method, an apparatus, an extended reality device and a readable storage medium, aiming to solve the problem in the prior art that the user experience is affected when a character is blocked.

[0004] To achieve the above object, the present invention provides an image display method, the method comprising the steps of:

[0005] Capturing an image of the real-time viewing environment to obtain a first real-time image including a person image, wherein the person image has an obscured portion;

[0006] Acquire character feature data corresponding to the character image;

[0007] Restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image;

[0008] The second real-time image is displayed.

[0009] Optionally, the acquiring character feature data corresponding to the character image includes:

[0010] Determining a target person corresponding to the person image, and a target device corresponding to the target person;

[0011] A feature acquisition request is sent to the target device to receive complete appearance data corresponding to the target person returned by the target device based on the feature acquisition request, wherein the person feature data includes the complete appearance data.

[0012] Optionally, the acquiring character feature data corresponding to the character image includes:

[0013] Determining a target person corresponding to the person image, and a target device corresponding to the target person;

[0014] Sending a feature acquisition request to the target device to receive identity information corresponding to the target person returned by the target device based on the feature acquisition request;

[0015] Matching the complete appearance data corresponding to the identity information in an image database, wherein the character feature data includes the complete appearance data.

[0016] Optionally, the step of obtaining the character feature data corresponding to the character image includes:

[0017] Capturing images of a target person from multiple angles to obtain multi-view image data, wherein the multi-view image data includes the complete appearance of the target person;

[0018] Creating complete appearance data corresponding to the target person according to the multi-view image data;

[0019] The character feature data including the complete appearance data is generated.

[0020] Optionally, the acquiring character feature data corresponding to the character image includes:

[0021] Get the character image display preference set by the user;

[0022] determining whether the character image display preference indicates displaying a complete character image;

[0023] If the character image display preference indicates displaying a complete character image, character feature data corresponding to the character image is acquired.

[0024] Optionally, restoring the blocked portion of the character image based on the character feature data to obtain a complete character image includes:

[0025] Determining a person's posture of the person image, wherein the person's posture corresponds to a capture viewing angle of the first real-time image;

[0026] Complete appearance data in the character feature data is acquired, and the complete character image is generated according to the complete appearance data and the character posture.

[0027] Optionally, adjusting the model posture of the complete appearance data to the character posture to obtain the complete character image comprises:

[0028] After generating an action matching image according to the complete appearance data and the character posture, determining a target person corresponding to the character image;

[0029] Acquire real-time audio data corresponding to the target person;

[0030] The facial posture of the action matching image is adjusted based on the real-time audio data to obtain the complete character image.

[0031] To achieve the above object, the present invention further provides an image display method, the image display method comprising:

[0032] Capturing images of the real-time viewing environment to obtain a first real-time image including an image of a person;

[0033] Acquire character feature data corresponding to the character image and display flag bits;

[0034] Determine whether the display flag is not displayed;

[0035] If the display flag is not displayed, determining the person position of the person image in the first real-time image based on the person feature data;

[0036] Restoring the image background corresponding to the position of the person to obtain a third real-time image that does not include the image of the person;

[0037] The third real-time image is displayed.

[0038] To achieve the above object, the present invention further provides an image display device, the image display device comprising:

[0039] A first acquisition module is used to acquire images of the real-time viewing environment to obtain a first real-time image including a person image, wherein the person image has an obscured portion;

[0040] A first acquisition module, used to acquire character feature data corresponding to the character image;

[0041] A first restoration module, used for restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image;

[0042] The first display module is used to display the second real-time image.

[0043] To achieve the above-mentioned purpose, the present invention also provides an extended reality device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program, when executed by the processor, implements the steps of the image display method described above.

[0044] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the image display method described above are implemented.

[0045] The present invention proposes an image display method, device, extended reality device and readable storage medium, which collect images of a real-time viewing environment to obtain a first real-time image containing a person image, wherein the person image has an obscured portion; obtain person feature data corresponding to the person image; restore the obscured portion of the person image based on the person feature data to obtain a complete person image, and a second real-time image containing the complete person image; and display the second real-time image. When a person is obscured in the real-time image, the corresponding person feature data can be obtained to restore the obscured portion, displaying the complete image of the person, avoiding the sense of fragmentation caused by the obscuration, so that the user can choose to display the complete image or the obscured image based on actual usage needs, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 It is a flowchart of a first embodiment of an image display method of the present invention;

[0049] Figure 2 A schematic diagram of an application scenario of the image display method of the present invention;

[0050] Figure 3 It is a schematic diagram of the overall process of the image display method of the present invention;

[0051] Figure 4 It is a schematic diagram of the module structure of the extended reality device of the present invention. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0053] The present invention provides an image display method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of an image display method of the present invention, wherein the method comprises the following steps:

[0054] Step S10, performing image acquisition on the real-time viewing environment to obtain a first real-time image including a person image, wherein the person image has an obscured portion;

[0055] It can be understood that the image display method of the present invention is applied to extended display devices, such as augmented reality AR devices, virtual reality VR devices, mixed reality MR devices, etc.; in this embodiment and subsequent embodiments, AR devices are used as examples for explanation, and other types of devices can be implemented by analogy and will not be repeated here.

[0056] See also Figure 2 The real-time viewing environment is the area included in the real-time acquisition viewing angle of the camera of the AR device. It can be understood that when the user uses the AR device, the posture of the AR device changes with the user's movements, and at the same time, the area that can be collected by the camera on the AR device changes, that is, the real-time viewing environment changes.

[0057] The camera of the AR device acquires an image to obtain a first real-time image; it can be understood that the first real-time image is obtained by acquiring an image of the real-time viewing environment.

[0058] When there is a person in the real-time perspective environment, the first real-time image collected includes a person image; it is understandable that the number of person images can be one or more, and the number of person images is consistent with the number of obscured people in the real-time perspective environment. When the number of person images is multiple, the subsequent method can be executed for each person image to obtain its corresponding complete person image; it is understandable that this embodiment is to restore the obscured part of the person, therefore, when the person in the real-time perspective environment is not obscured, there is no need to restore the obscured part, therefore, if there is a complete human portrait in the first real-time image, it can be directly displayed, and the corresponding part is not a person image, such as H2; it should be noted that in this embodiment and subsequent embodiments, the completeness described for the image refers to the part of the person facing the perspective at a specific perspective, that is, when there is no obstacle between the person and the VR device, the image of the person can be captured, and it does not refer to the complete body image of the person; and the part corresponding to the person with the obscured part is used as the person image, such as H1. Furthermore, in practical applications, when a person's face is obscured, there will generally be communication barriers and visual fragmentation problems. Therefore, when determining a person's image, the part of the person whose face is obscured in the first real-time image can be used as the person's image, while for the portrait with other parts obscured, it can be directly displayed, and the corresponding part is not the person's image. At the same time, the subsequent restoration operation is for the person's face, and does not need to pay attention to the situation where other parts of the person are obscured.

[0059] Step S20, obtaining character feature data corresponding to the character image;

[0060] The character image is obtained by collecting the character in the real-time viewing environment; the character feature data is data reflecting the appearance features of the character corresponding to the character image; specifically, the character feature data at least includes the appearance features of the occluded part of the character corresponding to the character image;

[0061] The specific form of the character feature data can be set based on actual needs, such as image, model, and parameters.

[0062] It should be noted that in actual applications, users may need to display the complete character image or not. Users can set whether to display the complete character image based on actual needs through the provided trigger method. When the complete character image does not need to be displayed, it can be displayed directly based on the first real-time image. When the complete character image needs to be displayed, the character feature data is obtained, and the second real-time image is obtained and displayed based on subsequent operations.

[0063] Step S30, restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image;

[0064] It can be understood that since the character feature data corresponds to the character image and reflects the character's appearance features, the character feature data can be used to know the occluded parts in the character image, and then the occluded parts can be restored through the character feature data to obtain a complete character image; the character in the complete character image does not have the problem of being occluded.

[0065] After obtaining the complete character image, the second real-time image can be obtained; it can be understood that the part outside the character image in the second real-time image is consistent with the first real-time image, while the character image part is the complete character image; that is, the second real-time image is an image obtained by restoring the obscured part of the character image based on the first real-time image.

[0066] Step S40: displaying the second real-time image.

[0067] After the second real-time image is obtained, the second real-time image may be displayed, so that the user can see the complete image of the person in the second real-time image.

[0068] This embodiment can obtain corresponding character feature data to restore the occluded part when a person is occluded in the real-time image, thereby displaying a complete image of the person and avoiding the sense of fragmentation caused by the occlusion. This allows users to choose to display the complete image or the occluded image based on actual usage needs, thereby improving user experience.

[0069] Further, see also Figure 2 In the second embodiment of the image display method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:

[0070] Step S21, determining a target person corresponding to the person image, and a target device corresponding to the target person;

[0071] Step S22, sending a feature acquisition request to the target device to receive the complete appearance data corresponding to the target person returned by the target device based on the feature acquisition request, wherein the person feature data includes the complete appearance data.

[0072] The character image is obtained by capturing the image of the target person; therefore, before obtaining the character feature data, it is necessary to first determine the target person corresponding to the character image; the method of determining the target person can be set based on actual needs, such as analyzing the appearance and posture of the target person to determine the identity of the target person; or by establishing a connection with the target device, determining the target person corresponding to the target device, and then confirming the target device worn by the character image to determine the target person wearing the target device; or by the user to determine the target person corresponding to the character image, and determine the specific identity of the target person through the user's selection.

[0073] In this embodiment, the complete appearance data is obtained through the target device corresponding to the target person. The target device is a device used by the target person; the target device is such as a VR device or a mobile phone; in this embodiment, the target device is a VR device as an example for description.

[0074] The target device pre-stores the complete appearance data of its user, that is, the target person; the complete appearance data can be obtained by the target device performing all-round image acquisition on the target person, and can also be obtained from other devices storing the complete appearance data of the target person. It can be understood that the complete appearance data reflects the complete appearance of the target person, so in this embodiment, the character feature data including the complete appearance data is set so that any obscured part in the character image can be restored.

[0075] The feature acquisition request indicates the acquisition of the target person's feature data. It can be understood that the feature acquisition request contains indication information for the target person. When it is necessary to obtain the feature data of the target person, the feature acquisition request is sent to the target device. After receiving the feature acquisition request, the target device first generates a request reminder based on the feature acquisition request, and displays the request reminder to let the target person know that the feature data of the target person is requested to be obtained at this time. When the target person agrees, the target person is determined according to the indication information in the feature acquisition request, and the complete appearance data corresponding to the target person is returned to the image display device as the feature data of the target person. When the target person disagrees, a rejection signal is returned to the image display device, and the image display device cannot obtain the feature data of the target person from the target device. When communicating with the target device, it can be achieved through but not limited to Bluetooth, wireless LAN, server forwarding and the like.

[0076] It is understandable that in actual applications, the image display device may not be able to directly obtain the complete appearance of the target person. Therefore, in this embodiment, the character feature data is obtained by sending a feature acquisition request to the target device corresponding to the target person, so that the source of the character feature data can be increased and it can be applied to more application scenarios.

[0077] Further, in a third embodiment of the image display method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:

[0078] Step S23, determining a target person corresponding to the person image, and a target device corresponding to the target person;

[0079] Step S24, sending a feature acquisition request to the target device to receive identity information corresponding to the target person returned by the target device based on the feature acquisition request;

[0080] Step S25, matching the complete appearance data corresponding to the identity information in the image database, wherein the character feature data includes the complete appearance data.

[0081] In actual applications, due to limitations on the device's storage space and specific settings, the target device may not directly store the character feature data corresponding to the target person; therefore, an image database is set up in this embodiment, and the image database stores complete appearance data corresponding to different characters; the image database can be set up on a cloud server.

[0082] When it is necessary to obtain character feature data, a feature acquisition request is sent to the target device. After receiving the feature acquisition request, the target device first generates a request reminder based on the feature acquisition request, and displays the request reminder to let the target person know that the character feature data is requested to be obtained at this time. When the target person agrees, the target person is determined according to the indication information in the feature acquisition request. When the cloud server stores the character feature data of the target person, the identity information corresponding to the target person is returned to the image display device. After receiving the identity information, the image display device sends a request instruction containing the identity information to the cloud server. After receiving the request instruction, the cloud server determines the complete appearance data corresponding to the identity information and returns the complete appearance data to the image display device. When the target person disagrees, a rejection signal is returned to the image display device, and the image display device cannot directly or indirectly obtain the character feature data from the target device. When communicating with the cloud server, it can be achieved through but not limited to mobile data networks, wireless networks, etc.

[0083] Further, in a fourth embodiment of the image display method of the present invention proposed based on the first embodiment of the present invention, the step before step S20 includes the following steps:

[0084] Step S50, collecting images of the target person from multiple angles to obtain multi-view image data, wherein the multi-view image data includes the complete appearance of the target person;

[0085] Step S60, creating complete appearance data corresponding to the target person according to the multi-view image data;

[0086] Step S70, generating the character feature data including the complete appearance data.

[0087] In this embodiment, the character feature data is obtained by directly creating complete appearance data of the target person; specifically, when there is no obstruction between the target person and the image display device, the image display device captures the image of the target person, and the user moves around the target person, so that the relative position between the image display device and the target person changes, so as to capture images of the target person from different perspectives; the multi-perspective image data includes image data captured of the target person from different perspectives; after the complete appearance of the target person is captured from different perspectives, the complete appearance data of the target person can be created based on the captured multi-perspective image data; the specific method of creating the complete appearance data can be set based on actual needs, such as multimodal AI; in order to improve the accuracy of the complete appearance data, the number of images contained in the multi-perspective image data can be set to be larger, and multi-eye cameras, TOF cameras, etc. that can record spatial information can also be used to capture images of the target person.

[0088] After the complete appearance data is generated, the complete appearance data is stored in the image display device and can also be uploaded to the cloud server at the same time; if the character feature data needs to be obtained, the complete appearance data stored locally can be directly obtained without obtaining it through other means.

[0089] It should be noted that in the aforementioned embodiments, the character feature data can be obtained through the target device or the cloud server. After the character feature data is obtained in this way, the character feature data can be stored in the image display device; when the character feature data of the same target person needs to be obtained subsequently, the character feature data in the image display device can be directly obtained without repeatedly requesting the target device.

[0090] Further, in a fifth embodiment of the image display method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:

[0091] Step S26, obtaining the character image display preference set by the user;

[0092] Step S27, determining whether the character image display preference indicates displaying a complete character image;

[0093] Step S28: If the character image display preference indicates displaying a complete character image, obtaining character feature data corresponding to the character image.

[0094] The character image display preference indicates the display status of the target character that the user wants in the real-time image; the character image display preference is set by the user based on his / her own needs.

[0095] If the character image display preference indicates displaying the complete character image, it is considered that the user wants to see the complete appearance of the target character. Therefore, character feature data corresponding to the character image is obtained, and subsequent operations are performed to restore the obscured portion of the target character.

[0096] If the character image display preference indicates displaying the original image, it is considered that the user wants to see the actual state of the target character in the real scene. Therefore, the first real-time image is displayed at this time, and the blocked part of the target character is not restored.

[0097] In this embodiment, the character image display preference is set so that the user can select the display method based on his / her own needs.

[0098] Further, in a sixth embodiment of the image display method of the present invention proposed based on the first embodiment of the present invention, step S30 includes the steps of:

[0099] Step S31, determining a person's posture of the person image, wherein the person's posture corresponds to a capture viewing angle of the first real-time image;

[0100] Step S32, obtaining complete appearance data in the character feature data, and generating the complete character image according to the complete appearance data and the character posture.

[0101] The character posture is the real-time posture of the target character corresponding to the character image; it is understandable that the complete appearance data can only reflect the appearance characteristics of the target character, but cannot reflect the action characteristics of the target character. Therefore, in this embodiment, the character posture of the character image is obtained to clarify the action characteristics of the target character. The character posture can be determined by performing posture recognition on the character image, and the specific recognition method can be set based on actual needs.

[0102] After determining the character's posture, the action characteristics of the target character are determined, and the complete appearance data reflects the appearance characteristics of the target character. Therefore, through the complete appearance data and the task posture, a complete character image consistent with the target character's action can be obtained, realizing the restoration of the occluded part of the character image.

[0103] Furthermore, the step S32 includes the steps of:

[0104] Step S321, after generating an action matching image according to the complete appearance data and the character posture, determining a target person corresponding to the character image;

[0105] Step S322, obtaining real-time audio data corresponding to the target person;

[0106] Step S323, adjusting the facial posture of the action matching image based on the real-time audio data to obtain the complete character image.

[0107] The character posture mainly refers to the torso movement of the target character, and in addition, the facial posture of the target character also needs to be determined; it is understandable that when the face of the character image of the target character is blocked, such as when the target character wears a VR device, the facial posture of the target character cannot be determined by image recognition; therefore, in this embodiment, the facial posture of the target character is determined by obtaining the real-time audio data of the target character; the real-time audio data is the data generated by the target character speaking obtained through audio collection. The real-time audio data can be collected through the microphone on the VR device.

[0108] It is understandable that different speech content and speaking tone of the target person will affect the target person's facial posture. Therefore, real-time audio data can at least determine the facial posture that conforms to the facial characteristics of the target person when the real-time audio data is generated; thereby, the obtained facial posture can match the actual speaking situation of the target person.

[0109] After generating an action matching image based on the complete appearance data and the character posture, the action of the action matching image matches the target character. Afterwards, the facial posture of the action matching image is adjusted by the isomorphic real-time audio data so that the facial posture of the action matching image matches the target character, thereby obtaining a complete character image whose action and facial posture both match the target character.

[0110] Specifically, analysis of real-time audio data can obtain specific speaking content and speaking tone, and facial posture can be determined based on speaking content and speaking tone. The specific implementation method can be set based on actual needs, such as multimodal AI.

[0111] It should be noted that, considering the performance of the image display device, the generation of a complete character image can be directly performed by the locally set multimodal AI, and the character posture and complete appearance data can also be sent to the cloud server and generated by the multimodal AI of the cloud server.

[0112] It should be noted that the image display method of the present invention is applied to a scene where a specific person is occluded; similarly, it can also be applied to a scene where a specific object is occluded. The implementation method of the object can be similar to the implementation method of the above-mentioned person, and will not be repeated here.

[0113] The present invention provides an image display method, the method comprising the steps of:

[0114] Step S110, collecting images of the real-time viewing environment to obtain a first real-time image including a person image;

[0115] Step S120, obtaining character feature data corresponding to the character image and a display flag;

[0116] Step S130, determining whether the display flag is not displayed;

[0117] Step S140, if the display flag is not displayed, determining the person position of the person image in the first real-time image based on the person feature data;

[0118] Step S150, restoring the image background corresponding to the position of the person to obtain a third real-time image that does not include the image of the person;

[0119] Step S160: displaying the third real-time image.

[0120] The display flag is used to indicate whether the character image is displayed or not; it is understandable that the display flag is set by the user; the user can set the display flag for a specific character; it is understandable that in the actual use of the AR device, when there are people in the use environment, the collected real-time image will include the character image, and when the real-time image contains the character image, it may affect the immersive experience of the AR device. Therefore, the user can set the display flag of the character, thereby eliminating the corresponding character image from the real-time image and improving the user's usage experience.

[0121] When the display flag indicates display, the character image is allowed to be displayed in the first real-time image; at this time, the second real-time image can be generated based on the method of the above embodiment or the first real-time image can be directly displayed;

[0122] When the display flag indicates not to display, it is considered that the person image needs to be eliminated from the first real-time image.

[0123] It should be noted that there may be multiple character images in the first real-time image, and the user can set corresponding display flags for different character images respectively.

[0124] It can be seen from the above description that the character feature data is data reflecting the appearance features of the character corresponding to the character image; therefore, the character feature data can be used to accurately identify the position of the character from the first real-time image, that is, the character position; specifically, the character position is a set of pixels where the character is located in the first real-time image; the character image can be eliminated by eliminating the character position; it can be understood that if the pixels corresponding to the character position are directly deleted, the original character image will be displayed as missing and the integrity of the picture will be destroyed. Therefore, in order to ensure the integrity of the picture, in this embodiment, the image background corresponding to the character position is restored, so that the environmental background is displayed in part of the original character image, so that the third real-time image can have picture integrity.

[0125] The specific method of restoring the image background can be set based on actual needs, such as analogously to the method of restoring the obscured part of the character image in the aforementioned embodiment; analogizing the character position to the obscured part of the character image, and analogizing the background to the character image, to achieve restoration of the background.

[0126] After the third real-time image is obtained, the third real-time image may be displayed, so that the third real-time image viewed by the user does not contain the image of the person.

[0127] In this embodiment, the user's immersive experience can be enhanced by eliminating the character image.

[0128] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0130] The present application also provides an image display device for implementing the above-mentioned image display method, the image display device comprising:

[0131] A first acquisition module is used to acquire images of the real-time viewing environment to obtain a first real-time image including a person image, wherein the person image has an obscured portion;

[0132] A first acquisition module, used to acquire character feature data corresponding to the character image;

[0133] A first restoration module, used for restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image;

[0134] The first display module is used to display the second real-time image.

[0135] When a person is obscured in a real-time image, the image display device can obtain corresponding character feature data to restore the obscured part, thereby displaying a complete image of the person and avoiding the sense of fragmentation caused by the occlusion. This allows users to choose to display the complete image or the obscured image based on actual usage needs, thereby improving user experience.

[0136] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first acquisition module in this embodiment can be used to execute step S20 in the embodiment of the present application, the first restoration module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the first display module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0137] Furthermore, the first acquisition module includes:

[0138] A first determining unit, configured to determine a target person corresponding to the person image, and a target device corresponding to the target person;

[0139] The first sending unit is used to send a feature acquisition request to the target device to receive the complete appearance data corresponding to the target person returned by the target device based on the feature acquisition request, wherein the person feature data includes the complete appearance data.

[0140] Furthermore, the first acquisition module includes:

[0141] A second determining unit, configured to determine a target person corresponding to the person image, and a target device corresponding to the target person;

[0142] A second sending unit, configured to send a feature acquisition request to the target device, so as to receive identity information corresponding to the target person returned by the target device based on the feature acquisition request;

[0143] The first matching unit is used to match the complete appearance data corresponding to the identity information in the image database, wherein the character feature data includes the complete appearance data.

[0144] Furthermore, the device also includes:

[0145] A second acquisition module is used to acquire images of a target person from multiple angles to obtain multi-view image data, wherein the multi-view image data includes the complete appearance of the target person;

[0146] A first creation module, used to create complete appearance data corresponding to the target person according to the multi-view image data;

[0147] The first generating module is used to generate the character feature data including the complete appearance data.

[0148] Furthermore, the first acquisition module includes:

[0149] A first acquisition unit, used to acquire a character image display preference set by a user;

[0150] A first determination unit, configured to determine whether the character image display preference indicates displaying a complete character image;

[0151] The second acquisition unit is configured to acquire character feature data corresponding to the character image if the character image display preference indicates displaying a complete character image.

[0152] Furthermore, the first restoration module includes:

[0153] a third determining unit, configured to determine a person's posture of the person image, wherein the person's posture corresponds to a capture viewing angle of the first real-time image;

[0154] The third acquisition unit is used to acquire the complete appearance data in the character feature data, and generate the complete character image according to the complete appearance data and the character posture.

[0155] Furthermore, the third acquisition unit includes:

[0156] A first adjustment subunit is used to determine a target person corresponding to the person image after generating an action matching image based on the complete appearance data and the person posture;

[0157] A first acquisition subunit is used to acquire real-time audio data corresponding to the target person;

[0158] The first adjustment subunit is used to adjust the facial posture of the action matching image based on the real-time audio data to obtain the complete character image.

[0159] The present application also provides an image display device for implementing the above-mentioned image display method, the image display device comprising:

[0160] A second acquisition module is used to acquire images of the real-time viewing environment to obtain a first real-time image including a person image;

[0161] A second acquisition module, used to acquire character feature data corresponding to the character image and a display flag;

[0162] A first judging module, used for judging whether the display flag is not displayed;

[0163] A first determination module, configured to determine a person position of the person image in the first real-time image based on the person feature data if the display flag is not displayed;

[0164] A second restoration module is used to restore the image background corresponding to the position of the person to obtain a third real-time image that does not contain the image of the person;

[0165] The second display module is used to display the third real-time image.

[0166] The image display device can enhance the user's immersive experience by eliminating the character image.

[0167] It should be noted that the second acquisition module in this embodiment can be used to execute step S110 in the embodiment of the present application, the second acquisition module in this embodiment can be used to execute step S120 in the embodiment of the present application, the first judgment module in this embodiment can be used to execute step S130 in the embodiment of the present application, the first determination module in this embodiment can be used to execute step S140 in the embodiment of the present application, the second restoration module in this embodiment can be used to execute step S150 in the embodiment of the present application, and the second display module in this embodiment can be used to execute step S160 in the embodiment of the present application.

[0168] Reference Figure 4 In terms of hardware structure, the extended reality device may include components such as a communication module 10, a memory 20, and a processor 30. In the extended reality device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0169] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other extended reality devices, servers or Internet of Things devices, such as televisions, etc.

[0170] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as image acquisition of a real-time viewing environment), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0171] The processor 30 is the control center of the extended reality device. It uses various interfaces and lines to connect various parts of the entire extended reality device. By running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, it performs various functions of the extended reality device and processes data, thereby monitoring the extended reality device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0172] although Figure 4 Although not shown, the above-mentioned extended reality device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 4 The extended reality device structure shown in the figure does not constitute a limitation of the extended reality device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0173] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 4The memory 20 in the extended reality device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a TV, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0174] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0175] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0176] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An image display method, characterized in that: The image display method comprises: Capturing an image of the real-time viewing environment to obtain a first real-time image including an image of a person, wherein the image of the person has an obscured portion; Acquire character feature data corresponding to the character image; Restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image; The second real-time image is displayed.

2. The image display method according to claim 1, wherein: The acquiring of character feature data corresponding to the character image comprises: Determining a target person corresponding to the person image, and a target device corresponding to the target person; A feature acquisition request is sent to the target device to receive complete appearance data corresponding to the target person returned by the target device based on the feature acquisition request, wherein the person feature data includes the complete appearance data.

3. The image display method according to claim 1, wherein: The acquiring of character feature data corresponding to the character image comprises: Determining a target person corresponding to the person image, and a target device corresponding to the target person; Sending a feature acquisition request to the target device to receive identity information corresponding to the target person returned by the target device based on the feature acquisition request; Matching the complete appearance data corresponding to the identity information in an image database, wherein the character feature data includes the complete appearance data.

4. The image display method according to claim 1, wherein: The step of obtaining the character feature data corresponding to the character image includes: Capturing images of a target person from multiple angles to obtain multi-view image data, wherein the multi-view image data includes the complete appearance of the target person; Creating complete appearance data corresponding to the target person according to the multi-view image data; The character feature data including the complete appearance data is generated.

5. The image display method according to claim 1, wherein: The restoring the blocked part of the character image based on the character feature data to obtain a complete character image comprises: Determining a person's posture of the person image, wherein the person's posture corresponds to a capture viewing angle of the first real-time image; Complete appearance data in the character feature data is acquired, and the complete character image is generated according to the complete appearance data and the character posture.

6. The image display method according to claim 5, characterized in that: Generating the complete character image according to the complete appearance data and the character posture comprises: Generate an action matching image according to the complete appearance data and the character posture, and determine a target person corresponding to the character image; Acquire real-time audio data corresponding to the target person; The facial posture of the action matching image is adjusted based on the real-time audio data to obtain the complete character image.

7. An image display method, characterized in that: The image display method comprises: Capturing images of the real-time viewing environment to obtain a first real-time image including an image of a person; Acquire character feature data corresponding to the character image and display flag bits; Determine whether the display flag is not displayed; If the display flag is not displayed, determining the person position of the person image in the first real-time image based on the person feature data; Restoring the image background corresponding to the position of the person to obtain a third real-time image that does not include the image of the person; The third real-time image is displayed.

8. An image display device, characterized in that: The image display device comprises: A first acquisition module is used to acquire images of the real-time viewing environment to obtain a first real-time image including a person image, wherein the person image has an obscured portion; A first acquisition module, used to acquire character feature data corresponding to the character image; A first restoration module, used for restoring the blocked part of the character image based on the character feature data to obtain a complete character image and a second real-time image including the complete character image; The first display module is used to display the second real-time image.

9. An image display device, characterized in that: The image display device comprises: A second acquisition module is used to acquire images of the real-time viewing environment to obtain a first real-time image including a person image; A second acquisition module, used to acquire character feature data corresponding to the character image and a display flag; A first judging module, used for judging whether the display flag is not displayed; A first determination module, configured to determine a person position of the person image in the first real-time image based on the person feature data if the display flag is not displayed; A second restoration module is used to restore the image background corresponding to the position of the person to obtain a third real-time image that does not contain the image of the person; The second display module is used to display the third real-time image.

10. An extended reality device, characterized in that: The extended reality device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the image display method according to any one of claims 1 to 7 when executed by the processor.

Images