Hybrid reality head-mounted device and depth-of-field correction system and method

Abstract

A depth-of-field correction method includes performing, by a first hybrid reality head-mounted device, superimposing a first background image and a first virtual object to generate a superimposed image, inputting the superimposed image to a depth-of-field correction model to generate a display image, and presenting the display image, receiving at least one user feedback signal corresponding to the display image, and updating a depth-of-field parameter of the depth-of-field correction model according to the at least one user feedback signal, so that the depth-of-field correction model generates an updated virtual object.

Description

Technical Field

[0001] The present invention relates to a mixed reality head mounted device and a depth of field correction system and method. Background Art

[0002] At present, mixed reality (MR) technology is becoming a trend and gradually showing up in the public eye. The continuous development of optical see-through (OST) mixed reality head-mounted devices brings people a more immersive mixed reality experience. Through these excellent technologies, users can experience a seamless and realistic fusion between virtual objects and physical scenes.

[0003] At present, the main means of depth effect presented by optical perspective mixed reality head-mounted devices is to control the depth rendering effect on the screen based on structured light and / or stereoscopic vision. However, these control methods cannot meet the user's real visual experience because there is no reference index. Summary of the invention

[0004] In view of the above, the present invention provides a mixed reality head mounted device and a depth of field correction system and method for solving the above problems.

[0005] A depth of field correction method according to an embodiment of the present invention includes being executed by a first mixed reality head mounted device: superimposing a first background image and a first virtual object to generate a superimposed image; inputting the superimposed image into a depth of field correction model to generate a display image, and presenting the display image; receiving at least one user feedback signal corresponding to the display image; and updating the depth of field parameters of the depth of field correction model with the at least one user feedback signal so that the depth of field correction model generates an updated virtual object.

[0006] According to an embodiment of the present invention, a mixed reality head mounted device includes: an imaging element, a storage element, an input element, and a processing device. The imaging element is used to obtain a first background image. The storage element is used to store a depth correction model. The input element is used to obtain a user feedback signal. The processing device is connected to the imaging element, the storage element, and the input element, and is used to perform: superimposing the first background image and the first virtual object to generate a superimposed image; inputting the superimposed image to the depth correction model to generate a display image, and presenting the display image; receiving the at least one user feedback signal corresponding to the display image; and updating the depth parameter of the depth correction model with the user feedback signal, so that the depth correction model generates an updated virtual object.

[0007] A depth of field correction system according to an embodiment of the present invention includes: a first mixed reality head mounted device, a second mixed reality head mounted device, a sensor, and a processing device. The first mixed reality head mounted device is the mixed reality head mounted device described above. The second mixed reality head mounted device is used to obtain a second background image. The sensor is disposed on the second mixed reality head mounted device. The processing device is connected to the second mixed reality head mounted device and the sensor, and is used to perform: superimposing the second background image and the second virtual object to generate and present an initial image; obtaining perception data in response to the initial image from the sensor; generating training data according to the perception data and the depth of field parameter of the initial image; and using the training data for training to generate a depth of field correction model.

[0008] In summary, the mixed reality head mounted device and the corresponding depth of field correction method according to one or more embodiments of the present invention can dynamically adjust the depth of field through the depth of field correction model, and can instantly update the displayed image to achieve a more realistic effect. In addition, by instantly updating the depth of field correction model according to the user feedback signal, it can be ensured that the focus changes of the user in different scenes are immediately reflected on the virtual object, which can meet the user's real visual experience. The depth of field correction system according to one or more embodiments of the present invention can effectively establish an accurate depth of field correction model. In addition, by establishing a depth of field correction model based on the perception data, the mixed reality image presented by the mixed reality head mounted device after leaving the factory can be close to the mixed reality image that the user may expect.

[0009] The above description of the disclosed content and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and to provide further explanation of the claims of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 FIG. 4 is a block diagram of a mixed reality head mounted device according to an embodiment of the present invention.

[0011] Figure 2 FIG. 4 is a flow chart of a depth of field correction method according to an embodiment of the present invention.

[0012] Figure 3A is a schematic diagram illustrating a mixed reality image without depth correction. Figure 3B FIG. 4 is a schematic diagram illustrating a mixed reality image after depth correction according to the present invention.

[0013] Figure 4 The flowchart of a method for updating depth parameters of a depth calibration model using a user feedback signal according to an embodiment of the present invention is shown.

[0014] Figure 5 FIG. 4 is a block diagram of a mixed reality head mounted device according to another embodiment of the present invention.

[0015] Figure 6 FIG. 4 is a block diagram of a depth of field correction system according to an embodiment of the present invention.

[0016] Figure 7 FIG. 4 is a flow chart of generating a depth of field correction model according to an embodiment of the present invention.

[0017] Figure 8 FIG. 4 is a block diagram of a depth of field correction system according to another embodiment of the present invention. DETAILED DESCRIPTION

[0018] The detailed features and advantages of the present invention are described in detail in the following embodiments, and the contents are sufficient to enable any person skilled in the art to understand the technical content of the present invention and implement it accordingly, and according to the contents disclosed in this specification, claims and drawings, any person skilled in the art can easily understand the relevant purposes and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention in any viewpoint.

[0019] It should be noted that the mixed reality head mounted device described below may be an optical see-through (OST) mixed reality head mounted device.

[0020] Please refer to Figure 1 ,in Figure 1 FIG. 1 is a block diagram of a mixed reality head mounted device according to an embodiment of the present invention. Figure 1 As shown, the mixed reality head mounted device 10 includes an imaging element 101, a storage element 102, an input element 103 and a processing device 104. The processing device 104 is electrically or communicatively connected to the imaging element 101, the storage element 102 and the input element 103.

[0021] The image capturing element 101 is a photographic element for obtaining real-world images. The storage element 102 may be a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory and / or a non-volatile random-access memory (NVRM). The storage element 102 is used to store a trained depth correction model. The depth correction model may be a convolutional neural network (CNN) model or a recurrent neural network (RNN) model. The input element 103 is used to obtain user feedback signals. The input element 103 may be an element such as a touch screen, a keyboard or a mouse, for receiving data input by the user, wherein the touch screen may be a touch screen of a smart phone or a tablet computer; the input element 103 may be a sensing element for sensing sensory data associated with the user; or the input element 103 may be an imaging element of the mixed reality head mounted device 10, which may display a virtual screen to receive user feedback signals.

[0022] The processing device 104 may include one or more processors, such as a central processing unit, a graphics processor, a microcontroller, a programmable logic controller, or other processors with signal computing functions. The processing device 104 is used to update the trained depth of field correction model stored in the storage element 102 based on the user feedback signal, so that the updated depth of field correction model can better meet the usage requirements of each user.

[0023] For a more detailed description of the operation of the mixed reality head mounted device 10, please refer to Figure 1 and Figure 2 ,in Figure 2 FIG. 1 is a flow chart of a depth of field correction method according to an embodiment of the present invention. Figure 2 As shown, the depth of field correction method is executed by the mixed reality head-mounted device 10, including: step S101: superimposing a first background image and a first virtual object to generate a superimposed image; step S103: inputting the superimposed image into the depth of field correction model to generate a display image, and presenting the display image; step S105: receiving at least one user feedback signal corresponding to the display image; and step S107: updating the depth of field parameters of the depth of field correction model with the at least one user feedback signal, so that the depth of field correction model generates an updated virtual object.

[0024] In step S101, the real image obtained by the imaging element 101 shooting the real environment can be used as the first background image, and the processing device 104 is used to perform superimposing the first virtual object on the first background image to generate a superimposed image. The first virtual object can be an object image pre-stored in the storage element 102, an object image obtained by the processing device 104 from the cloud, or an object image generated by a generative artificial intelligence model. The present invention does not limit the source of the first virtual object.

[0025] In step S103, the processing device 104 is used to execute inputting the superimposed image into the depth correction model stored in the storage element 102, generating a display image by the depth correction model, and displaying the display image by the display element of the mixed reality head mounted device 10 for the user to watch, wherein the display image includes the first background image and the calibrated virtual object that is the same object as the first virtual object. The depth parameter of the display image may be different from that of the superimposed image. Further, one or more of the depth parameter, focal length parameter, and aperture value (F number) corresponding to the display image may be different from that of the superimposed image, and the calibrated virtual object in the display image may be clearer than the first virtual object in the superimposed image.

[0026] In step S105, the input element 103 receives a user feedback signal corresponding to the displayed image, and outputs the user feedback signal to the processing device 104, wherein the user feedback signal indicates the user's viewing experience of the displayed image. As mentioned above, the user feedback signal can be data input by the user himself, or data perceived by the user.

[0027] In step S107, the processing device 104 is used to execute the user feedback signal to update the depth correction model stored in the storage element 102, so that the depth correction model generates another display image, and the other display image includes an updated virtual object. The display content of the other display image can be the same as the display content of the display image generated in step S103, except that the depth parameter of the updated virtual object can better meet the user's needs (for example, clearer) than the virtual object in the display image generated in step S103.

[0028] In other words, the mixed reality head mounted device and depth of field correction method according to one or more of the above embodiments achieves natural defocus and / or blur effects corresponding to different depths by automatically adjusting the depth of field parameters of virtual objects using a depth of field correction model. Whether observing details at close range or feeling the vastness of distant scenes, users can enjoy a richer and more realistic visual experience in mixed reality.

[0029] Please refer to Figure 3A and Figure 3B ,in Figure 3A is a schematic diagram illustrating a mixed reality image without depth correction. Figure 3B FIG. 2 is a schematic diagram showing a mixed reality image after depth correction of the present invention. Figure 3A and Figure 3B In the image without depth correction, the arm is a virtual object and the rest of the image is the background image. Figure 3A In the mixed reality image with depth correction, the virtual objects are blurry; Figure 3B In the mixed reality images, virtual objects are obviously clearer.

[0030] Please refer to Figure 1 and Figure 4 ,in Figure 4 FIG. 1 is a flow chart of a method for updating depth parameters of a depth calibration model using user feedback signals according to an embodiment of the present invention. Figure 4 As shown, in an embodiment where the user feedback signal is data input by the user, the method for updating the depth of field parameters of the depth of field correction model includes: step S201: receiving at least one user feedback signal corresponding to the displayed image; step S203: determining whether the at least one user feedback signal is a positive feedback result or a negative feedback result; if the determination result of step S203 is a "positive feedback result", executing step S205: generating a first weight value corresponding to the depth of field parameter; and if the determination result of step S203 is a "negative feedback result", executing step S205: generating a second weight value corresponding to the depth of field parameter, wherein the first weight value is greater than the second weight value. Step S201 can be Figure 2 The steps S203, S205 and S207 are the same as step S105, so they are not described here in detail. Figure 2 Detailed flowchart of an embodiment of step S107.

[0031] In step S203, the processing device 104 may determine whether the user feedback signal is a positive feedback result or a negative feedback result according to the user's instruction on the input element 103. A positive feedback result indicates that the depth of field parameters of the virtual object in the displayed image meet the user's expectations, and a negative feedback result indicates that the depth of field parameters of the virtual object in the displayed image do not meet the user's expectations. For example, the input element 103 is a touch screen, and a "clear enough" button and a "not clear enough" button are displayed on the touch screen. Alternatively, the input element 103 is an imaging element of the mixed reality head mounted device 10, and the virtual screen displayed by the imaging element includes a "clear enough" virtual button and a "not clear enough" virtual button. "Clear enough" may be a positive feedback result, and "not clear enough" may be a negative feedback result. The input element 103 may generate a user feedback signal corresponding to the triggered button. The above clarity and the corresponding relationship between it and the feedback result are only examples, and the present invention is not limited thereto.

[0032] If the user feedback signal indicates a positive feedback result, then in step S205, the processing device 104 may generate a first weight value corresponding to the depth parameter of the depth correction model equal to or greater than 1 after execution. On the contrary, if the user feedback signal indicates a negative feedback result, then in step S207, the processing device 104 may generate a second weight value corresponding to the depth parameter of the depth correction model less than the first weight value after execution. Accordingly, the updated virtual object generated by the updated depth correction model may better meet the user's needs.

[0033] In addition, taking the case where the depth of field parameters include aperture value, rendering range and focal length range as an example, if the judgment result of step S203 is a "negative feedback result", the processing device 104 can generate multiple test images after execution, and the test images have different combinations of aperture value, rendering range and focal length range. Next, the processing device 104 receives multiple user feedback signals corresponding to the multiple test images from the input element 103. When the user feedback signal is a positive feedback result, it means that the combination of the depth of field parameters (aperture value, rendering range and focal length range) of the corresponding test image is good, and the processing device 104 executes step S205 for the depth of field parameters of the test image; on the contrary, when the user feedback signal is a negative feedback result, it means that the combination of the depth of field parameters (aperture value, rendering range and focal length range) of the corresponding test image may not be good enough, and the processing device 104 executes step S207 for the depth of field parameters of the test image. The processing device 104 may utilize the above results to retrain the depth correction model in the storage element 102 , so that the depth correction model will avoid using the same combination when generating the next display image.

[0034] Please refer to Figure 5 ,in Figure 5 FIG. 1 is a block diagram of a mixed reality head mounted device according to another embodiment of the present invention. Figure 5 As shown, the mixed reality head mounted device 20 includes an imaging element 201, a storage element 202, an input element 203 and a processing device 204. The implementation and operation of the imaging element 201, the storage element 202 and the processing device 204 can be respectively Figure 1 The imaging element 101, the storage element 102 and the processing device 104 are the same and thus will not be described in detail here.

[0035] like Figure 5 As shown, when the input element 203 is a sensing element, it may include an eye tracker 203a and a depth sensor 203b.

[0036] In an embodiment of the eye tracker 203a, the user feedback signal is the user's pupil size, and the depth of field parameter of the depth of field correction model may include an aperture value. Figure 2In step S107, the processing device 204 is used to execute the judgment whether the pupil size is greater than the upper limit value or less than the lower limit value, and retrain the depth of field correction model to increase the aperture value when the pupil size is judged to be greater than the upper limit value; and the processing device 204 retrains the depth of field correction model to reduce the aperture value when the pupil size is judged to be less than the lower limit value. In other words, when the pupil size is greater than the upper limit value, it means that the aperture value may be insufficient and the depth of field is too deep, so the aperture value of the retrained depth of field correction model can be increased; conversely, when the pupil size is less than the lower limit value, it means that the aperture value may be too high and the depth of field is too shallow, so the aperture value of the retrained depth of field correction model can be reduced. The interval between the upper limit value and the lower limit value can be a normal interval, that is, when the pupil size falls within the normal interval, the aperture value does not need to be adjusted.

[0037] In an embodiment of the eye tracker 203a, the user feedback signal is the user's eye focus position, and the depth of field parameter of the depth of field correction model may include a rendering range. Figure 2 In step S107, the processing device 204 may first determine whether the eye focus position overlaps with the first virtual object. When it is determined that the two do not overlap, the depth of field correction model is retrained using the determination result to adjust the rendering range. In other words, when the processing device 204 determines that the eye focus position is not mapped to the first virtual object, the depth of field correction model is retrained to adjust the depth of field parameters of the depth of field correction model.

[0038] In the embodiment of the depth sensor 203b, the user feedback signal is the depth distance between the mixed reality head mounted device 20 and the physical object in the first background image, and the depth parameter of the depth correction model may include the focal length range. Figure 2 In step S107, the processing device 204 may retrain the depth of field correction model according to the depth distance to adjust the focal length range. For example, if the depth distance is larger, the processing device 204 may adjust the focal length range to a larger value.

[0039] Furthermore, when the processing device 204 determines that the user feedback signal is a positive feedback result, the depth of field parameters of the depth of field correction model may be stored as the current aperture value, rendering range and / or focal length range; when the processing device 204 determines that the user feedback signal is a negative feedback result, the processing device 204 may adjust the depth of field parameters according to one or more of the above embodiments, and then store the depth of field parameters of the depth of field correction model as the updated depth of field parameters.

[0040] In other words, in the embodiment where the input element 203 includes both the eye tracker 203a and the depth sensor 203b, that is, in the embodiment where the user feedback signal includes the pupil size, the eye focus position and the depth distance at the same time, when the user feedback signal is a positive feedback result, it means that the combination of the depth parameters (aperture value, rendering range and focal length range) of the depth correction model is good; on the contrary, when the user feedback signal is a negative feedback result, it means that the combination of the depth parameters (aperture value, rendering range and focal length range) of the depth correction model may not be good enough, so the depth correction model will avoid using the same combination when generating the next display image.

[0041] Please refer to Figure 6 ,in Figure 6 FIG. 1 is a block diagram of a depth of field correction system according to an embodiment of the present invention. Figure 6 As shown, the depth of field correction system 100 includes a first mixed reality head mounted device 1, a second mixed reality head mounted device 2 and a processing device 3. The processing device 3 is connected to the first mixed reality head mounted device 1 and the second mixed reality head mounted device 2. The first mixed reality head mounted device 1 is the mixed reality head mounted device 10 or the mixed reality head mounted device 20 described in one or more embodiments above. The second mixed reality head mounted device 2 includes an imaging element 21 and a sensor 22, and the imaging element 21 is used to obtain a second background image. The sensor 22 is embedded in the housing (such as a glasses device) of the second mixed reality head mounted device 2. The sensor 22 can be a sensing element for sensing to generate perception data associated with the user. Further, the sensor 22 can be implemented by the input element 103 of the aforementioned embodiment. Further, the second mixed reality head mounted device 2 can be the mixed reality head mounted device 10 or the mixed reality head mounted device 20 described in one or more embodiments above. The processing device 3 may be a server and may include one or more processors, such as a central processing unit, a graphics processor, a microcontroller, a programmable logic controller, or other processors with computing functions. The first mixed reality head mounted device 1 in the depth correction system 100 may be a mixed reality head mounted device that has been shipped from the factory, and the second mixed reality head mounted device 2 may be a mixed reality head mounted device that is used to establish the aforementioned depth correction model in the factory.

[0042] For a more detailed description of the operation of the depth of field correction system 100, please refer to Figure 6 and Figure 7 ,in Figure 7 FIG. 1 is a flow chart of generating a depth of field correction model according to an embodiment of the present invention. Figure 7As shown, the method for generating a depth of field correction model is executed by the depth of field correction system 100, including: step S301: superimposing a second background image and a second virtual object to generate and present an initial image; step S303: obtaining perception data in response to the initial image from a sensor; step S305: generating training data based on the perception data and the depth of field parameters of the initial image; and step S307: using the training data for training to generate a depth of field correction model.

[0043] In step S301, the imaging element of the second mixed reality head mounted device 2 captures the real image as the second background image, the processing device 3 is used to perform superimposing the second virtual object on the second background image to generate an initial image, and the initial image is displayed for viewing by the imaging element of the second mixed reality head mounted device 2. The method of generating the second virtual object can be the same as that of the first virtual object, and will not be repeated here.

[0044] In step S303, the processing device 3 is used to execute the perception data obtained from the sensor 22 of the second mixed reality head mounted device 2 in response to the initial image. As mentioned above, the perception data can be the perception data associated with the user generated by sensing the user. The perception data indicates the user's perception of the depth of field of the initial image.

[0045] In step S305, the processing device 3 generates training data according to the perception data and the depth of field parameters of the initial image. For example, if the perception data indicates that the depth of field of the initial image is insufficient, the processing device 3 may use the judgment result of insufficient depth of field of the initial image and the depth of field parameters of the initial image as training data, so that the trained depth of field correction model can improve the value of the depth of field parameters of the image during actual operation; if the perception data indicates that the depth of field of the initial image is sufficient, the processing device 3 may use the judgment result of sufficient depth of field of the initial image and the depth of field parameters of the initial image as training data, so that the trained depth of field correction model can maintain the value of the depth of field parameters of the image during actual operation.

[0046] In step S307, the processing device 3 uses the training data to perform training to generate a depth correction model, and can store the trained depth correction model in the storage element of the first mixed reality head mounted device 1. The depth correction model can be a convolutional neural network model or a recurrent neural network model.

[0047] Accordingly, by combining a two-stage process of sensing collection (generating a depth of field correction model based on the second mixed reality head-mounted device) and evaluation verification (correcting the depth of field parameters of the depth of field correction model based on the first mixed reality head-mounted device), the depth of field of the mixed reality image can be automatically controlled, thereby achieving natural defocus and blur effects for objects at different depths.

[0048] Please refer to Figure 8 ,in Figure 8FIG. 1 is a block diagram of a depth of field correction system according to another embodiment of the present invention. Figure 8 As shown, the depth of field correction system 200 includes a first mixed reality head mounted device 1, a second mixed reality head mounted device 2' and a processing device 3. The implementation and operation of the first mixed reality head mounted device 1 and the processing device 3 can be respectively Figure 6 The first mixed reality head mounted device 1 and the processing device 3 are the same, so they are not described here in detail.

[0049] like Figure 8 As shown, the second mixed reality head mounted device 2' comprises an imaging element 21 and a sensor 22', wherein the implementation and operation of the imaging element 21 can be the same as Figure 6 The sensor 22' is embedded in the housing (eg, eyeglasses) of the second mixed reality head mounted device 2'. The sensor 22' may include an eye tracker 221' and a depth sensor 222'.

[0050] In an embodiment of the eye tracker 221', the perception data is the user's pupil size, and the depth of field parameter of the initial image may include an aperture value. Figure 7 In step S305, the processing device 3 may first determine whether the pupil size is greater than the upper limit or less than the lower limit. When it is determined that the pupil size is greater than the upper limit, the determination result and the aperture value of the initial image are used as training data. Accordingly, the trained depth of field correction model may generate an image with a reduced aperture value to avoid the pupil size being less than the lower limit, and / or may generate an image with an increased aperture value to avoid the pupil size being greater than the upper limit. In other words, when the pupil size is greater than the upper limit, it indicates that the aperture value may be insufficient and the depth of field is too deep, so the processing device may increase the aperture value of the depth of field correction model; conversely, when the pupil size is less than the lower limit, it indicates that the aperture value may be too high and the depth of field is too shallow, so the processing device may reduce the aperture value of the depth of field correction model. The interval between the upper limit and the lower limit may be a normal interval, that is, when the pupil size falls within the normal interval, the aperture value of the initial image may be used.

[0051] In the embodiment of the eye tracker 221', the perception data is the user's eye focus position, and the depth of field parameter of the initial image may include a rendering range. Figure 7 In step S305, the processing device 3 may first determine whether the eye focus position overlaps with the first virtual object. When it is determined that the two do not overlap, the processing device 3 inputs the judgment result of the non-overlapping and the initial image into the depth of field correction model for training, so that the depth of field correction model adjusts the rendering range according to the judgment result of the non-overlapping. In other words, the depth of field correction model can adjust the depth of field parameters of the depth of field correction model based on the mapping relationship between the eye focus position of the processing device and the first virtual object.

[0052] In the embodiment of the depth sensor 222', the perception data is the depth distance between the second mixed reality head mounted device 2' and the physical object in the second background image, and the depth parameter of the initial image may include the focal length range. Figure 7 In step S305, the processing device 3 may use the depth distance and the focal length range of the initial image as training data. For example, if the depth distance is larger, the trained depth of field correction model may adjust the focal length range to a larger value.

[0053] In summary, the mixed reality head mounted device and the corresponding depth of field correction method according to one or more embodiments of the present invention can dynamically adjust the depth of field through the depth of field correction model, and can instantly update the displayed image to achieve a more realistic effect. In addition, by instantly updating the depth of field correction model according to the user feedback signal, it can be ensured that the focus changes of the user in different scenes are immediately reflected on the virtual object, which can meet the user's real visual experience. The depth of field correction system according to one or more embodiments of the present invention can effectively establish an accurate depth of field correction model. In addition, by establishing a depth of field correction model based on the perception data, the mixed reality image presented by the mixed reality head mounted device after leaving the factory can be close to the mixed reality image that the user may expect.

[0054]

Explanation of symbols

[0055] 100,200: Depth of field correction system

[0056] 10,20,1,2,2': Mixed reality head mounted device

[0057] 101,201,21: Image acquisition element

[0058] 102,202: Storage element

[0059] 103,203: Input components

[0060] 104,204: Processing device

[0061] 203a,221': Eye tracker

[0062] 203b,222': Depth sensor

[0063] 22,22': Perceptron

[0064] 3: Processing device

[0065] S101, S103, S105, S107, S201, S203, S205, S207, S301, S303, S305, S307: steps.

Claims

1. A depth of field correction method, characterized in that: The method includes executing, by a first mixed reality head mounted device: superimposing the first background image and the first virtual object to generate a superimposed image; Inputting the superimposed image into a depth correction model to generate a display image, and presenting the display image; receiving at least one user feedback signal corresponding to the displayed image; as well as The depth parameter of the depth calibration model is updated by the at least one user feedback signal, so that the depth calibration model generates an updated virtual object.

2. The depth of field correction method according to claim 1, characterized in that: Updating the depth parameter of the depth calibration model using the at least one user feedback signal comprises: Determining whether the at least one user feedback signal is a positive feedback result or a negative feedback result; When the at least one user feedback signal is determined to be the positive feedback result, generating a first weight value corresponding to the depth of field parameter; as well as When the at least one user feedback signal is determined to be the negative feedback result, a second weight value corresponding to the depth of field parameter is generated, The first weight value is greater than the second weight value.

3. The depth of field correction method according to claim 1, characterized in that: The at least one user feedback signal indicates a pupil size, and the depth of field parameter includes an aperture value. Updating the depth of field parameter of the depth of field correction model using the at least one user feedback signal includes: Determine whether the pupil size is greater than an upper limit or less than a lower limit; When it is determined that the pupil size is greater than the upper limit value, increasing the aperture value; as well as When it is determined that the pupil size is smaller than the lower limit, the aperture value is reduced.

4. The depth of field correction method according to claim 1, characterized in that: The at least one user feedback signal indicates an eye focus position, the depth of field parameter includes a rendering range, and updating the depth of field parameter of the depth of field correction model using the at least one user feedback signal includes: When it is determined that the eye focus position does not overlap with the first virtual object, the rendering range is adjusted.

5. The depth of field correction method according to claim 1, characterized in that: The at least one user feedback signal indicates a depth distance between the first mixed reality head mounted device and a physical object in the first background image, and the depth of field parameter includes a focal length range. Updating the depth of field parameter of the depth of field correction model using the at least one user feedback signal includes: adjusting the focal length range.

6. The depth of field correction method according to claim 1, characterized in that: Also comprising: executing by the processing device: Superimposing a second background image and a second virtual object obtained by a second mixed reality head mounted device to generate and present an initial image; Obtaining sensory data in response to the initial image from a sensor disposed in the second mixed reality head mounted device; Generate training data according to the perception data and the depth parameter of the initial image; and The training data is used for training to generate the depth correction model.

7. The depth of field correction method according to claim 6, characterized in that: The perception data includes pupil size, and adjusting the depth parameter of the initial image according to the perception data to generate the training data includes: Determine whether the pupil size is greater than an upper limit or less than a lower limit; as well as When it is determined that the pupil size is greater than the upper limit or less than the lower limit, the determination result and the aperture value of the initial image are used as the training data.

8. The depth of field correction method according to claim 6, characterized in that: The perception data includes an eye focus position, and adjusting the depth parameter of the initial image according to the perception data to generate the training data includes: When it is determined that the eye focus position does not overlap with the second virtual object, the determination result and the rendering range of the initial image are used as the training data.

9. The depth of field correction method according to claim 6, characterized in that: The perception data includes a depth distance between the second mixed reality head mounted device and a physical object in the second background image, and adjusting the depth parameter of the initial image according to the perception data to generate the training data includes: The depth distance and the focal length range of the initial image are used as the training data.

10. A mixed reality head mounted device, characterized in that: Include: An imaging element, used for acquiring a first background image; A storage element, used for storing a depth of field correction model; An input element, used to obtain at least one user feedback signal; as well as A processing device, connected to the image capturing element, the storage element and the input element, for executing: Overlaying the first background image and the first virtual object to generate an overlay image; Inputting the superimposed image into the depth correction model to generate a display image, and presenting the display image; receiving the at least one user feedback signal corresponding to the displayed image; as well as The depth parameter of the depth calibration model is updated by the at least one user feedback signal, so that the depth calibration model generates an updated virtual object.

11. The mixed reality head mounted device according to claim 10, characterized in that The processing device determines whether the at least one user feedback signal is a positive feedback result or a negative feedback result; when the at least one user feedback signal is determined to be the positive feedback result, a first weight value corresponding to the depth of field parameter is generated; and when the at least one user feedback signal is determined to be the negative feedback result, a second weight value corresponding to the depth of field parameter is generated, wherein the first weight value is greater than the second weight value.

12. The mixed reality head mounted device according to claim 10, characterized in that The depth of field parameter includes the aperture value, and the input component includes: an eye tracker for generating pupil size of the at least one user feedback signal, The processing device increases the aperture value when the pupil size is larger than an upper limit value, and decreases the aperture value when the pupil size is smaller than a lower limit value.

13. The mixed reality head mounted device according to claim 10, characterized in that The depth of field parameter contains the rendering range, and the input component contains: an eye tracker for generating an eye focus position of the at least one user feedback signal, The processing device adjusts the rendering range when the eye focus position does not overlap with the first virtual object.

14. The mixed reality head mounted device according to claim 10, characterized in that The depth of field parameter includes the focal length range, and the input components include: A depth sensor for generating a depth distance of the at least one user feedback signal, The depth distance is used to adjust the focal length range.

15. A depth of field correction system, characterized in that: Include: A first mixed reality head mounted device, which is a mixed reality head mounted device according to any one of claims 10 to 14; a second mixed reality head mounted device, for acquiring a second background image, and comprising a sensor; and A processing device, connected to the second mixed reality head mounted device, and configured to execute: superimposing the second background image and the second virtual object to generate and present an initial image; Acquire sensory data in response to the initial image from the sensor; Adjusting the depth parameter of the initial image according to the perception data to generate training data; and The training data is used for training to generate the depth correction model.

16. The depth of field correction system according to claim 15, characterized in that The sensor includes an eye tracker for generating pupil size for the sensor data. The processing device is used to determine whether the pupil size is greater than an upper limit or less than a lower limit, and use the determination result and the aperture value of the initial image as the training data.

17. The depth of field correction system according to claim 15, characterized in that The sensor includes an eye tracker for generating eye focus position of the sensor data, When the processing device determines that the eye focus position does not overlap with the second virtual object, the processing device uses the determination result and the rendering range of the second virtual object as the training data.

18. The depth of field correction system according to claim 15, characterized in that The sensor includes a depth sensor for sensing a depth distance between the second mixed reality head mounted device and a physical object in the second background image. The processing device uses the depth distance and the focal length range of the initial image as the training data.

Images