Augmented reality image generation method and device, equipment and computer readable medium

Abstract

The embodiment of the invention discloses an augmented reality image generation method and device in a mixed reality space, equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: in response to a detected shooting operation of a user, extracting a current virtual picture frame from an original augmented reality image frame sequence, and generating a target virtual picture according to the current virtual picture frame; according to depth information of the target virtual picture in the scene, the focal length of a camera of the mixed reality equipment is adjusted to a position corresponding to the depth information, and a shot high-definition scene image of the real space is acquired from camera data of the mixed reality equipment; and superposing the target virtual picture into the high-definition scene image of the real space in a mode of anchoring with the real space to obtain an augmented reality image. According to the embodiment, the waste of computing resources is reduced, and the definition of the augmented reality image and the experience of browsing the augmented reality image by the user are improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an augmented reality image generation method, apparatus, device, and computer-readable medium. Background Art

[0002] The augmented reality image is a superposition of the virtual image and the real image. The method for obtaining the augmented reality image is generally to use ARKit rendering, or to remove the same part of the augmented reality image rendered by ARKit and the high-definition scene image taken by the mixed reality device to obtain the virtual image part, and then superimpose the virtual image part on the high-definition scene image.

[0003] However, the real environment part of the augmented reality image rendered by ARKit's sceneView.snapshot() method has low definition, and the best image quality can only reach 1080p. The method of removing the same part of the augmented reality image rendered by ARKit and the high-definition scene image taken by the mixed reality device and then superimposing it is too complicated, resulting in a waste of computing resources and a long time consumption, which leads to a poor user experience.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose an augmented reality image generation method, device, electronic device and computer-readable medium in a mixed reality space to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for generating an augmented reality image in a mixed reality space, the method comprising: in response to determining that a mixed reality device is successfully matched with a real space, displaying the mixed reality space in the mixed reality device, wherein the mixed reality space includes a virtual screen anchored to the real space; in response to detecting a shooting operation of a user, extracting a current virtual screen frame from an original augmented reality image frame sequence, and generating a target virtual screen based on the current virtual screen frame; based on the depth information of the target virtual screen in the scene, adjusting the focal length of the camera of the mixed reality device to a position corresponding to the depth information, and acquiring a high-definition scene image of the real space captured from the camera data of the mixed reality device; and superimposing the target virtual screen on the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image.

[0008] Optionally, before extracting the current virtual picture frame from the original augmented reality image frame sequence and generating the target virtual picture based on the current virtual picture frame, the method also includes: in response to detecting that the posture of a character in the real scene picture reaches a preset condition, capturing a high-definition scene image of the real space.

[0009] Optionally, the method further includes: in response to detecting that a watermark setting function is turned on, superimposing preset watermark information on the augmented reality image.

[0010] Optionally, before acquiring the captured high-definition scene image of the real space, the method further includes: in response to determining that the configuration information of the mixed reality device does not meet a preset configuration condition, turning off the tracker.

[0011] Optionally, after acquiring the captured high-definition scene image of the real space, the method further includes: in response to determining that the configuration information of the mixed reality device does not meet the preset configuration conditions, turning on the tracker, and repositioning to restore the association between the virtual media information and the real space.

[0012] Optionally, the above-mentioned extracting the current virtual picture frame from the original augmented reality image frame sequence, and generating the target virtual picture based on the above-mentioned current virtual picture frame, include: creating an initial virtual view; obtaining virtual media information and scene attribute information of the scene corresponding to the above-mentioned current virtual picture frame; assigning the above-mentioned virtual media information and the above-mentioned scene attribute information to the above-mentioned initial virtual view to obtain the target view; and generating the target virtual picture according to the above-mentioned target view.

[0013] Optionally, the method further includes: drawing the augmented reality image on the display screen; in response to detecting screen rotation, determining a display direction of the augmented reality image according to the rotation direction.

[0014] In a second aspect, some embodiments of the present disclosure provide an augmented reality image generating device in a mixed reality space, comprising: a display unit, configured to display the mixed reality space in the mixed reality device in response to determining that the mixed reality device and the real space are successfully matched, wherein the mixed reality space includes a virtual screen anchored to the real space; a generation unit, configured to extract a current virtual screen frame from an original augmented reality image frame sequence in response to detecting a user's shooting operation, and generate a target virtual screen based on the current virtual screen frame; an adjustment unit, configured to adjust the focal length of the camera of the mixed reality device to a position corresponding to the depth information of the target virtual screen in the scene, and obtain a high-definition scene image of the real space captured from the camera data of the mixed reality device; and an overlay unit, configured to overlay the target virtual screen onto the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image.

[0015] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0016] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0017] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the augmented reality image generation method in the mixed reality space of some embodiments of the present disclosure, the clarity of the augmented reality image is improved, the waste of computing resources is reduced, and the user's experience of browsing the augmented reality image is improved. Specifically, the reasons for the poor clarity of the augmented reality image, the high consumption of computing resources, and the poor user experience of browsing the augmented reality image are: the real environment part of the augmented reality image rendered by the sceneView.snapshot() method of ARKit has low clarity, and the best quality of the image can only reach 1080p. The method of removing the same part of the augmented reality image rendered by ARKit and the high-definition scene image taken by the mixed reality device and then superimposing it is too complicated, resulting in a waste of computing resources and a long time, thereby resulting in a poor user experience. Based on this, the augmented reality image generation method in the mixed reality space of some embodiments of the present disclosure, first, in response to determining that the mixed reality device and the real space are successfully matched, the mixed reality space is displayed in the above-mentioned mixed reality device, wherein the above-mentioned mixed reality space includes a virtual screen anchored with the real space. Thus, the mixed reality space can be browsed by the user. Then, in response to detecting the shooting operation of the user, the current virtual screen frame is extracted from the original augmented reality image frame sequence, and the target virtual screen is generated according to the current virtual screen frame. Thus, the target virtual screen can be obtained. Afterwards, according to the depth information of the target virtual screen in the scene, the focal length of the camera of the mixed reality device is adjusted to the position corresponding to the depth information, and the high-definition scene image of the real space captured is obtained from the camera data of the mixed reality device. Thus, the high-definition scene image of the real space can be obtained. Finally, the target virtual screen is superimposed on the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image. Thus, an augmented reality image superimposed with the target virtual screen and the high-definition scene image can be obtained. Also because of the augmented reality image generation method implemented in the mixed reality space, the target virtual screen is extracted from the pre-rendered low-resolution augmented reality image, and then the target virtual screen is superimposed on the high-definition scene image captured by the mixed reality device in an anchored manner, and no complex algorithm is required. Thereby, the clarity of the augmented reality image is improved, the waste of computing resources is reduced, and the user's experience of browsing the augmented reality image is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0019] Figure 1 is a flow chart of some embodiments of a method for generating an augmented reality image in a mixed reality space according to the present disclosure;

[0020] Figure 2 are flowcharts of other embodiments of a method for generating an augmented reality image in a mixed reality space according to the present disclosure;

[0021] Figure 3 is a schematic structural diagram of some embodiments of an augmented reality image generating device in a mixed reality space according to the present disclosure;

[0022] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0028] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0029] Figure 1A process 100 of some embodiments of a method for generating an augmented reality image in a mixed reality space according to the present disclosure is shown. The method for generating an augmented reality image in a mixed reality space comprises the following steps:

[0030] Step 101: In response to determining that the mixed reality device and the real space are successfully matched, displaying the mixed reality space in the mixed reality device.

[0031] In some embodiments, the execution subject (e.g., a computing device) of the method for generating an augmented reality image in a mixed reality space may display the mixed reality space in the mixed reality device in response to determining that the mixed reality device successfully matches the real space. The mixed reality space may include a virtual screen anchored to the real space. The mixed reality device may be, but is not limited to, a mobile phone or a tablet computer. In one or more embodiments, the successful matching of the mixed reality device with the real space may indicate that the mixed reality device can determine its position and posture relative to the real space. The execution subject may use a sensor to determine the position and posture of the mixed reality device relative to the real space. The sensor may be, but is not limited to, an accelerometer or a gyroscope.

[0032] Step 102 , in response to detecting a shooting operation of a user, extracting a current virtual picture frame from the original augmented reality image frame sequence, and generating a target virtual picture according to the current virtual picture frame.

[0033] In some embodiments, in response to detecting a user's shooting operation, the execution subject may extract a current virtual screen frame from the original augmented reality image frame sequence, and generate a target virtual screen based on the current virtual screen frame. The shooting operation may be at least one of the following: click, slide, voice control, gesture control, and head posture control. The original augmented reality image frame sequence may be each augmented reality image rendered within a preset time period before and after the shooting event. Here, the specific setting of the preset time period is not limited. The current virtual screen frame may be a frame of augmented reality image corresponding to the time point when the user shoots. The target virtual screen may be a screen that only includes virtual media information.

[0034] In some optional implementations of some embodiments, the execution subject may extract a current virtual picture frame from the original augmented reality image frame sequence in response to detecting a shooting operation of the user, and generate a target virtual picture according to the current virtual picture frame through the following steps:

[0035] The first step is to create an initial virtual view. The initial virtual view may be a view that displays a 3D model object. In practice, the execution subject may use the SCNView framework to create the initial virtual view.

[0036] The second step is to obtain the virtual media information and scene attribute information of the scene corresponding to the current virtual picture frame. The virtual media information may include but is not limited to at least one of the following types of data: image, text. The scene attribute information may be a pointer to the root node of the scene data corresponding to the current virtual picture. The scene attribute information may characterize the anchor point, origin position, size and other attributes of the scene corresponding to the current virtual picture. The origin position may be the origin coordinates of the spatial coordinate system corresponding to the scene.

[0037] The third step is to assign the virtual media information and the scene attribute information to the initial virtual view to obtain a target view, wherein the target view may be a view including the virtual media information and the scene attribute information.

[0038] The fourth step is to generate a target virtual screen according to the target view. In practice, the execution subject can use the snapshot method of the SCNView framework to obtain a snapshot of the target view to obtain the target virtual screen.

[0039] Optionally, before step 102, in response to detecting that the posture of a person in the real scene screen reaches a preset condition, the above-mentioned execution subject can also shoot a high-definition scene image of the real space. Among them, the above-mentioned real scene screen can be a scene screen of the real space recognized by the mixed reality device. The above-mentioned preset condition can be a condition that needs to be met to trigger the shooting event. The above-mentioned preset condition can be that the posture of the person in the scene reaches a preset posture. Here, the specific setting of the preset posture is not limited.

[0040] Step 103, according to the depth information of the target virtual image in the scene, the focal length of the camera of the mixed reality device is adjusted to the position corresponding to the depth information, and the high-definition scene image of the captured real space is obtained from the camera data of the mixed reality device.

[0041] In some embodiments, the execution subject may adjust the focal length of the camera of the mixed reality device to the position corresponding to the depth information according to the depth information of the target virtual screen in the scene, and obtain the captured high-definition scene image of the real space from the camera data of the mixed reality device. Among them, the depth information may be the distance between the target virtual screen and the camera. The high-definition scene image may be a high-definition photo of the real space taken by the mixed reality device. The camera data may be a sequence of high-definition photo frames of the real space taken by the camera of the mixed reality device. The high-definition photo frame in the sequence of high-definition photo frames may be a frame of high-definition image of the real space. The resolution of the high-definition photo frame may be 4K. In practice, the execution subject may determine the high-definition photo frame of the real space corresponding to the time point of the shooting event in the sequence of high-definition photo frames of the real space taken by the camera of the mixed reality device as the high-definition scene image.

[0042] Step 104 , superimpose the target virtual picture onto the high-definition scene image of the real space in a manner of anchoring the target virtual picture to the real space, so as to obtain an augmented reality image.

[0043] In some embodiments, the above-mentioned execution subject can superimpose the above-mentioned target virtual screen onto the high-definition scene image of the above-mentioned real space in a manner of anchoring with the real space to obtain an augmented reality image. Among them, the above-mentioned augmented reality image can be an image obtained after anchoring the target virtual screen on the above-mentioned high-definition scene image. In one or more embodiments, anchoring can mean fixing the target virtual screen relatively to a preset position in the real space. When the real space is updated, the position and direction of the target virtual screen in the real space can be updated accordingly.

[0044] Optionally, first, the execution entity may also draw the augmented reality image on the display screen. Then, in response to detecting screen rotation, the display direction of the augmented reality image is determined according to the rotation direction. The display screen may be a display screen of a mixed reality device. In practice, the execution entity may use a UIGraphics engine to draw the augmented reality image. The execution entity may use an AVCapture framework to determine the display direction of the augmented reality image.

[0045] Optionally, in response to detecting that the watermark setting function is turned on, the execution subject may also overlay preset watermark information on the high-definition scene image. The preset watermark information may be a watermark selected by a user. The watermark information may be, but is not limited to, at least one of the following types of data: text, image. For example, the watermark information may include personal information of the photographer.

[0046] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the augmented reality image generation method in the mixed reality space of some embodiments of the present disclosure, the clarity of the augmented reality image is improved, the waste of computing resources is reduced, and the user's experience of browsing the augmented reality image is improved. Specifically, the reasons for the poor clarity of the augmented reality image, the high consumption of computing resources, and the poor user experience of browsing the augmented reality image are: the real environment part of the augmented reality image rendered by the sceneView.snapshot() method of ARKit has low clarity, and the best quality of the image can only reach 1080p. The method of removing the same part of the augmented reality image rendered by ARKit and the high-definition scene image taken by the mixed reality device and then superimposing it is too complicated, resulting in a waste of computing resources and a long time, thereby resulting in a poor user experience. Based on this, the augmented reality image generation method in the mixed reality space of some embodiments of the present disclosure, first, in response to determining that the mixed reality device and the real space are successfully matched, the mixed reality space is displayed in the above-mentioned mixed reality device, wherein the above-mentioned mixed reality space includes a virtual screen anchored with the real space. Thus, the mixed reality space can be browsed by the user. Then, in response to detecting the shooting operation of the user, the current virtual screen frame is extracted from the original augmented reality image frame sequence, and the target virtual screen is generated according to the current virtual screen frame. Thus, the target virtual screen can be obtained. Afterwards, according to the depth information of the target virtual screen in the scene, the focal length of the camera of the mixed reality device is adjusted to the position corresponding to the depth information, and the high-definition scene image of the real space captured is obtained from the camera data of the mixed reality device. Thus, the high-definition scene image of the real space can be obtained. Finally, the target virtual screen is superimposed on the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image. Thus, an augmented reality image superimposed with the target virtual screen and the high-definition scene image can be obtained. Also because of the augmented reality image generation method implemented in the mixed reality space, the target virtual screen is extracted from the pre-rendered low-resolution augmented reality image, and then the target virtual screen is superimposed on the high-definition scene image captured by the mixed reality device in an anchored manner, and no complex algorithm is required. Thereby, the clarity of the augmented reality image is improved, the waste of computing resources is reduced, and the user's experience of browsing the augmented reality image is improved.

[0047] Further references Figure 2 , shows a process 200 of some embodiments of the method for generating an augmented reality image in a mixed reality space according to the present disclosure. The method for generating an augmented reality image in a mixed reality space comprises the following steps:

[0048] Step 201, in response to determining that the mixed reality device and the real space are successfully matched, displaying the mixed reality space in the mixed reality device.

[0049] Step 202 , in response to detecting a shooting operation of a user, extracting a current virtual picture frame from the original augmented reality image frame sequence, and generating a target virtual picture according to the current virtual picture frame.

[0050] In some embodiments, the specific implementation of steps 201-202 and the technical effects brought about can be referred to Figure 1 The corresponding steps 101-102 in the embodiments are not described in detail here.

[0051] Step 203: In response to determining that the configuration information of the mixed reality device does not meet the preset configuration condition, turn off the tracker.

[0052] In some embodiments, in response to determining that the configuration information of the mixed reality device does not meet the preset configuration condition, the execution subject may turn off the tracker. The configuration information may be a version code of the operating system of the mixed reality device. For example, the version code may be IOS16. The preset configuration condition may be that the version of the operating system of the mixed reality device is IOS16 or above. The tracker may be ARSession.

[0053] Step 204, according to the depth information of the target virtual image in the scene, adjust the focal length of the camera of the mixed reality device to the position corresponding to the depth information, and obtain the captured high-definition scene image of the real space from the camera data of the mixed reality device.

[0054] In some embodiments, the specific implementation of step 204 and the technical effects brought about can be referred to Figure 1 The corresponding step 103 in the embodiments will not be described in detail here.

[0055] Step 205 , in response to determining that the configuration information of the mixed reality device does not meet the preset configuration condition, start the tracker and reposition to restore the association relationship between the virtual media information and the real space.

[0056] In some embodiments, in response to determining that the configuration information of the mixed reality device does not meet the preset configuration conditions, the execution entity may turn on the tracker and reposition to restore the association between the virtual media information and the real space. In one or more embodiments, the repositioning may be to recalculate and calibrate the position and posture of the mixed reality device relative to the real space. In practice, the execution entity may use image recognition and inertial measurement technology to recalculate and calibrate the position and posture of the mixed reality device relative to the real space.

[0057] Step 206 , superimpose the target virtual picture onto the high-definition scene image of the real space in a manner of anchoring the target virtual picture to the real space, so as to obtain an augmented reality image.

[0058] In some embodiments, the specific implementation of step 206 and the technical effects brought about can be referred to Figure 1 The corresponding step 104 in the embodiments will not be described in detail here.

[0059] from Figure 2 It can be seen that Figure 1 Compared with the description of some corresponding embodiments, Figure 2 The process 200 of the augmented reality image generation method in the mixed reality space in some corresponding embodiments reflects the process of obtaining a high-definition scene image of the real space when the operating system version of the mixed reality device is below IOS16. When the operating system version of the mixed reality device is below IOS16, ARSession is closed before obtaining the high-definition scene image from the camera data of the mixed reality device. After the high-definition scene image is obtained, ARSession is turned on to obtain the association between the virtual media information and the real space. Therefore, regardless of whether the operating system version of the mixed reality device is below IOS16 or IOS16 and above, high-definition augmented reality images can be quickly obtained. The efficiency and clarity of generating augmented reality images are improved. Thereby, the user's experience of browsing augmented reality images is improved.

[0060] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an augmented reality image generation device in a mixed reality space. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0061] like Figure 3As shown, in some embodiments, an augmented reality image generating device 300 in a mixed reality space includes: a display unit 301, a generation unit 302, an adjustment unit 303 and an overlay unit 304. The display unit 301 is configured to display the mixed reality space in the mixed reality device in response to determining that the mixed reality device and the real space are successfully matched, wherein the mixed reality space includes a virtual screen anchored to the real space; the generation unit 302 is configured to extract the current virtual screen frame from the original augmented reality image frame sequence in response to detecting the user's shooting operation, and generate a target virtual screen according to the current virtual screen frame; the adjustment unit 303 is configured to adjust the focal length of the camera of the mixed reality device to the position corresponding to the depth information of the target virtual screen in the scene, and obtain the captured high-definition scene image of the real space from the camera data of the mixed reality device; the overlay unit 304 is configured to overlay the target virtual screen onto the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image.

[0062] Optionally, the generation unit 302 may include: a creation unit, an acquisition unit, an assignment unit, and a second generation unit (not shown in the figure). The creation unit is configured to create an initial virtual view; the acquisition unit is configured to acquire virtual media information and scene attribute information of the scene corresponding to the current virtual picture frame. The assignment unit is configured to assign the virtual media information and the scene attribute information to the initial virtual view to obtain a target view; the second generation unit is configured to generate a target virtual picture according to the target view.

[0063] Optionally, the augmented reality image generating device 300 in the mixed reality space may further include: a shooting unit (not shown in the figure), configured to shoot a high-definition scene image of the real space in response to detecting that the posture of a person in the real scene screen meets a preset condition.

[0064] Optionally, the augmented reality image generating device 300 in the mixed reality space may further include: a second superimposition unit (not shown in the figure), configured to superimpose preset watermark information on the above-mentioned augmented reality image in response to detecting that the watermark setting function is turned on.

[0065] Optionally, the augmented reality image generating device 300 in the mixed reality space may further include: a shut-down unit (not shown in the figure), configured to shut down the tracker in response to determining that the configuration information of the mixed reality device does not meet the preset configuration conditions.

[0066] Optionally, the augmented reality image generating device 300 in the mixed reality space may further include: an activation unit (not shown in the figure), configured to activate the tracker and reposition in response to determining that the configuration information of the mixed reality device does not meet the preset configuration conditions, so as to restore the association between the virtual media information and the real space.

[0067] Optionally, the augmented reality image generating device 300 in the mixed reality space may further include: a drawing unit and a determination unit (not shown in the figure). The drawing unit is configured to draw the augmented reality image on the display screen; the determination unit is configured to determine the display direction of the augmented reality image according to the rotation direction in response to detecting screen rotation.

[0068] It can be understood that the units recorded in the augmented reality image generating device 300 in the mixed reality space are similar to the units described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the augmented reality image generation device 300 and the units contained therein in the mixed reality space, and will not be repeated here.

[0069] Reference below Figure 4 , which shows a schematic structural diagram of an electronic device (eg, a computing device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0070] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0071] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0072] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0073] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0074] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0075] The computer-readable medium may be included in the electronic device, or may exist separately without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to determining that the mixed reality device matches the real space successfully, the mixed reality space is displayed in the mixed reality device, wherein the mixed reality space includes a virtual screen anchored to the real space; in response to detecting the user's shooting operation, the current virtual screen frame is extracted from the original augmented reality image frame sequence, and the target virtual screen is generated according to the current virtual screen frame; according to the depth information of the target virtual screen in the scene, the focal length of the camera of the mixed reality device is adjusted to the position corresponding to the depth information, and the high-definition scene image of the real space captured is obtained from the camera data of the mixed reality device; the target virtual screen is superimposed on the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image.

[0076] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0077] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including a display unit, a generation unit, an adjustment unit, and an overlay unit. Among them, the names of these units do not constitute limitations on the units themselves in certain circumstances, for example, the overlay unit may also be described as "a unit that overlays the above-mentioned target virtual picture onto the above-mentioned high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image".

[0079] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0080] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for generating an augmented reality image in a mixed reality space, comprising: In response to determining that the mixed reality device and the real space are successfully matched, displaying the mixed reality space in the mixed reality device, wherein the mixed reality space includes a virtual picture anchored to the real space; In response to detecting a shooting operation of the user, extracting a current virtual picture frame from the original augmented reality image frame sequence, and generating a target virtual picture according to the current virtual picture frame; According to the depth information of the target virtual picture in the scene, adjusting the focal length of the camera of the mixed reality device to the position corresponding to the depth information, and acquiring the captured high-definition scene image of the real space from the camera data of the mixed reality device; The target virtual picture is superimposed on the high-definition scene image of the real space in a manner of anchoring with the real space to obtain an augmented reality image.

2. The method according to claim 1, wherein: Before extracting the current virtual picture frame from the original augmented reality image frame sequence and generating the target virtual picture according to the current virtual picture frame, the method further includes: In response to detecting that the posture of a person in a real scene image reaches a preset condition, a high-definition scene image of the real space is captured.

3. The method according to claim 1, wherein: The method further comprises: In response to detecting that the watermark setting function is turned on, preset watermark information is superimposed on the augmented reality image.

4. The method according to claim 1, wherein: Before acquiring the captured high-definition scene image of the real space, the method further includes: In response to determining that the configuration information of the mixed reality device does not meet a preset configuration condition, turning off the tracker.

5. The method according to claim 4, wherein: After acquiring the captured high-definition scene image of the real space, the method further includes: In response to determining that the configuration information of the mixed reality device does not meet the preset configuration condition, the tracker is turned on and repositioned to restore the association between the virtual media information and the real space.

6. The method according to claim 2, wherein: The step of extracting a current virtual picture frame from the original augmented reality image frame sequence and generating a target virtual picture according to the current virtual picture frame comprises: Create an initial virtual view; Acquire virtual media resource information and scene attribute information of the scene corresponding to the current virtual picture frame; Assigning the virtual media resource information and the scene attribute information to the initial virtual view to obtain a target view; A target virtual screen is generated according to the target view.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: Drawing the augmented reality image on a display screen; In response to detecting screen rotation, a display direction of the augmented reality image is determined according to the rotation direction.

8. An augmented reality image generation device in a mixed reality space, comprising: A display unit, configured to display a mixed reality space in the mixed reality device in response to determining that the mixed reality device is successfully matched with the real space, wherein the mixed reality space includes a virtual picture anchored to the real space; A generating unit, configured to extract a current virtual picture frame from the original augmented reality image frame sequence in response to detecting a shooting operation of the user, and generate a target virtual picture according to the current virtual picture frame; an adjustment unit configured to adjust the focal length of the camera of the mixed reality device to a position corresponding to the depth information according to the depth information of the target virtual picture in the scene, and to obtain a high-definition scene image of the real space captured from the camera data of the mixed reality device; The superimposition unit is configured to superimpose the target virtual picture onto the high-definition scene image of the real space in a manner anchored to the real space to obtain an augmented reality image.

9. An electronic device, comprising: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images