Augmented reality image processing method, device, electronic device, and storage medium

Abstract

An augmented reality image processing method, device, electronic device, and storage medium include the steps of acquiring a target image including a target object in response to an image acquisition command triggered by a user, acquiring an augmented reality model of the target object, and combining the target object with the augmented reality model to output the augmented reality model, acquiring target audio data selected by the user and determining time-sequential audio characteristics based on the target audio data, and, when the target audio data is output, driving the augmented reality model according to the playback progress and audio characteristics of the target audio data. When the augmented reality model is output, it can be combined with the audio characteristics of the target audio data selected by the user to drive the output of an augmented display model, allowing the user to participate in the display process of the augmented display model, and selecting different target audio data can drive the augmented reality model to display according to the audio characteristics of the target audio data, improving usability.

Description

[Technical field]

[0001] The present invention relates to a virtual reality technology, and more particularly to an image processing method, device, electronic device, and storage medium for augmented reality. [Background technology]

[0002] Augmented reality (AR) is a technology that superimposes real information and virtual information. First, a computer system processes real information, matches it based on the real information, and generates virtual information in the form of virtual objects, sounds, or characters. Then, the virtual information is superimposed on a man-machine interaction interface that displays the real information, thereby improving the user's perception of the real world.

[0003] Currently, augmented reality models can only be displayed in a predefined manner, which means that the display method is single, lacks interactivity, and is difficult to use. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention provides an image processing method, device, electronic device, and storage medium for augmented reality in order to improve the interactivity of an augmented reality model and improve usability. [Means for solving the problem]

[0005] In a first aspect, an image processing method for augmented reality provided in an embodiment of the present invention includes: capturing a target image including the target object in response to an image capture command triggered by a user; obtaining an augmented reality model of the target object and combining the augmented reality model with the target object and outputting the augmented reality model; obtaining target audio data selected by a user, and determining audio characteristics having a time series characteristic based on the target audio data; When the target audio data is output, driving the augmented reality model according to the playback progress and audio characteristics of the target audio data.

[0006] In a second aspect, an embodiment of the present invention further provides an augmented reality image processing device, comprising: a target image capture module for capturing a target image including the target object in response to an image capture command triggered by a user; an augmented reality model acquisition module for acquiring an augmented reality model of the target object and combining the augmented reality model with the target object and outputting the augmented reality model; a target audio acquisition module for acquiring target audio data selected by a user; an audio characteristic determination module for determining time-series audio characteristics based on the target audio data; and an output module for driving the augmented reality model according to the playback progress and audio characteristics of the target audio data when the target audio data is output.

[0007] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, they cause the one or more processors to implement the augmented reality image processing method illustrated in the embodiments of the present disclosure.

[0008] In a fourth aspect, an embodiment of the present invention further provides a storage medium including computer-executable instructions, which, when executed by a computer processor, are used to perform the augmented reality image processing method set forth in the embodiment of the present disclosure.

[0009] In a fifth aspect, an embodiment of the present invention further provides a computer program product, the computer program product including a computer program stored in a readable storage medium, wherein one or more processors of an electronic device are capable of reading the computer program from the readable storage medium, and the one or more processors execute the computer program to cause the electronic device to perform the augmented reality image processing method illustrated in the embodiment of the present disclosure.

[0010] In a sixth aspect, an embodiment of the present disclosure further provides a computer program stored in a readable storage medium, wherein one or more processors of an apparatus are capable of reading the computer program from the readable storage medium, and the one or more processors execute the computer program to cause the electronic apparatus to perform the augmented reality image processing method illustrated in the embodiment of the present disclosure. Effect of the Invention

[0011] The image processing means of augmented reality disclosed in the embodiment of the present disclosure can respond to an image acquisition command triggered by a user to acquire a target image including a target object, acquire an augmented reality model of the target object, combine the augmented reality model with the target object, and output the augmented reality model, acquire target audio data selected by a user, and determine audio characteristics having time series based on the target audio data, and when the target audio data is output, drive the augmented reality model according to the playback progress and audio characteristics of the target audio data. Currently, augmented reality models lack interactivity and are difficult to use, whereas the image processing means of augmented reality disclosed in the embodiment of the present disclosure can combine the audio characteristics of the target audio data selected by a user to generate an augmented reality model when the augmented reality model is output. reality The output of the model can be driven and extended by the user. realityThe user can participate in the display process of the model, and by selecting different target audio data, the augmented reality model is driven to display according to the audio characteristics of the target audio data, thereby improving usability. [Brief description of the drawings]

[0012] [Figure 1] 1 is a flowchart of an image processing method for augmented reality in accordance with a first embodiment of the present invention. [Diagram 2] FIG. 11 is a structural schematic diagram of an augmented reality image processing device according to a second embodiment of the present invention; [Diagram 3] FIG. 11 is a structural schematic diagram of an electronic device according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present invention will be described in more detail with reference to the drawings and examples below. It will be understood that the specific examples described herein are only for the purpose of interpreting the present invention, and do not limit the present invention. For the sake of convenience, the drawings show only parts related to the present invention, and do not show all structures. Example 1

[0014] FIG. 1 is a flowchart of an image processing method for augmented reality provided in a first embodiment of the present invention. This embodiment can be applied to displaying an augmented reality model. The method can be performed by an electronic device that realizes augmented reality. The electronic device can be a smartphone, a tablet, etc. The method The method includes the following steps:

[0015] At step 110, in response to an image capture command triggered by a user, a target image including the target object is captured.

[0016] A user can issue an image acquisition command in the preset application, and when the user issues the image acquisition command, the electronic device acquires a target image with a camera. A user can launch the preset application with the intention of using the augmented reality model. A screen of a target object is photographed by the electronic device. The target object may be an object with an augmented reality model, such as a landmark building. The landmark building may be a building with a unique design style in a certain area.

[0017] The preset application may be a photography application of the electronic device or an application with an augmented reality function. The camera of the electronic device captures a target image, and the electronic device displays the captured target image on a preview page. The preview page can provide a user with a real-time image captured by the camera.

[0018] In step 120, an augmented reality model of the target object is obtained and combined with the target object to output the augmented reality model.

[0019] If it is recognized that there exists an augmented reality model corresponding to the subject in the current image, the augmented reality model is mapped to the target object in the target image, and the augmented reality model is output in combination with the target object.

[0020] In addition, an interface can be provided to the user to manually adjust the size of the augmented reality model. When the machine's combination effect is insufficient, for example, the augmented reality model and the target object cannot be accurately combined, the size of the augmented reality model can be adjusted through the interface, further enabling the user to adjust the size of the augmented reality model and improving usability.

[0021] Furthermore, in one implementation, when the target object is a landmark building, obtaining an augmented reality model of the target object can be performed in the following manner.

[0022] the goal When it is monitored that the image contains a building, a target object identifier is determined based on the current position information and the shooting angle of the electronic device, and the augmented reality model indicated by the target object identifier is determined as the augmented reality model of the target object.

[0023] Obtain positioning information from the Global Positioning System (GPS) of the electronic device. Obtain the orientation of the electronic device using a gyroscope, and use the orientation as the shooting angle. Determine whether or not a landmark building is present within a predetermined shooting range based on the positioning information and the shooting angle. If a landmark building is present, use an augmented reality model of the landmark building as the augmented reality model of the target object.

[0024] Furthermore, in another implementation, obtaining an augmented reality model of a target object can be performed in the following manner.

[0025] A base image of the target object is determined from the target image, and an augmented reality model of the target object is determined based on the base image. The base image may be an image of the target object in the target image. The network-side server may perform image analysis on the base image to further determine what the target object in the base image is specifically. The target object may be an object such as a building, a vehicle, clothing, footwear, or a cap.

[0026] The above two implementations may be used to retrieve an augmented reality model of a target object, and may be used to verify whether the augmented reality model of a target object is accurate.

[0027] In step 130, target audio data selected by a user is obtained, and a time-series audio characteristic is determined based on the target audio data.

[0028] The target audio data may be audio data such as a song or a recording selected by a user, the audio data may be locally stored audio data, the audio data may be audio data selected by a user from a list of songs provided by a server, or the target audio data may be audio data input by a user in real time.

[0029] In one embodiment, Audio feature detection is performed on the target audio data to obtain audio features having time series of the target audio data, including one or more combinations of accents, strong beats, or beats.

[0030] Audio characteristic detection is used to analyze the regularity of beats, accents, and the positions and regularity of strong beats in the target audio data. A beat is a unit that represents the rhythm of the target audio data. In the target audio data, a series of beats with a predetermined strength and weakness appear repeatedly at a predetermined time interval. For example, there are 2 / 4 beats (2 / 4 beats), 4 / 4 beats (4 / 4 beats), and 3 / 4 beats (3 / 4 beats). Beats change periodically over time. An accent is a note in the target audio that has a louder volume. A strong beat refers to a beat with stress in the beat.

[0031] In step 140, when the target audio data is output, the augmented reality model is driven according to the playback progress and audio characteristics of the target audio data.

[0032] After the user selects the target audio data, the target audio data is output by the audio output module of the electronic device. At the same time as outputting the target audio data, the current playback progress and BIO The augmented reality model is driven according to the audio characteristics.

[0033] For example, the appearance of an accent, an underbeat, or a strong beat may be driven to change the color of the entire augmented reality model, with different color changes corresponding to the strong beat, the underbeat, or the accent.

[0034] In one embodiment, The augmented reality model includes a plurality of model units. Driving the augmented reality model in response to a playback progress and audio characteristics of the target audio data includes driving model units in the augmented reality model in response to the playback progress and audio characteristics of the target audio data.

[0035] The augmented reality model may be composed of multiple model units, each of which may be a cube unit. Multiple cube units are joined together to form an augmented reality model of the target object. unit may be processed in parallel by a shader. A model unit in the augmented reality model is driven according to the playback progress and audio characteristics of the target audio data.

[0036] In one embodiment, The step of driving the model unit in the augmented reality model according to the playback progress and audio characteristics of the target audio data includes the following steps.

[0037] In step 1, a target time and a target width for a model unit to change its shape are determined in response to audio characteristics having a time series characteristic.

[0038] The time during which an upbeat, a upbeat, and an accent are present in the beat may be set as the target time, and the target width may be determined according to the degree of the upbeat, the upbeat, and the accent.

[0039] In step 2, if the playback progress of the target audio data is within the target time, the model unit in the augmented reality model is driven according to the target width.

[0040] The driving method of the model unit may include protrusion movement, color change, transparency change, and the like.

[0041] Exemplarily, the step of driving the model unit in the augmented reality model according to the target width includes: The method includes driving a preset model unit in the augmented reality model to perform a protrusion operation, or driving a plurality of model units in the augmented reality model to perform a color change, or driving a plurality of model units in the augmented reality model to perform a transparency change.

[0042] When a preset model unit in the augmented reality model is driven to perform a protrusion operation, the preset model may be a randomly selected model unit. A protrusion width of the protrusion operation is determined according to a target width. When multiple model units in the augmented reality model are driven to perform a color change, multiple model units may be randomly selected from all model units to perform a color change, or a color change may be performed for all model units. Transparency refers to the transparency of the texture pattern of the model unit. The texture image may be a solid color or an actual texture pattern of the target object.

[0043] Furthermore, after obtaining the target audio data selected by the user, The method comprises: The method further includes the steps of obtaining emotional features of the target audio data, determining first deformation features of the augmented reality model based on the emotional features, wherein the first deformation features are used to drive a shape of the augmented reality model to match the emotion expressed by the emotional features, and driving the augmented reality model based on the first deformation features when the target audio data is output.

[0044] The emotion feature may be determined based on the title of the target audio data. Or, the emotion feature may be determined based on the content of the text input by the user in the target audio. The emotion feature is used to express the emotion movement of the audio content of the target audio data, for example, cheerful, sad, etc. Deformation features corresponding to different emotion features may be pre-arranged. For example, in the first deformation feature corresponding to the cheerful emotion feature, the protrusion width of the model unit is high and the frequency of protrusion is fast. For example, in the first deformation feature corresponding to the sad emotion feature, the protrusion width of the model unit is low and the frequency of protrusion is slow. For example, after photographing a TV tower, if the emotion feature is heartbroken, the TV tower model will have a bent shape. If there is a deformation in the building model, the original image of the building is deleted by a trimming method, and the original image area is covered with the model.

[0045] Furthermore, after obtaining the target audio data selected by the user, The method further includes the steps of acquiring a user's body movement or facial expression, determining a second deformation characteristic of the augmented reality model in accordance with the body movement or facial expression, the second deformation characteristic being used to drive a shape of the augmented reality model to match the body movement or facial expression, and driving the augmented reality model based on the second deformation characteristic when the target audio data is output.

[0046] The user's limb motion can be obtained by limb detection. The user's facial expression can be obtained by face recognition. A second deformation feature is generated based on the limb motion or facial expression. For example, after photographing a TV tower, if the facial expression is sad, the second deformation feature is a bent shape, and the TV tower model then has a bent shape. If the building model is deformed, the original image of the building is deleted by a cropping method, and the original image area is covered with the model.

[0047] The augmented reality image processing method disclosed in the embodiment of the present disclosure responds to an image acquisition command triggered by a user to acquire a target image including a target object, acquire an augmented reality model of the target object, combine the augmented reality model with the target object, output the augmented reality model, acquire target audio data selected by the user, determine audio characteristics having time series based on the target audio data, and when the target audio data is output, drive the augmented reality model according to the playback progress and audio characteristics of the target audio data. Currently, augmented reality models lack interactivity and are difficult to use, whereas the augmented reality image processing method disclosed in the embodiment of the present disclosure acquires a target image including a target object in response to an image acquisition command triggered by a user, acquires an augmented reality model of the target object, combines the audio characteristics of the target audio data selected by the user with the augmented reality model when the augmented reality model is output. reality The output of the model can be driven and extended by the user. reality The user can participate in the display process of the model, and by selecting different target audio data, the augmented reality model is driven to display according to the audio characteristics of the target audio data, thereby improving usability. Example 2

[0048] FIG. 2 is a structural schematic diagram of an augmented reality image processing device provided in the second embodiment of the present disclosure, which can be applied to displaying an augmented reality model. , this The apparatus includes a target image acquisition module 210 , an augmented reality model acquisition module 220 , a target audio acquisition module 230 , an audio characteristic determination module 240 or an output module 250 .

[0049] The target image capture module 210 is used to capture a target image including a target object in response to an image capture command triggered by a user; The augmented reality model acquisition module 220 is used for acquiring an augmented reality model of a target object, and outputting the augmented reality model in combination with the target object; The target audio acquisition module 230 is used to acquire target audio data selected by a user; The audio characteristic determination module 240 is used to determine a time-series audio characteristic based on the target audio data; The output module 250 is used to drive the augmented reality model according to the playback progress and audio characteristics of the target audio data when the target audio data is output.

[0050] Additionally, the audio characteristic determination module 240 Audio feature detection is performed on the target audio data to obtain time-series audio features of the target audio data, including one or more combinations of accents, strong beats, or beats.

[0051] Furthermore, the augmented reality model includes a plurality of model units, and the output module 250 The target audio data is used to drive a model unit in the augmented reality model according to the playback progress and audio characteristics of the target audio data.

[0052] Additionally, the output module 250 determining a target time and a target width for the model unit to undergo a morphological change in response to a time-series audio characteristic; If the playback progress of the target audio data is the target time, it is used to drive a model unit in the augmented reality model according to the target width.

[0053] Additionally, the output module 250 Driving a preset model unit in the augmented reality model to perform a protruding motion, or Driving a plurality of model units in the augmented reality model to perform a color change, or A number of model units in the augmented reality model are used to drive the implementation of transparency changes.

[0054] moreover, The device comprises: The method further includes a first deformation feature acquisition module, the first deformation feature acquisition module comprising: Obtaining emotion features of the target audio data; determining a first deformation characteristic of the augmented reality model based on the emotion characteristic, the first deformation characteristic being used to drive a shape of the augmented reality model to conform to the emotion expressed by the emotion characteristic; and When the target audio data is output, it is used to drive the augmented reality model based on the first deformation characteristic.

[0055] moreover, The device comprises: The method further includes a second deformation feature acquisition module, the second deformation feature acquisition module comprising: Acquiring a user's body movement or facial expression; determining a second deformation characteristic of the augmented reality model in response to a body movement or a facial expression, the second deformation characteristic being used to drive a shape of the augmented reality model to match the body movement or the facial expression; and When the target audio data is output, it is used to drive the augmented reality model based on the second deformation characteristic.

[0056] In the augmented reality image processing device disclosed in the embodiment of the present disclosure, the target image acquisition module 210 acquires a target image including a target object in response to an image acquisition command triggered by a user, the augmented reality model acquisition module 220 acquires an augmented reality model of the target object and combines it with the target object to output the augmented reality model, the target audio acquisition module 230 acquires target audio data selected by the user, the audio characteristic determination module 240 determines audio characteristics having time series based on the target audio data, and the output module 250 drives the augmented reality model according to the playback progress and audio characteristics of the target audio data when the target audio data is output. Currently, augmented reality models lack interactivity and are difficult to use, whereas the augmented reality image processing device disclosed in the embodiment of the present disclosure combines the audio characteristics of the target audio data selected by the user to generate an augmented reality model when the augmented reality model is output. realityThe output of the model can be driven and extended by the user. reality The user can participate in the display process of the model, and by selecting different target audio data, the augmented reality model is driven to display according to the audio characteristics of the target audio data, thereby improving usability.

[0057] The augmented reality image processing device provided in an embodiment of the present invention can execute the augmented reality image processing method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to executing the method. Example 3

[0058] Referring to FIG. 3 below, a structural schematic diagram of an electronic device 800 suitable for implementing the third embodiment of the present disclosure is shown. The terminal device in the embodiment of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a personal digital assistant (PDA), a portable android device (PAD), a portable media player (PMP), an in-vehicle terminal (e.g., an in-vehicle navigation terminal), and a fixed terminal such as a digital TV (Television) and a desktop computer. The electronic device shown in FIG. 3 is only an example and does not impose any restrictions on the function and scope of use of the embodiment of the present disclosure.

[0059] 3, the electronic device 800 may include a processing unit (e.g., a central processing unit, a graphic processor, etc.) 801, which can execute various appropriate operations and processes according to a program stored in a read only memory (ROM) 802 or loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 further stores various programs and data necessary for the operation of the electronic device 800. The processing unit 801, the ROM 802, and the RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0060] In general, the I / O interface 805 can be connected to input devices 806 including a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc., output devices 807 including a liquid crystal display (LCD), speaker, vibrator, etc., storage devices 808 including a magnetic tape, hard disk, etc., and communication devices 809. The communication devices 809 enable the electronic device 800 to communicate and exchange data with other devices wirelessly or via wires. Although FIG. 3 illustrates the electronic device 800 having various devices, it should be understood that it is not required to implement or include all of the devices shown. Alternatively, more or fewer devices may be implemented or included.

[0061] In particular, according to an embodiment of the present disclosure, the above process described with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a computer-readable storage medium, the computer program including program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication device 809, or may be installed from the storage device 808 or from the ROM 802. When the computer program is executed by the processing device 801, it performs the steps of the method of the embodiment of the present disclosure and realizes the limited functions described above.

[0062] A computer program product provided in one or more embodiments of the present disclosure includes a computer program stored in a readable storage medium, the computer program being readable from the readable storage medium by one or more processors of an electronic device, the one or more processors executing the computer program to cause the electronic device to perform the means provided in any of the above embodiments.

[0063] A computer program provided in one or more embodiments of the present disclosure is stored in a readable storage medium, and one or more processors of an electronic device can read the computer program from the readable storage medium, and the one or more processors execute the computer program to cause the electronic device to perform the means provided in any of the above embodiments.

[0064] It should be noted that the computer readable medium of the present disclosure may be a computer readable signal medium, or a computer readable storage medium, or any combination of the two. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.The program code contained in the computer readable medium may be transmitted over any suitable medium, including but not limited to electrical wire, optical cable, radio frequency (RF), or the like, or any suitable combination of the above.

[0065] The computer-readable medium may be included in the electronic device, or may exist independently of the electronic device.

[0066] The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to obtain at least two Internet Protocol addresses, send a node evaluation request to a node evaluation device including the at least two Internet Protocol addresses, where the node evaluation device selects and returns an Internet Protocol address from the at least two Internet Protocol addresses, and receive the Internet Protocol address returned by the node evaluation device, where the obtained Internet Protocol address indicates an edge node in a content distribution network.

[0067] Or, the computer readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to receive a node evaluation request including at least two Internet Protocol addresses, select an Internet Protocol address from the at least two Internet Protocol addresses, and return the selected Internet Protocol address, where the received Internet Protocol address indicates an edge node in the content distribution network.

[0068] Computer program code for carrying out the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages, or a combination thereof. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer may be connected to the user's computer by any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., connected by the Internet using an Internet Service Provider).

[0069] The flowcharts and block diagrams in the drawings illustrate possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code that includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some permutation implementations, the functions noted in the blocks may occur in a different order than the order noted in the drawings. For example, two blocks shown in succession may actually be executed essentially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated system with hardware that executes the specified functions or operations, or may be implemented in a combination of dedicated hardware and computer instructions.

[0070] The units described in the embodiments of the present disclosure may be implemented in a software manner or a hardware manner, where the names of the units may not be limited to the units themselves, for example, the first acquiring unit may be described as "a unit for acquiring at least two Internet Protocol addresses".

[0071] The above description is merely an explanation of the preferred embodiments of the present disclosure and the principles of its application technology. Those skilled in the art will understand that the scope of the present disclosure is not limited to the technical solutions based on the specific combinations of the above technical features, but also includes other technical solutions based on any combination of the above technical features or their equivalent features without departing from the concept of the above disclosure. For example, the above features are technical solutions formed by mutually replacing technical features having similar functions disclosed in the present disclosure (but not limited thereto).

[0072] This application claims priority to a Chinese patent application bearing application number 202010662819.X and entitled "Augmented reality image processing method, apparatus, electronic device and storage medium" filed with the China Patent Office on July 10, 2020, the entire contents of which are incorporated herein by reference.

Claims

1. capturing a target image including the target object in response to an image capture command triggered by a user; obtaining an augmented reality model of the target object and combining the augmented reality model with the target object and outputting the augmented reality model; obtaining target audio data selected by the user, and determining audio characteristics having a time series characteristic based on the target audio data; when the target audio data is output, driving the augmented reality model according to a playback progress status and the audio characteristics of the target audio data; the augmented reality model includes a plurality of model units, the plurality of model units being a plurality of cubic units, and the plurality of cubic units are joined together to form the augmented reality model; The step of outputting the augmented reality model in combination with the target object includes: When the user receives a trigger command via an interface for manually adjusting the size of the augmented reality model, adjusting the size of the augmented reality model of the target object based on the command, and outputting the augmented reality model in combination with the target object.

2. An augmented reality image processing method comprising:

2. The step of determining a time-series audio characteristic based on the target audio data includes: performing audio feature detection on the target audio data to obtain audio features having the time series of the target audio data, the audio features including one or more combinations of accents, strong beats, or beats; 2. The method of claim 1 .

3. Driving the augmented reality model according to the playback progress of the target audio data and the audio characteristics, driving a model unit in the augmented reality model according to a playback progress of the target audio data and the audio characteristics; 3. The method according to claim 1 or 2.

4. The step of driving a model unit in the augmented reality model according to the playback progress of the target audio data and the audio characteristics includes: determining a target time and a target width for the model unit to change its shape in accordance with the time-series audio characteristics; and if the playback progress of the target audio data is equal to the target time, driving a model unit in the augmented reality model according to the target width.

4. The method according to claim 3 .

5. The step of driving a model unit in the augmented reality model in accordance with the target width includes: Driving a preset model unit in the augmented reality model to perform a protruding motion; or driving a plurality of model units in the augmented reality model to perform a color change; or driving a plurality of model units in the augmented reality model to perform transparency changes; 5. The method of claim 4.

6. After obtaining the target audio data selected by the user, obtaining emotion features of the target audio data; determining first deformation features of the augmented reality model based on the emotion features, the first deformation features being used to drive a shape of the augmented reality model to conform to an emotion expressed by the emotion features; and driving the augmented reality model based on the first deformation characteristic when the target audio data is output. The method according to any one of claims 1 to 5.

7. After obtaining the target audio data selected by the user, acquiring a body movement or a facial expression of the user; determining second deformation characteristics of the augmented reality model in response to the body movement or facial expression, the second deformation characteristics being used to drive a shape of the augmented reality model to match the body movement or facial expression; and driving the augmented reality model based on the second deformation characteristic when the target audio data is output. The method according to any one of claims 1 to 5.

8. a target image capture module for capturing a target image including the target object in response to an image capture command triggered by a user; an augmented reality model acquisition module for acquiring an augmented reality model of the target object and outputting the augmented reality model in combination with the target object; a target audio acquisition module for acquiring target audio data selected by the user; an audio characteristic determination module for determining a time-series audio characteristic based on the target audio data; an output module for driving the augmented reality model according to a playback progress of the target audio data and the audio characteristics when the target audio data is output; the augmented reality model includes a plurality of model units, the plurality of model units being a plurality of cubic units, and the plurality of cubic units are joined together to form the augmented reality model; The augmented reality model acquisition module is used to, when the user receives a trigger command via an interface for manually adjusting the size of the augmented reality model, adjust the size of the augmented reality model of the target object according to the command, and output the augmented reality model in combination with the target object. An augmented reality image processing device comprising:

9. one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are caused to realize the augmented reality image processing method according to any one of claims 1 to 7.

1. An electronic device comprising:

10. A storage medium containing computer executable instructions, which when executed by a computer processor, cause the computer processor to perform the augmented reality image processing method according to any one of claims 1 to 7. storage medium.

11. A method for processing an augmented reality image according to any one of claims 1 to 7, A computer program comprising:

Images