Method and device for processing augmented reality picture, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure provide a processing method and device for an augmented reality picture, an electronic device and a storage medium. The method comprises: in response to a rendering trigger request for an augmented reality picture, displaying a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture; and in response to a display adjustment operation for the linear object, displaying an adjusted three-dimensional model of the linear object in the augmented reality picture, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation. The technical solution of the embodiments of the present disclosure displays a three-dimensional model of a linear object in an augmented reality picture, not only more realistically integrates the linear object into the augmented reality picture, but also improves the stereoscopic and realistic sense of the linear object in the augmented reality picture, and effectively interacts with the linear object according to the individual needs of the user, thereby improving the user experience.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the computer technology field, and in particular, to an augmented reality picture processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Augmented reality (AR) technology is a technology that acquires a real world picture by real-time shooting of the real world and superimposes virtual information on the real world picture.

[0003] In related technologies, a two-dimensional line is used to enrich and assist the augmented reality picture, that is, the two-dimensional line is drawn on the augmented reality picture. However, this processing manner often makes the line in the augmented reality picture appear stiff and lack stereoscopic effect, thereby affecting the picture display quality. Moreover, generally, the display manner of the two-dimensional line is relatively fixed with the display manner of the picture, and cannot well meet the personalized interaction demand of a user, thereby affecting the user experience. SUMMARY

[0004] The present disclosure provides an augmented reality picture processing method and device, an electronic device, and a storage medium, to realize not only more realistic integration of a linear object into an augmented reality picture, to improve the stereoscopic effect and reality of the linear object in the augmented reality picture, but also effective interaction of the linear object according to the personalized demand of a user, to improve the user experience.

[0005] In a first aspect, the embodiments of the present disclosure provide an augmented reality picture processing method, which comprises:

[0006] displaying a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture;

[0007] displaying an adjusted three-dimensional model of the linear object in the augmented reality picture in response to a display adjustment operation for the linear object, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

[0008] In a second aspect, the embodiments of the present disclosure further provide an augmented reality picture processing device, which comprises:

[0009] a request module configured to display a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture;

[0010] display a three-dimensional model of the linear object in the augmented reality picture in response to a display adjustment operation on the three-dimensional model, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

[0011] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises:

[0012] one or more processors;

[0013] a storage device configured to store one or more programs,

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the processing method of the augmented reality picture according to any of the embodiments of the present disclosure.

[0015] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are configured to perform the processing method of the augmented reality picture according to any of the embodiments of the present disclosure.

[0016] The technical solution of the embodiments of the present disclosure can display a three-dimensional model of a linear object corresponding to an augmented reality picture in the augmented reality picture in response to a rendering trigger request on the augmented reality picture, which can improve the stereoscopic and realistic sense of the linear object in the augmented reality picture. In response to a display adjustment operation on the linear object, a three-dimensional model of the linear object is displayed in the augmented reality picture, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation. The technical solution of the embodiments of the present disclosure can display a three-dimensional model of a linear object in an augmented reality picture, which not only more realistically integrates the linear object into the augmented reality picture, but also enables effective interaction with the linear object according to the individual needs of the user, thereby improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent upon reading the following detailed description in conjunction with the accompanying drawings, in which like reference numerals refer to like elements. It is to be understood that the drawings are designed solely for the purpose of illustration and not as a definition of the limits of the disclosure, for which reference should be made only to the appended claims. Throughout the drawings, like reference numerals indicate like elements.

[0018] Figure 1 A flowchart of a processing method of an augmented reality picture according to an embodiment of the present disclosure;

[0019] Figure 2A flowchart of a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0020] Figure 3 A flowchart of a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0021] Figure 4 A flowchart of a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0022] Figure 5 A flowchart of a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0023] Figure 6A An example diagram of rendering of a three-dimensional model of a linear object in a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0024] Figure 6B An example diagram of rendering of a three-dimensional model of a linear object in another processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0025] Figure 7 A flowchart of an acquisition manner of a key point of a linear object in a virtual scene editing method provided by an embodiment of the present disclosure;

[0026] Figure 8 An example diagram of a triangular primitive on a cross section of a three-dimensional model of a linear object in a virtual scene editing method provided by an embodiment of the present disclosure;

[0027] Figure 9 An example diagram of a triangular primitive on a connecting surface of a three-dimensional model of a linear object in a virtual scene editing method provided by an embodiment of the present disclosure;

[0028] Figure 10 An example diagram of intermittent rendering of a three-dimensional model of a linear object in a processing method of an augmented reality picture provided by an embodiment of the present disclosure;

[0029] Figure 11 A structural schematic diagram of a processing device of an augmented reality picture provided by an embodiment of the present disclosure;

[0030] Figure 12 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are provided so as to deliver a more thorough and complete understanding of the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of the present disclosure.

[0032] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising, but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions will be given in the description below.

[0034] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0035] It should be noted that the adjectives "one", "more" mentioned in the present disclosure are illustrative rather than limiting, and those skilled in the art should understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0036] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0037] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0038] For example, when responding to the active request of the user, a prompt message is sent to the user to explicitly prompt the user that the operation requested to be performed will require the acquisition and use of the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware, such as electronic devices, application programs, servers or storage media, etc. that perform the operation of the technical solutions of the present disclosure according to the prompt message.

[0039] As an optional but non-limiting implementation manner, in response to receiving the active request of the user, the manner of sending the prompt information to the user may be, for example, a pop-up window manner, in which the prompt information may be presented in a text manner. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0040] It can be understood that the above notification and user authorization obtaining process is only illustrative and does not limit the implementation manners of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0041] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0042] Figure 1 A flowchart of a processing method of an augmented reality picture provided by the embodiment of the present disclosure, the embodiment of the present disclosure is applicable to a case of rendering an augmented reality picture, and the method can be executed by a processing device of the augmented reality picture. The device can be realized in the form of software and / or hardware, and can be realized by an electronic device, which can be a mobile terminal, a PC terminal, or a server, etc.

[0043] As shown in FIG. 1, the method of the embodiment can specifically include the following steps. Figure 1

[0044] S110, in response to a rendering trigger request for an augmented reality picture, displaying a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture.

[0045] The augmented reality picture generally refers to a picture in which a real environment and a virtual object exist in the same space, that is, a picture generated after applying a virtual object to a real environment. In the embodiment of the present disclosure, the virtual object can be a three-dimensional model of a linear object. The linear object can be understood as an object with linear characteristics. The shape of the linear object can include at least one of a straight line, a curve, and a broken line. The representation style of the shape of the linear object can be a solid line or a dashed line.

[0046] In the embodiment of the present disclosure, the linear object can be a linear special effect object of the augmented reality picture. For example, it can be an identification line for at least one preset marker object, or a trajectory line for identifying the motion trajectory of a preset marker, or a preset linear special effect object, etc. It should be noted that the specific form of the linear special effect object can be various, which can be a whip or a stick, etc.

[0047] ​For example, when a preset marker (e.g., a ball, a paper airplane, or a dart) moving in a real environment is identified, a linear object can be determined based on a running track of the preset marker; when two preset markers appear in an augmented reality picture, a line connecting the two preset markers can be taken as a linear object; or, a rendering trigger request can be parsed to obtain object shape information (e.g., a straight line shape) describing a linear object corresponding to the augmented reality picture, and then a linear object corresponding to the object shape information described in the rendering trigger request can be determined according to a correspondence between preset linear objects and the object shape information.

[0048] In the embodiments of the present disclosure, the three-dimensional model of the linear object can be understood as a rendering model for characterizing three-dimensional features of the linear object, for example, a straight line three-dimensional model, a curve three-dimensional model, or a polyline three-dimensional model, etc. The type of the three-dimensional model of the linear object can be a mesh type or a point cloud type. That is, the three-dimensional model of the linear object can be a three-dimensional mesh model of the linear object, or can be a three-dimensional point cloud model of the linear object. The rendering trigger request can be understood as a trigger request for displaying the three-dimensional model of the linear object corresponding to the augmented reality picture in the augmented reality picture.

[0049] Considering the specific display form of the linear object and the underlying rendering logic, the number of the three-dimensional models of the linear object corresponding to the augmented reality picture can be one, two, or more than two.

[0050] In one embodiment, after receiving a rendering trigger request for an augmented reality picture, the three-dimensional model of the linear object corresponding to the augmented reality picture can be obtained from a database for storing three-dimensional models based on the rendering trigger request, that is, the three-dimensional model of the linear object corresponding to the augmented reality picture is read from the database for storing three-dimensional models and loaded into the memory. After the loading is completed, the three-dimensional model of the linear object can be rendered. Thus, the three-dimensional model of the linear object is displayed in the augmented reality picture, and the picture quality of the augmented reality picture is improved. Optionally, the database for storing three-dimensional models can be a local database or a remote database.

[0051] In another embodiment, after receiving the rendering trigger request for the augmented reality picture, a three-dimensional model of the linear object corresponding to the augmented reality picture can be constructed based on the rendering trigger request. After the model is constructed, the three-dimensional model of the linear object can be rendered. Thus, the three-dimensional model of the linear object is displayed in the augmented reality picture. Optionally, constructing the three-dimensional model of the linear object corresponding to the augmented reality picture based on the rendering trigger request includes: analyzing the rendering trigger request to obtain model feature data of the three-dimensional model of the linear object displayed in the augmented reality picture. Then, the three-dimensional model of the linear object corresponding to the augmented reality picture can be constructed based on the model feature data. This processing has the advantage of dynamically drawing the three-dimensional model of the linear object displayed in the augmented reality picture according to individual needs.

[0052] In the embodiments of the present disclosure, the way to obtain the rendering trigger request can be specifically as follows: receiving a trigger operation for triggering the rendering of the augmented reality picture, and generating the rendering trigger request for the augmented reality picture based on the trigger operation. It should be noted that the trigger operation for triggering the rendering of the augmented reality picture can have various trigger modes. For example, the trigger operation for triggering the rendering of the augmented reality picture can be a trigger operation generated by acting on a trigger control for triggering the rendering of the augmented reality picture; or a trigger operation generated based on collected voice instructions for rendering the augmented reality picture; or a trigger operation generated based on collected image information for rendering the augmented reality picture.

[0053] It should be noted that the icon style, display effect, and display position of the virtual trigger control can be set according to actual needs, which are not limited herein. Further, receiving the trigger operation for triggering the rendering of the augmented reality picture can be receiving a trigger operation (such as clicking or pressing a button) acting on a trigger control for triggering the rendering of the augmented reality picture.

[0054] In the embodiments of the present disclosure, the way to render the three-dimensional model of the linear object has various modes, which are not limited herein.

[0055] As an optional implementation of the embodiments of the present disclosure, rendering the three-dimensional model of the linear object can include: analyzing the rendering trigger request to obtain rendering parameters of the three-dimensional model of the linear object. Then, the three-dimensional model of the linear object can be rendered based on the rendering parameters. The rendering parameters can include material, light, and map, etc.

[0056] As another optional implementation of the embodiments of the present disclosure, the rendering processing of the three-dimensional model of the linear object can include: after receiving a rendering trigger request for the augmented reality picture, reading the rendering parameter corresponding to the three-dimensional model of the linear object in the pre-configured rendering parameter information based on the rendering trigger request. The three-dimensional model of the linear object is rendered based on the rendering parameter.

[0057] S120, in response to the display adjustment operation of the linear object, displaying the adjusted three-dimensional model of the linear object in the augmented reality picture.

[0058] The display adjustment operation can be understood as an operation for adjusting the linear object displayed in the augmented reality picture. The operation mode of the display adjustment operation can be various. For example, the display adjustment operation can be a touch operation, such as a single-click operation, a sliding operation or a double-click operation, acting on the linear object displayed in the augmented reality picture; or a click operation acting on a control for adjusting the linear object displayed in the augmented reality picture; or a pressing operation acting on a physical button for adjusting the linear object displayed in the augmented reality picture. The display adjustment operation can include a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation. In the embodiments of the present disclosure, based on the display adjustment operation, the display of the linear object can be adjusted to meet the individual needs of the user, and the three-dimensional model of the linear object can be displayed in all directions and at multiple angles, enriching the augmented reality picture.

[0059] In the embodiments of the present disclosure, the display position adjustment operation can be understood as an operation for adjusting the position of the linear object displayed in the augmented reality picture. In other words, the linear object displayed in the current display position of the augmented reality picture is adjusted from the current display position to a target display position in the augmented reality picture. The target display position can be understood as the display position of the linear object displayed in the augmented reality picture, which is obtained by moving the linear object from the current position of the linear object as the reference position in a certain direction. The certain direction can include moving left, moving right, moving up and moving down, etc. For example, the linear object displayed in the lower left corner of the augmented reality picture can be adjusted from the lower left corner of the augmented reality picture to the upper left corner of the augmented reality picture.

[0060] The display size adjustment operation can be understood as an operation for adjusting the size of the linear object displayed in the augmented reality picture, that is, adjusting the linear object displayed in the augmented reality picture from the current display size to the target display size. The display target size can be understood as the size of the linear object displayed in the augmented reality picture after being enlarged or reduced. The display angle adjustment operation can be understood as an operation for adjusting the angle of the linear object displayed in the augmented reality picture. In other words, the linear object displayed in the augmented reality picture at the current display angle is adjusted from the current display angle to the target display angle, so that the linear object is displayed in the augmented reality picture at the target display angle. The target display angle can be understood as the angle of the linear object displayed in the augmented reality picture after the display angle adjustment.

[0061] In one embodiment, after receiving the display position adjustment operation of the linear object, the display position operation information of the display position adjustment operation can be obtained. Then, based on the display position operation information, the target display position of the three-dimensional model of the linear object in the augmented reality picture can be determined. Then, based on the display position, the three-dimensional model of the linear object can be rendered. Thus, the three-dimensional model of the linear object is displayed at the target display position in the augmented reality picture.

[0062] In another embodiment, after receiving the display angle adjustment operation of the linear object, the display angle operation information of the display angle adjustment operation can be obtained. Then, based on the display angle operation information, the rotation angle and rotation axis of the display angle operation of the three-dimensional model of the linear object can be determined. Thus, the target display angle of the three-dimensional model of the linear object can be determined according to the rotation axis and rotation angle. Then, the three-dimensional model of the linear object can be displayed in the augmented reality picture at the target display angle.

[0063] In yet another embodiment, after receiving the display size adjustment operation of the linear object, the display size operation information of the display size adjustment operation can be obtained. Then, based on the display size operation information, the three-dimensional model of the linear object can be reconstructed. Thus, the reconstructed three-dimensional model of the linear object can be obtained. Further, the reconstructed three-dimensional model of the linear object can be rendered into the augmented reality picture. Thus, the reconstructed three-dimensional model of the linear object is displayed in the augmented reality picture.

[0064] It should be noted that in the embodiments of the present disclosure, there are various ways to perform model reconstruction on the three-dimensional model of the linear object. As an optional implementation of the embodiments of the present disclosure, it can include: determining a target display size for displaying the three-dimensional model of the linear object in the augmented reality picture based on the display size operation information; and performing model reconstruction on the three-dimensional model of the linear object based on the target display size.

[0065] As another optional implementation in the embodiments of the present disclosure, it can include: determining a current display size of the three-dimensional model of the linear object; obtaining a size ratio with respect to the current display size based on the display size operation information. Further, the target display size for displaying the three-dimensional model of the linear object in the augmented reality picture can be obtained based on the size ratio and the current display size. Thus, the three-dimensional model of the linear object with the target display size is displayed in the augmented reality picture.

[0066] The technical solutions of the embodiments of the present disclosure can improve the stereoscopic and realistic sense of the linear object in the augmented reality picture by displaying the three-dimensional model of the linear object corresponding to the augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture. In response to a display adjustment operation for the linear object, the three-dimensional model of the linear object after adjustment is displayed in the augmented reality picture, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation. The technical solutions of the embodiments of the present disclosure can not only more realistically integrate the linear object into the augmented reality picture by displaying the three-dimensional model of the linear object in the augmented reality picture, but also can effectively interact with the linear object according to the individualized needs of the user, thereby improving the user experience.

[0067] Figure 2 A flowchart of a processing method of an augmented reality picture is provided in the embodiments of the present disclosure. The technical solutions of the present embodiment further refine how to display the three-dimensional model of the linear object in the augmented reality picture on the basis of the above-mentioned embodiments. Optionally, the displaying the three-dimensional model of the linear object corresponding to the augmented reality picture in the augmented reality picture includes: obtaining a plurality of key points of the linear object corresponding to the augmented reality picture; and rendering the three-dimensional model of the linear object based on the plurality of key points, and displaying the three-dimensional model in the augmented reality picture. The specific implementation can be referred to the description of the present embodiment. The technical features same as or similar to the foregoing embodiments are not described herein.

[0068] As Figure 2 shown, the method of the present embodiment can specifically include:

[0069] S210, in response to a rendering trigger request for an augmented reality picture, obtaining a plurality of key points of a linear object corresponding to the augmented reality picture.

[0070] The key points of the linear object can be understood as feature points of the linear object. The plurality of key points of the linear object can depict the shape of the linear object, such as a straight line, a curve, or a polyline, etc. In order to more accurately reflect the position of each key point of the linear object, a coordinate system can be constructed in advance, so that the position information of each key point of the linear object can be represented using the coordinate points in the coordinate system.

[0071] Specifically, after receiving the rendering trigger request for the augmented reality picture, the linear object corresponding to the augmented reality picture can be determined based on the rendering trigger request. Then, the plurality of key points of the linear object can be obtained.

[0072] In the embodiments of the present disclosure, there are various ways to obtain the plurality of key points of the linear object corresponding to the augmented reality picture.

[0073] As an optional implementation of the embodiments of the present disclosure, obtaining the plurality of key points of the linear object corresponding to the augmented reality picture can include: generating the plurality of key points of the linear object corresponding to the augmented reality picture based on a preset algorithm. The preset algorithm can be an algorithm for generating key points that is set in advance.

[0074] Optionally, generating the plurality of key points of the linear object corresponding to the augmented reality picture based on the preset algorithm can include: randomly generating the plurality of key points of the linear object corresponding to the augmented reality picture based on the preset algorithm; or, after receiving the rendering trigger request for the augmented reality picture, the rendering trigger request can be parsed. Thus, the preset drawing frame rate of the key points of the linear object corresponding to the augmented reality picture can be obtained. Then, the plurality of key points of the linear object corresponding to the augmented reality picture can be generated based on the preset drawing frame rate and the preset algorithm.

[0075] As another optional implementation of the embodiments of the present disclosure, obtaining the plurality of key points of the linear object corresponding to the augmented reality picture can include: determining an associated object of the linear object to be rendered in the augmented reality picture, and determining the plurality of key points of the linear object corresponding to the augmented reality picture based on the motion trajectory of the associated object. The linear object to be rendered can be understood as a linear object that needs to be rendered in the augmented reality picture. The associated object can be understood as an object in the augmented reality picture that has an association relationship with the linear object to be rendered. For example, the associated object having an association relationship with the linear object to be rendered can be a paper airplane displayed in the augmented reality picture. The linear object to be rendered can be the flight trajectory of the paper airplane displayed in the augmented reality picture.

[0076] Specifically, an associated object associated with the linear object to be rendered in the augmented reality picture is determined. After the associated object is determined, a motion trajectory of the associated object can be obtained. After the motion trajectory is obtained, a feature extraction process can be performed on the motion trajectory. Further, a plurality of feature points of the motion trajectory can be obtained, and the plurality of extracted feature points can be used as a plurality of key points of the linear object corresponding to the augmented reality picture.

[0077] For example, the associated object of the linear object to be rendered can be an airplane flying in the augmented reality picture. The linear object to be rendered can be a motion trajectory of a certain position on the airplane flying in the augmented reality picture. The plurality of key points of the linear object to be rendered can be understood as feature points of the motion trajectory of a certain position on the airplane flying in the augmented reality picture.

[0078] As an optional implementation of the embodiment of the present disclosure, obtaining the plurality of key points of the linear object corresponding to the augmented reality picture includes: determining a shape of the linear object corresponding to the augmented reality picture. Further, the plurality of key points of the linear object can be obtained according to the shape of the linear object. Further, obtaining the plurality of key points of the linear object according to the shape of the linear object can include: based on the shape of the linear object, obtaining the key points corresponding to the shape in a database for storing key points of the linear object; or, based on the shape of the linear object, generating the key points of the linear object.

[0079] S220, rendering a three-dimensional model of the linear object based on the plurality of key points, and displaying the three-dimensional model in the augmented reality picture.

[0080] Specifically, after the plurality of key points of the linear object are obtained, a three-dimensional model of the linear object corresponding to the plurality of key points can be obtained. Further, the three-dimensional model can be rendered into the augmented reality picture, so that the three-dimensional model is displayed in the augmented reality picture.

[0081] In the embodiment of the present disclosure, there are various ways to obtain the three-dimensional model of the linear object corresponding to the plurality of key points.

[0082] As an optional implementation of the embodiment of the present disclosure, a fitting process can be performed on the plurality of key points. Thus, a fitting result is obtained. Further, a model reconstruction can be performed based on the fitting result. Thus, the three-dimensional model of the linear object corresponding to the plurality of key points can be obtained, and the advantage of this is that the three-dimensional model of the linear object can be drawn according to individual needs.

[0083] As another optional implementation of the embodiments of the present disclosure, a three-dimensional model matched with the plurality of key points can be matched from a database for storing three-dimensional models, and the matched three-dimensional model is taken as the three-dimensional model of the linear object corresponding to the plurality of key points. This has the advantage that the three-dimensional model of the linear object can be obtained more quickly and effectively, and the response speed of the rendering triggering request of the augmented reality picture is further improved.

[0084] S230, in response to the display adjustment operation for the linear object, display the adjusted three-dimensional model of the linear object in the augmented reality picture.

[0085] The technical solutions of the embodiments of the present disclosure can obtain a plurality of key points of a linear object corresponding to the augmented reality picture, render a three-dimensional model of the linear object based on the plurality of key points, and display the three-dimensional model in the augmented reality picture. This can realize targeted acquisition of the three-dimensional model of the linear object, thereby enriching the augmented reality picture.

[0086] Figure 3 A flowchart of an augmented reality picture processing method provided by the embodiments of the present disclosure is shown. The technical solutions of the present embodiment further refine how to render a three-dimensional model of a linear object based on a plurality of key points on the basis of the above-mentioned embodiments. Optionally, the rendering of the three-dimensional model of the linear object based on the plurality of key points comprises: for each key point, making a circle with the key point as the center, determining a plurality of vertices of the three-dimensional model of the linear object based on points located on the circle, and rendering the three-dimensional model of the linear object based on the plurality of vertices. The specific embodiments can be seen from the description of the present embodiment. The technical features same as or similar to the foregoing embodiments are not described herein again.

[0087] As Figure 3 shown, the method of the present embodiment can specifically include:

[0088] S310, in response to a rendering triggering request for an augmented reality picture, obtaining a plurality of key points of a linear object corresponding to the augmented reality picture.

[0089] S320, for each key point, making a circle with the key point as the center, determining a plurality of vertices of the three-dimensional model of the linear object based on points located on the circle, rendering the three-dimensional model of the linear object based on the plurality of vertices, and displaying the three-dimensional model in the augmented reality picture.

[0090] Optionally, the circle with the key point as the center can include: obtaining a preset circle radius corresponding to the key point; taking the key point as the center; and performing the circle based on the preset circle radius corresponding to the key point and the center. The preset circle radius can be understood as a radius of the circle preset for each key point. It should be noted that the preset circle radius corresponding to each key point can be the same or different. Optionally, the obtaining of the preset circle radius corresponding to the key point can include: analyzing the rendering trigger request to obtain the preset circle radius corresponding to each key point included in the rendering trigger request; or obtaining circle radius configuration information configured for the plurality of key points, the circle radius configuration information being configured with the preset circle radius of each key point; and matching the circle radius configuration information with the preset circle radius corresponding to the key point.

[0091] In the embodiments of the present disclosure, there are various ways to determine the plurality of vertices of the three-dimensional model of the linear object based on the points located on the circle.

[0092] As an optional implementation in the embodiments of the present disclosure, a diameter of the circle is taken with the center of the circle, the diameter is rotated by a preset rotation angle (such as 5 degrees, 10 degrees, or 15 degrees, etc.), and in the rotation process, each intersection point of the diameter and the circle is determined, and each intersection point is taken as a plurality of vertices of the three-dimensional model of the linear object.

[0093] As another optional implementation in the embodiments of the present disclosure, a point is selected on the circle, the selected point is taken as a fixed point, and a plurality of straight lines are taken with the fixed point. Each intersection point of each straight line and the circle is taken as a plurality of vertices of the three-dimensional model of the linear object.

[0094] In the embodiments of the present disclosure, the three-dimensional model of the linear object is rendered based on the plurality of vertices, and the three-dimensional model is displayed in the augmented reality picture, which can include: performing curve fitting processing on the plurality of vertices, and then fitting each vertex into a curve, so that a plurality of curves can be obtained. After obtaining the plurality of curves, each curve can be constructed into a surface based on a preset surface construction manner, so that a plurality of surfaces can be obtained. After obtaining the plurality of surfaces, the three-dimensional model of the linear object can be constructed based on the plurality of surfaces. After the three-dimensional model is constructed, the three-dimensional model can be rendered based on rendering information (such as texture, material, map, light, and model bone action, etc.) of the three-dimensional model. After rendering is completed, the rendered three-dimensional model is displayed in the augmented reality picture.

[0095] S330, in response to the display adjustment operation for the linear object, displaying the three-dimensional model of the adjusted linear object in the augmented reality picture.

[0096] The technical solution of the embodiments of the present disclosure is that, for each key point, a circle is drawn with the key point as the center, a plurality of vertices of the three-dimensional model of the linear object are determined based on points located on the circle, and the three-dimensional model of the linear object is rendered based on the plurality of vertices, so that the dynamic construction of the three-dimensional model of the linear object can be realized.

[0097] Figure 4 A flowchart of a processing method of an augmented reality picture provided by the embodiments of the present disclosure is shown. The technical solution of the present embodiment further refines how to render the three-dimensional model of the linear object based on a plurality of vertices on the basis of the above-mentioned embodiments. Optionally, the rendering of the three-dimensional model of the linear object based on the plurality of vertices comprises: taking the circle as a cross section of the three-dimensional model, for each cross section, determining a rotation matrix corresponding to the cross section based on an adjacent cross section of the cross section, determining the spatial coordinates of the vertex corresponding to the cross section based on the rotation matrix, and rendering the three-dimensional model of the linear object based on the spatial coordinates of the plurality of vertices. The specific implementation can be seen from the description of the present embodiment. The technical features same as or similar to the foregoing embodiments are not described herein again.

[0098] As shown in Figure 4 , the method of the present embodiment can specifically comprise:

[0099] S410, in response to a rendering trigger request for an augmented reality picture, a plurality of key points of a linear object corresponding to the augmented reality picture are acquired.

[0100] S420, for each key point, a circle is drawn with the key point as the center, and a plurality of vertices of the three-dimensional model of the linear object are determined based on points located on the circle.

[0101] S430, taking the circle as a cross section of the three-dimensional model, for each cross section, a rotation matrix corresponding to the cross section is determined based on an adjacent cross section of the cross section.

[0102] In the embodiments of the present disclosure, the determination of the rotation matrix corresponding to the cross section based on the adjacent cross section of the cross section comprises: taking a vector between the center of the cross section and the center of the adjacent cross section of the cross section as a reference vector, and calculating the rotation matrix corresponding to the cross section according to a horizontal direction vector and the reference vector.

[0103] The reference vector can be understood as a directed line segment from the center of the cross section to the center of the cross section adjacent to the cross section. The directed line segment from the center of the cross section to the center of the cross section adjacent to the cross section can be a directed line segment from the center of the cross section to the center of the cross section adjacent to the cross section, or a directed line segment from the center of the cross section adjacent to the cross section to the center of the cross section. The horizontal direction vector can be understood as any vector parallel to the X axis in a three-dimensional coordinate system. In other words, the horizontal direction vector can be understood as any vector perpendicular to the YZ plane in a three-dimensional coordinate system. The YZ plane is a plane composed of the Y axis and the Z axis.

[0104] In the embodiments of the present disclosure, the calculation of the rotation matrix corresponding to the cross section according to the horizontal direction vector and the reference vector can include: passing the horizontal direction vector and the reference vector as actual parameters to the entry parameters of a pre-defined rotation matrix method for calculating the rotation matrix corresponding to the cross section. After the parameter passing is completed, the rotation matrix method is executed. Then the rotation matrix corresponding to the cross section can be calculated.

[0105] Specifically, the horizontal direction vector and the reference vector are substituted into the dot product formula to calculate the rotation angle between the horizontal direction vector and the reference vector. Then the rotation axis of the cross section and the cross section adjacent to the cross section can be determined based on the rotation angle. Further, the rotation matrix corresponding to the cross section can be calculated based on the rotation angle, the rotation axis and the Rodrigues rotation formula.

[0106] S440, determining the spatial coordinates of the vertices corresponding to the cross section based on the rotation matrix, rendering a three-dimensional model of the linear object based on the spatial coordinates of the plurality of vertices, and displaying the three-dimensional model in the augmented reality picture.

[0107] In the embodiments of the present disclosure, the determination of the spatial coordinates of the vertices corresponding to the cross section based on the rotation matrix can include: determining the spatial coordinates of the vertices corresponding to the cross section adjacent to the cross section; and then determining the spatial coordinates of the vertices corresponding to the cross section based on the spatial coordinates of the vertices corresponding to the cross section adjacent to the cross section and the rotation matrix. This processing has the advantage that the rotation orientation of the cross section can be corrected by the rotation matrix of the cross section.

[0108] Specifically, based on the spatial coordinates of the vertex corresponding to the cross section adjacent to the cross section and the rotation matrix, the spatial coordinates of the vertex corresponding to the cross section are determined, including: according to the spatial coordinates of the vertex corresponding to the cross section adjacent to the cross section, a coordinate matrix of the vertex corresponding to the cross section adjacent to the cross section can be constructed. Then the coordinate matrix of the vertex can be multiplied by the rotation matrix to perform matrix calculation. Thus, the spatial coordinates of the vertex corresponding to the cross section can be determined based on the matrix calculation result.

[0109] S450, in response to the display adjustment operation for the linear object, display the adjusted three-dimensional model of the linear object in the augmented reality picture.

[0110] The technical scheme of the embodiment of the present disclosure can achieve more efficient and accurate construction of the three-dimensional model of the linear object by taking the circle as the cross section of the three-dimensional model, determining the rotation matrix corresponding to each cross section based on the cross section adjacent to the cross section, determining the spatial coordinates of the vertex corresponding to the cross section based on the rotation matrix, and rendering the three-dimensional model of the linear object based on the spatial coordinates of multiple vertices.

[0111] Figure 5 A flowchart of a processing method of an augmented reality picture is provided in the embodiment of the present disclosure. The technical scheme of the present embodiment further refines how to render the three-dimensional model of the linear object based on the spatial coordinates of multiple vertices on the basis of the above-mentioned embodiment. Optionally, the rendering of the three-dimensional model of the linear object based on the spatial coordinates of multiple vertices includes: determining a to-be-rendered face of the three-dimensional model based on a preset rendering mode of the three-dimensional model and multiple vertices of the three-dimensional model, wherein the to-be-rendered face includes a to-be-rendered cross section and a connecting face between two adjacent cross sections; for each to-be-rendered cross section, constructing a triangular primitive based on every three vertices located on the cross section; for each to-be-rendered connecting face, constructing a triangular primitive based on every three vertices located on different cross sections; and rendering the three-dimensional model of the linear object based on the spatial coordinates of multiple vertices and the triangular primitives of the to-be-rendered face. The specific implementation can be seen from the description of the present embodiment. The technical features same as or similar to those of the foregoing embodiments are not described herein again.

[0112] As Figure 5 shown, the method of the present embodiment can specifically include:

[0113] S510, in response to a rendering trigger request for an augmented reality picture, acquiring multiple key points of a linear object corresponding to the augmented reality picture.

[0114] S520, for each of the key points, making a circle with the key point as the center, and determining a plurality of vertices of the three-dimensional model of the linear object based on points located on the circle.

[0115] S530, taking the circle as a cross section of the three-dimensional model, and for each cross section, determining a rotation matrix corresponding to the cross section based on a cross section adjacent to the cross section.

[0116] S540, determining spatial coordinates of a vertex corresponding to the cross section based on the rotation matrix.

[0117] S550, determining a to-be-rendered face of the three-dimensional model based on a preset rendering manner of the three-dimensional model and the plurality of vertices of the three-dimensional model, wherein the to-be-rendered face includes a to-be-rendered cross section and a connecting face between two adjacent cross sections.

[0118] The preset rendering manner can be understood as a rendering manner preset for the plurality of vertices of the three-dimensional model, and can include continuous rendering and / or discontinuous rendering. The continuous rendering can be used to render a linear object with a solid line shape representation style. The discontinuous rendering can be used to render a linear object with a dashed line shape representation style. The to-be-rendered face can be understood as a face in the three-dimensional model that needs to be rendered. The to-be-rendered face can include at least two to-be-rendered cross sections and a connecting face between two adjacent cross sections. For example, the three-dimensional model of the linear object can be a cylindrical model, the to-be-rendered cross sections can be two bottom faces of the cylindrical model, and the connecting face between the two adjacent cross sections can be a side face between the two bottom faces of the cylindrical model.

[0119] In one embodiment, the determining of the to-be-rendered face of the three-dimensional model based on the preset rendering manner of the three-dimensional model and the plurality of vertices of the three-dimensional model can include: in a case where the preset rendering manner of the three-dimensional model is continuous rendering, taking a starting cross section and an ending cross section of the three-dimensional model as to-be-rendered cross sections, and taking a connecting face between all two-by-two adjacent cross sections as a to-be-rendered connecting face. The starting cross section can be understood as a first cross section constructed in the process of constructing the three-dimensional model. The ending cross section can be understood as a last cross section constructed in the process of constructing the three-dimensional model.

[0120] In another embodiment, the preset rendering mode of the three-dimensional model and the plurality of vertices of the three-dimensional model are used to determine the surfaces to be rendered of the three-dimensional model, including: in the case that the preset rendering mode of the three-dimensional model is intermittent rendering, the starting cross section, the ending cross section and at least two cross sections other than the starting cross section and the ending cross section of the three-dimensional model are all determined as cross sections to be rendered; and the connecting surfaces to be rendered are determined based on the cross sections to be rendered, so that the connecting surfaces are displayed intermittently.

[0121] In the embodiments of the present disclosure, whether a cross section is to be rendered can be determined according to the arrangement serial number of the cross section. There are various ways to determine whether a cross section is to be rendered according to the arrangement serial number of the cross section. For example, the cross sections with odd arrangement serial numbers can be determined as cross sections to be rendered; or the cross sections with even arrangement serial numbers can be determined as cross sections to be rendered. It should be noted that the intermittent rendering can be regular intermittent rendering (see FIG. 2A) or irregular intermittent rendering (see FIG. 2B). Figure 6A Figure 6B

[0122] S560, for each cross section to be rendered, a triangular primitive is constructed based on each three vertices located on the cross section.

[0123] The triangular primitive can be understood as a triangular patch. Specifically, for each cross section to be rendered, all vertices of the cross section to be rendered can be determined. A preset model patch construction algorithm (such as a region generation algorithm) can be used to establish a line connection between each three vertices of all vertices located on the cross section, so that a triangular primitive can be constructed.

[0124] S570, for each connecting surface to be rendered, a triangular primitive is constructed based on each three vertices located on different cross sections.

[0125] Specifically, for each connecting surface to be rendered, all vertices of the connecting surface to be rendered can be determined. A preset model patch construction algorithm can be used to establish a line connection between each three vertices of all vertices located on the connecting surface, so that a triangular primitive can be constructed.

[0126] S580, the three-dimensional model of the linear object is rendered based on the spatial coordinates of the plurality of vertices and the triangular primitives of the surfaces to be rendered, and the three-dimensional model is displayed in the augmented reality picture.

[0127] ​​In this embodiment of the disclosure, a three-dimensional model of the linear object can be generated based on the spatial coordinates of the multiple vertices and the triangular primitives of the surface to be rendered. After generating the three-dimensional model, it can be rendered based on the rendering parameters of the three-dimensional model. After rendering is complete, the rendered three-dimensional model can be displayed in the augmented reality screen.

[0128] To enhance the realism of the 3D model in augmented reality visuals, after generating the 3D model of the linear object, the triangular primitives in the 3D model can be refined based on model refinement parameters set for the 3D model. These refinement parameters can include patch shape, individual patch size, and tension between adjacent patches.

[0129] S590. In response to the display adjustment operation for the linear object, the adjusted three-dimensional model of the linear object is displayed in the augmented reality screen.

[0130] The technical solution of this disclosure determines the surface to be rendered of the 3D model based on a preset rendering method and multiple vertices of the 3D model. The surface to be rendered includes a cross-section to be rendered and a connecting surface between two adjacent cross-sections. For each cross-section to be rendered, triangular primitives are constructed based on every three vertices located on the cross-section. For each connecting surface to be rendered, triangular primitives are constructed based on every three vertices located on different cross-sections. The 3D model of the linear object is rendered based on the spatial coordinates of multiple vertices and the triangular primitives of the surface to be rendered. This achieves rendering of the 3D model of the linear object in multiple ways, thereby obtaining multiple display forms of the 3D model of the linear object and further enriching the content in the augmented reality image.

[0131] This disclosure provides an optional example of a method for processing augmented reality images. In this embodiment, the trajectory of a moving object is used as a linear object, which can be a parabola; that is, the linear object can be a parabola. Detailed implementation can be found in the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here.

[0132] like Figure 7 As shown, after receiving a rendering trigger request for the augmented reality image, multiple key points of the parabola corresponding to the augmented reality image can be obtained. Figure 7 (P0, P1, P2, ..., PN in the original text).

[0133] For example, acquiring multiple key points of the parabola corresponding to the augmented reality image may include: acquiring the initial velocity vector of the moving object (V), the initial coordinates (P(x0, y0)), and the gravitational acceleration (g). The frame rate for drawing the trajectory points of the moving object can be f. The x-axis and z-axis components of the initial velocity vector V can be combined into a horizontal component Vx, and the vertical direction can be treated as a separate vector Vy. The vertical height can be calculated every 1 / f of the horizontal distance based on the drawing frame rate. The virtual time increment t can be 1 / f.

[0134] The vertical height y of a moving object can be calculated using the following formula:

[0135] y = y0 + Vy*t + 1 / 2*g*t 2

[0136] The horizontal distance x of a moving object can be calculated using the following formula:

[0137] x = x0 + Vx*t

[0138] After the calculation is complete, the horizontal distance can be decomposed into x-axis distance and z-axis distance based on the initial degree V. This allows us to obtain multiple trajectory points of the moving object, i.e., multiple key points of the parabola.

[0139] For each key point, use the key point ( Figure 7 In this context, P(n) is the center of a circle, and multiple vertices of the 3D model of the linear object are determined based on the points located on the circle. Figure 7 In the context of v(n)(0), v(n)(1), ..., v(n)(m)); the circle is used as a cross-section of the 3D model, and for each cross-section, the vector between the center of the cross-section and the center of the adjacent cross-section is used as a reference vector. Figure 7 The vectors obtained from calculating p(n-1) and p(n) are used to calculate the rotation matrix corresponding to the cross-section based on the horizontal direction vector and the reference vector. Figure 7 M(n)).

[0140] The spatial coordinates of the vertices corresponding to the cross-sections are determined based on the rotation matrix. Based on the preset rendering method of the 3D model and multiple vertices of the 3D model, the surfaces to be rendered are determined; these surfaces can include the cross-sections to be rendered and the connecting surfaces between adjacent cross-sections. For each cross-section to be rendered, triangular primitives are constructed based on every three vertices located on the cross-section (see...). Figure 8 For each connected surface to be rendered, construct triangular primitives based on every three vertices located on different cross sections (see...). Figure 9 ).

[0141] Optionally, in the case that the preset rendering mode of the parabola three-dimensional model is intermittent rendering, the starting cross section, the ending cross section and at least two cross sections other than the starting cross section and the ending cross section of the parabola three-dimensional model are all taken as cross sections to be rendered; and a connecting surface to be rendered is determined based on the cross sections to be rendered, so that the connecting surface is displayed intermittently (see Figure 10 ).

[0142] The three-dimensional model of the linear object is displayed in the augmented reality picture based on the spatial coordinates of the plurality of vertices and the triangular primitives of the face to be rendered.

[0143] The technical scheme of the embodiments of the present disclosure displays the three-dimensional model of the linear object in the augmented reality picture, not only more realistically integrates the linear object into the augmented reality picture, but also enables effective interaction with the linear object according to the personalized needs of the user, thereby improving the user experience.

[0144] Figure 11 A structural schematic diagram of an augmented reality picture processing device provided by the embodiments of the present disclosure is shown in FIG. 6, which includes a request module 610 and a display module 620. Figure 11

[0145] The request module 610 is configured to display a three-dimensional model of a linear object corresponding to an augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture; and the display module 620 is configured to display the three-dimensional linear object after adjustment in the augmented reality picture in response to a display adjustment operation for the three-dimensional model, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation.

[0146] The technical scheme of the embodiments of the present disclosure displays the three-dimensional model of the linear object in the augmented reality picture in response to the rendering trigger request for the augmented reality picture, which can improve the stereoscopic and realistic sense of the linear object in the augmented reality picture. The three-dimensional model of the linear object after adjustment is displayed in the augmented reality picture in response to the display adjustment operation for the linear object, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation. The technical scheme of the embodiments of the present disclosure displays the three-dimensional model of the linear object in the augmented reality picture, not only more realistically integrates the linear object into the augmented reality picture, but also enables effective interaction with the linear object according to the personalized needs of the user, thereby improving the user experience.

[0147] ​On the basis of each of the optional technical solutions above, optionally, the request module 610 comprises a key point acquisition unit and a key point rendering unit; wherein,

[0148] The key point acquisition unit is configured to acquire a plurality of key points of a linear object corresponding to the augmented reality picture.

[0149] The key point rendering unit is configured to render a three-dimensional model of the linear object based on the plurality of key points, and display the three-dimensional model in the augmented reality picture.

[0150] On the basis of each of the optional technical solutions above, optionally, the key point rendering unit is specifically configured to, for each key point, make a circle with the key point as the center, determine a plurality of vertices of the three-dimensional model of the linear object based on points located on the circle, and render the three-dimensional model of the linear object based on the plurality of vertices.

[0151] On the basis of each of the optional technical solutions above, optionally, the key point rendering unit comprises a rotation matrix determination subunit and a vertex rendering subunit, wherein,

[0152] The rotation matrix determination subunit is configured to take the circle as a cross section of the three-dimensional model, for each cross section, determine a rotation matrix corresponding to the cross section based on a cross section adjacent to the cross section.

[0153] The vertex rendering subunit is configured to determine spatial coordinates of a vertex corresponding to the cross section based on the rotation matrix, and render the three-dimensional model of the linear object based on spatial coordinates of the plurality of vertices.

[0154] On the basis of each of the optional technical solutions above, optionally, the rotation matrix determination subunit is specifically configured to take a vector between a center of the cross section and a center of a cross section adjacent to the cross section as a reference vector, and calculate the rotation matrix corresponding to the cross section according to a horizontal direction vector and the reference vector.

[0155] On the basis of each of the optional technical solutions above, optionally, the vertex rendering subunit is specifically configured to determine a to-be-rendered face of the three-dimensional model based on a preset rendering mode of the three-dimensional model and the plurality of vertices of the three-dimensional model, wherein the to-be-rendered face comprises a to-be-rendered cross section and a connecting face between two adjacent cross sections.

[0156] For each to-be-rendered cross section, a triangle primitive is constructed based on every three vertices located on the cross section.

[0157] For each to-be-rendered connecting face, a triangle primitive is constructed based on every three vertices located on different cross sections.

[0158] render the three-dimensional model of the linear object based on the spatial coordinates of the plurality of vertices and the triangular primitives of the face to be rendered.

[0159] On the basis of each of the optional technical solutions described above, the vertex rendering subunit can be configured to, in the case where the preset rendering mode of the three-dimensional model is continuous rendering, take the starting cross section and the ending cross section of the three-dimensional model as the cross sections to be rendered, and take the connecting surfaces between all the cross sections adjacent to each other as the connecting surfaces to be rendered.

[0160] On the basis of each of the optional technical solutions described above, the vertex rendering subunit can be configured to, in the case where the preset rendering mode of the three-dimensional model is discontinuous rendering, take the starting cross section, the ending cross section and at least two cross sections other than the starting cross section and the ending cross section of the three-dimensional model as the cross sections to be rendered; and determine the connecting surfaces to be rendered based on the cross sections to be rendered, so that the connecting surfaces are displayed discontinuously.

[0161] On the basis of each of the optional technical solutions described above, the key point acquisition unit can be configured to: generate a plurality of key points of a linear object corresponding to the augmented reality picture based on a preset algorithm; or determine an associated object of the linear object to be rendered in the augmented reality picture, and determine a plurality of key points of a linear object corresponding to the augmented reality picture based on a motion trajectory of the associated object.

[0162] The processing apparatus for an augmented reality picture provided in the embodiments of the present disclosure can perform the processing method for an augmented reality picture provided in any of the embodiments of the present disclosure, and has the function modules and beneficial effects corresponding to the execution method.

[0163] It is worth noting that each unit and module included in the above apparatus is only divided according to the function logic, but is not limited to the above division, as long as the corresponding function can be implemented; in addition, the specific name of each functional unit is only for convenient mutual distinction, and does not limit the protection scope of the embodiments of the present disclosure.

[0164] Figure 12 A structural schematic diagram of an electronic device provided in the embodiments of the present disclosure is shown in FIG. 7. Hereinafter, the structural schematic diagram of the electronic device (for example, a terminal device or a server) 700 suitable for implementing the embodiments of the present disclosure is shown in FIG. 7. Figure 12 The terminal device in the embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 12 The terminal device in the embodiments of the present disclosure can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like.Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0165] like Figure 12 As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An edit / output (I / O) interface 705 is also connected to the bus 704.

[0166] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 12 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0167] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.

[0168] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0169] The electronic device provided by the embodiments of the present disclosure and the processing method of the augmented reality picture provided by the above embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0170] The present disclosure provides a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the processing method of the augmented reality picture provided by the above embodiments.

[0171] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0172] In some embodiments, the client, server, or other computing machines utilized by the system can communicate over any known or later developed network protocol, such as the Hypertext Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any then-current or later developed networks.

[0173] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0174] The computer-readable medium described above has one or more programs carried thereon, which, when executed by the electronic device, cause the electronic device to: in response to a rendering trigger request for an augmented reality picture, display a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture; and in response to a display adjustment operation for the linear object, display an adjusted three-dimensional model of the linear object in the augmented reality picture, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

[0175] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages, including but not limited to object oriented programming languages such as Java, Smalltalk, C++, or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0176] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself. For example, the first obtaining unit can also be described as a unit for obtaining at least two Internet protocol addresses.

[0178] The functions described above in the specification can be implemented in part or in whole by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0179] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] According to one or more embodiments of the present disclosure, Example One provides a processing method of an augmented reality picture, comprising:

[0181] displaying a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture;

[0182] displaying an adjusted three-dimensional model of the linear object in the augmented reality picture in response to a display adjustment operation for the linear object, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

[0183] According to one or more embodiments of the present disclosure, Example Two provides a processing method of an augmented reality picture, comprising:

[0184] Optionally, the displaying a three-dimensional model of a linear object corresponding to the augmented reality picture in the augmented reality picture comprises:

[0185] obtaining a plurality of key points of the linear object corresponding to the augmented reality picture;

[0186] rendering a three-dimensional model of the linear object based on the plurality of key points, and displaying the three-dimensional model in the augmented reality picture.

[0187] According to one or more embodiments of the present disclosure, Example Three provides a processing method of an augmented reality picture, comprising:

[0188] Optionally, the rendering a three-dimensional model of the linear object based on the plurality of key points comprises:

[0189] for each key point, making a circle with the key point as the center, determining a plurality of vertices of the three-dimensional model of the linear object based on points located on the circle, and rendering the three-dimensional model of the linear object based on the plurality of vertices.

[0190] According to one or more embodiments of the present disclosure, Example Four provides a processing method of an augmented reality picture, comprising:

[0191] Optionally, the rendering a three-dimensional model of the linear object based on the plurality of vertices comprises:

[0192] taking the circle as a cross section of the three-dimensional model, for each cross section, determining a rotation matrix corresponding to the cross section based on a cross section adjacent to the cross section;

[0193] determining a spatial coordinate of a vertex corresponding to the cross section based on the rotation matrix, and rendering the three-dimensional model of the linear object based on spatial coordinates of the plurality of vertices.

[0194] According to one or more embodiments of the present disclosure, Example Five provides a processing method of an augmented reality picture, comprising:

[0195] Optionally, the determining the rotation matrix corresponding to the cross section based on the cross section adjacent to the cross section comprises:

[0196] Taking the vector between the center of the cross section and the center of the cross section adjacent to the cross section as a reference vector, the rotation matrix corresponding to the cross section is calculated according to the horizontal direction vector and the reference vector.

[0197] According to one or more embodiments of the present disclosure, Example Six provides a processing method of an augmented reality picture, comprising:

[0198] Optionally, the rendering the three-dimensional model of the linear object based on the spatial coordinates of the plurality of vertices comprises:

[0199] Based on the preset rendering mode of the three-dimensional model and the plurality of vertices of the three-dimensional model, a to-be-rendered surface of the three-dimensional model is determined, wherein the to-be-rendered surface comprises a to-be-rendered cross section and a connecting surface between two adjacent cross sections;

[0200] For each to-be-rendered cross section, a triangular primitive is constructed based on every three vertices located on the cross section;

[0201] For each to-be-rendered connecting surface, a triangular primitive is constructed based on every three vertices located on different cross sections;

[0202] The three-dimensional model of the linear object is rendered based on the spatial coordinates of the plurality of vertices and the triangular primitives of the to-be-rendered surface.

[0203] According to one or more embodiments of the present disclosure, Example Seven provides a processing method of an augmented reality picture, comprising:

[0204] Optionally, the determining the to-be-rendered surface of the three-dimensional model based on the preset rendering mode of the three-dimensional model and the plurality of vertices of the three-dimensional model comprises:

[0205] In a case where the preset rendering mode of the three-dimensional model is continuous rendering, a starting cross section and an ending cross section of the three-dimensional model are taken as to-be-rendered cross sections, and all connecting surfaces between two adjacent cross sections are taken as to-be-rendered connecting surfaces.

[0206] According to one or more embodiments of the present disclosure, Example Eight provides a processing method of an augmented reality picture, comprising:

[0207] Optionally, the determining the to-be-rendered surface of the three-dimensional model based on the preset rendering mode of the three-dimensional model and the plurality of vertices of the three-dimensional model comprises:

[0208] In a case where the preset rendering mode of the three-dimensional model is intermittent rendering, the starting cross section, the ending cross section and at least two cross sections other than the starting cross section and the ending cross section of the three-dimensional model are all taken as to-be-rendered cross sections; and a to-be-rendered connecting surface is determined based on the to-be-rendered cross sections, so that the connecting surface is displayed intermittently.

[0209] According to one or more embodiments of the present disclosure, Example Nine provides a processing method of an augmented reality picture, comprising:

[0210] Optionally, the obtaining the plurality of key points of the linear object corresponding to the augmented reality picture comprises:

[0211] generating the plurality of key points of the linear object corresponding to the augmented reality picture based on a preset algorithm;

[0212] or,

[0213] determining the associated object of the linear object to be rendered in the augmented reality picture, and determining the plurality of key points of the linear object corresponding to the augmented reality picture based on a motion trajectory of the associated object.

[0214] According to one or more embodiments of the present disclosure, Example Ten provides a processing device of an augmented reality picture, comprising:

[0215] a requesting module configured to display a three-dimensional model of a linear object corresponding to an augmented reality picture in the augmented reality picture in response to a rendering trigger request for the augmented reality picture;

[0216] a display module configured to display the adjusted three-dimensional linear object in the augmented reality picture in response to a display adjustment operation for the three-dimensional model, wherein the display adjustment operation comprises a display position adjustment operation, a display size adjustment operation and / or a display angle adjustment operation.

[0217] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the disclosed range of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above technical features are replaced with technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0218] Moreover, while operations are depicted in a particular order, this should not be understood as requiring such an order nor infringing on the scope of the disclosure. Certain of the operations described in the discussion are combinable into a single operation, and certain operations can be separated into several operations. In some embodiments, the operations described in the discussion can be performed in an order different than presented in the discussion. In some embodiments, the operations described in the discussion can be performed concurrently. Also, while several specific implementation details are discussed in the discussion, these should not be interpreted as limiting the scope of the disclosure. Rather, certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0219] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for processing augmented reality images, characterized in that, include: In response to a rendering trigger request for an augmented reality image, multiple key points of a linear object corresponding to the augmented reality image are obtained; For each key point, a circle is drawn with the key point as the center, and multiple vertices of the three-dimensional model of the linear object are determined based on the points located on the circle; Using the circle as a cross-section of the three-dimensional model, for each cross-section, a rotation matrix corresponding to the cross-section is determined based on the cross-sections adjacent to the cross-section. The spatial coordinates of the vertices corresponding to the cross section are determined based on the rotation matrix, and the three-dimensional model of the linear object is rendered based on the spatial coordinates of multiple vertices, and the three-dimensional model is displayed in the augmented reality screen; In response to a display adjustment operation for the linear object, the adjusted 3D model of the linear object is displayed in the augmented reality screen, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

2. The method for processing augmented reality images according to claim 1, characterized in that, Determining the rotation matrix corresponding to the cross-section based on the cross-section adjacent to the cross-section includes: Using the vector between the center of the cross-section and the center of the cross-section adjacent to the cross-section as a reference vector, the rotation matrix corresponding to the cross-section is calculated based on the horizontal direction vector and the reference vector.

3. The method for processing augmented reality images according to claim 1, characterized in that, The rendering of the 3D model of the linear object based on the spatial coordinates of multiple vertices includes: Based on the preset rendering method of the three-dimensional model and the multiple vertices of the three-dimensional model, the surface to be rendered of the three-dimensional model is determined, wherein the surface to be rendered includes the cross-section to be rendered and the connecting surface between two adjacent cross-sections. For each cross section to be rendered, a triangular primitive is constructed based on every three vertices located on the cross section; For each connected surface to be rendered, a triangular primitive is constructed based on every three vertices located on different cross sections; The three-dimensional model of the linear object is rendered based on the spatial coordinates of multiple vertices and the triangular primitives of the face to be rendered.

4. The method for processing augmented reality images according to claim 3, characterized in that, The method of determining the surface to be rendered of the 3D model based on the preset rendering method of the 3D model and the multiple vertices of the 3D model includes: When the preset rendering mode of the three-dimensional model is continuous rendering, the starting cross-section and the ending cross-section of the three-dimensional model are used as the cross-sections to be rendered, and the connecting surfaces between all pairs of adjacent cross-sections are used as the connecting surfaces to be rendered.

5. The method for processing augmented reality images according to claim 3, characterized in that, The method of determining the surface to be rendered of the 3D model based on the preset rendering method of the 3D model and the multiple vertices of the 3D model includes: When the preset rendering mode of the three-dimensional model is intermittent rendering, the starting cross-section, the ending cross-section, and at least two other cross-sections of the three-dimensional model are all taken as cross-sections to be rendered; and, based on the cross-sections to be rendered, the connecting surfaces to be rendered are determined so that the connecting surfaces are displayed intermittently.

6. The method for processing augmented reality images according to claim 1, characterized in that, The acquisition of multiple key points of the linear object corresponding to the augmented reality image includes: Multiple key points of a linear object corresponding to the augmented reality image are generated based on a preset algorithm; or, The associated objects of the linear object to be rendered in the augmented reality image are determined, and multiple key points of the linear object corresponding to the augmented reality image are determined based on the motion trajectory of the associated objects.

7. A processing apparatus for augmented reality images, characterized in that, include: The request module is used to respond to a rendering trigger request for an augmented reality image, obtain multiple key points of a linear object corresponding to the augmented reality image, and for each key point, draw a circle with the key point as the center, and determine multiple vertices of the three-dimensional model of the linear object based on the points located on the circle. Using the circle as a cross-section of the three-dimensional model, for each cross-section, a rotation matrix corresponding to the cross-section is determined based on the cross-sections adjacent to the cross-section. The spatial coordinates of the vertices corresponding to the cross section are determined based on the rotation matrix, and the three-dimensional model of the linear object is rendered based on the spatial coordinates of multiple vertices, and the three-dimensional model is displayed in the augmented reality screen; The display module is configured to display the adjusted 3D model of the linear object in the augmented reality screen in response to a display adjustment operation on the 3D model, wherein the display adjustment operation includes a display position adjustment operation, a display size adjustment operation, and / or a display angle adjustment operation.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the augmented reality image processing method as described in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the augmented reality image processing method as described in any one of claims 1-6.

Images