3D special effect display processing system suitable for edge domain chip and application

Abstract

The application discloses a 3D special effect display processing system and application suitable for an edge domain chip, and relates to the technical field of image processing. The system comprises a central processing unit and a special effect acceleration module located at a hardware layer, a special effect application software, a special effect acceleration module driver and an operating system located at a software layer, the special effect application software is used for collecting a special effect operation instruction of a user aiming at a target three-dimensional object on a screen display area; the special effect acceleration module driver sends the special effect operation instruction to the operating system after converting the special effect operation instruction into a specific special effect processing task; the operating system starts the special effect acceleration module to execute the aforementioned special effect processing task; the special effect acceleration module can perform real-time projection operation on the target three-dimensional object based on a viewpoint and a projection plane according to 3D setting parameters, and update the display result of the screen display area. The application does not need to use GPU hardware, has high graphic processing performance and does not occupy CPU resources, and has the advantages of low cost, high performance, high speed and the like.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a 3D special effects display processing system and application suitable for edge domain chips. Background Technology

[0002] Edge computing is a technology that utilizes computing and storage resources at the network edge to provide services to users at the network edge. It allows applications to process data near the data source, eliminating the need to upload large amounts of data to remote computing centers. Compared to cloud computing (a centralized computing service where all data is transmitted over the network to a cloud computing center for processing), edge computing offers the following advantages: 1) It alleviates network bandwidth and data center pressure; 2) It enhances real-time response; 3) It improves data security and reduces the risk of privacy breaches that may arise from data transmission to the cloud; and 4) It increases the reliability of data processing. Therefore, edge computing is particularly suitable for application scenarios with specific business requirements such as low latency, high bandwidth, high reliability, massive connectivity, heterogeneous aggregation, and local security and privacy protection.

[0003] In recent years, edge domain chips (also known as edge domain processor chips or edge computing chips) used to realize edge computing have been widely used in various fields such as medical, automotive, and the Internet of Things. With the application of edge domain chips in various fields, the demand for edge domain chip functions has also become increasingly diversified as products become more abundant. For example, in some computer terminals that require real-time user interaction, 3D (three-dimensional) special effects images may be used to achieve certain display effects, thereby increasing the intuitiveness and fun of the interaction. As an example, a 3D Earth image might be displayed on an interactive computer terminal, and this 3D Earth image could move, rotate, move away from, or move closer to the user's finger.

[0004] The process of generating 3D special effects images mainly includes steps such as 3D model building, perspective projection, and texture mapping. There are two main traditional methods for processing 3D special effects images.

[0005] The first method: uses a hardware GPU (Graphics Processing Unit) module and the OpenGL ES (OpenGL for Embedded Systems, a subset of the OpenGL 3D graphics API) software library interface. See also Figure 1As shown, the 3D special effects image processing system can include a central processing unit (CPU) and a graphics acceleration engine (GPU) at the hardware layer, and special effects application software, OpenGL ES, a hardware driver layer, and an operating system at the software layer. The special effects application software is user-oriented, collecting user operation commands and transmitting them sequentially through OpenGL ES, the hardware driver layer, and the operating system to the GPU and CPU at the hardware layer. This method utilizes the high-speed graphics processing capabilities of the GPU and the portable software interface of OpenGL. With the support of a powerful GPU computing module, it can provide real-time and diverse special effects support. However, the above-mentioned 3D special effects image processing has the following drawbacks: Firstly, the development cost and difficulty of GPU hardware are high, generally requiring the purchase of third-party IP, which is costly. Secondly, as a general-purpose graphics processing chip, the GPU hardware module has a large area and power consumption in order to support various graphics effects, making it difficult to meet the requirements of edge-domain chips for small area and low power consumption. At the same time, smaller area and lower power consumption are also one of the current development directions of various processor chips.

[0006] The second method involves using open-source graphics computing software libraries such as Mesa3D and the OpenGL ES software library interface. See also... Figure 2 As shown, the 3D special effects image processing system can include a central processing unit (CPU) at the hardware layer, and special effects application software, OpenGL ES, graphics computing software libraries, and an operating system at the software layer. The special effects application software is user-oriented; after collecting user operation commands through the application software, they are transmitted sequentially to the CPU at the hardware layer via OpenGL ES, the graphics computing software library, and the operating system. However, although adopting an open-source software solution eliminates the need for GPU hardware, reducing graphics processing costs, the reliance on the CPU for graphics calculations results in poor performance and high power consumption due to the CPU's limited graphics processing capabilities, making it difficult to meet real-time display requirements.

[0007] In summary, how to provide a low-cost, high-performance 3D effects processing method that does not consume CPU resources, based on the development needs of edge domain chips for small area and low power consumption, is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a 3D effects display processing system and application suitable for edge domain chips. The 3D effects display processing system provided by this invention does not require GPU hardware, but instead uses dedicated hardware modules for graphics computation acceleration. It offers high performance without consuming CPU resources, and has advantages such as low cost, high performance, and high speed, making it particularly suitable for configuration on edge domain chips.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A 3D special effects display processing system suitable for edge domain chips includes a central processing unit located at the hardware layer, and special effects application software and operating system located at the software layer. The special effects application software is user-oriented and configured to collect user special effects operation instructions for target three-dimensional objects on the screen display area.

[0011] A special effects acceleration module is also set at the hardware layer, and a special effects acceleration module driver is configured at the software layer corresponding to the special effects acceleration module. The special effects acceleration module driver provides an interface for calling the upper-layer software.

[0012] The special effects acceleration module driver is located between the special effects application software and the operating system. It is used to receive special effects operation instructions transmitted by the special effects application software, convert the special effects operation instructions into specific special effects processing tasks, and then send them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target 3D object. The operating system starts the special effects acceleration module to execute the aforementioned special effects processing tasks.

[0013] The special effects acceleration module is configured to: perform projection transformation, texture mapping and texture interpolation operations on the target 3D object based on the viewpoint and projection plane according to the aforementioned 3D setting parameters to obtain a real-time projection image of the target 3D object, and update the display result of the screen display area according to the real-time projection image; when the special effects operation command includes adjustment commands for multiple states of the target 3D object, display the projection images of the target 3D object in different states through the screen display area.

[0014] Furthermore, the special effects operation command is to continuously adjust the state of the target three-dimensional object based on preset rules. At this time, the 3D setting parameters of the target three-dimensional object are continuously and regularly adjusted based on the aforementioned preset rules.

[0015] The special effects acceleration module is configured to update the display result of the target 3D object in real time according to the adjusted 3D setting parameters, thereby outputting the 3D animation effect of the target 3D object.

[0016] Furthermore, the special effects acceleration module includes a projection transformation unit, a texture mapping unit, and a texture interpolation calculation unit;

[0017] The projection transformation unit is configured to: calculate normalized screen coordinates, wherein for any screen point P' on the screen display area, the screen point is transformed from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane; and calculate the coordinates of the object point corresponding to the screen point P' according to the perspective projection relationship, wherein for the aforementioned screen point P' on the projection plane, the coordinate position of the object point P corresponding to the screen point P' is obtained according to the perspective projection relationship.

[0018] The texture mapping unit is configured to: calculate the texture coordinates corresponding to object point P, wherein the rotation angle and rotation order information of the target three-dimensional object in the 3D setting parameters are obtained, the rotation inverse matrix is ​​calculated based on the aforementioned rotation angle and rotation order information, the coordinates of object point P rotated to the initial posture are obtained through the rotation inverse matrix, and the texture coordinates corresponding to the aforementioned object point P on the texture image under the initial posture are determined based on the preset texture mapping algorithm.

[0019] The texture interpolation unit is configured to: obtain the pixel value of the screen point by interpolation based on the aforementioned texture coordinates, wherein, after determining the aforementioned texture coordinates, the pixel value at the aforementioned texture coordinates is calculated by interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates, which is the pixel value of the screen point P' on the texture image.

[0020] Furthermore, the target three-dimensional object is a sphere, hemisphere, cylinder, or polyhedron;

[0021] The 3D setting parameters are related to the selected viewpoint and projection plane, and include near plane distance, far plane distance, center position of the 3D object, field of view, rotation angle of the 3D object, and rotation sequence information.

[0022] Furthermore, the target three-dimensional object is a sphere;

[0023] The projection transformation unit is configured to: for a screen point P'(x, y) on a screen display area with a width of W pixels and a height of H pixels, use the following formula (1)

[0024]

[0025] Transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the projection coordinates (X, X) of the screen point P' on the projection plane. n Y n (0, 0, -n), where n is the distance between the projection plane and the viewpoint. The projection plane is perpendicular to the Z-axis of the viewpoint coordinate system, and its center intersects the Z-axis at (0, 0, -n) in the viewpoint coordinate system. The width and height of the projection plane are W pixels and H pixels, respectively. Then, based on the projection point coordinates (X, -n) of screen point P' on the projection plane... n Y n ,-n), based on the spherical equation formula (2) and the perspective projection relationship formula (3).

[0026] X e 2 +Y e 2 +(Z e +d) 2=R 2 (2)

[0027]

[0028] Find the coordinates (X) of the object point P corresponding to the screen point P'. e Y e Z e ); where d represents the position of the center of the sphere, which is located at (0, 0, -d) in the viewpoint coordinate system; R represents the radius of the sphere; Indicates the aspect ratio of the projection plane; fov represents the field of view angle;

[0029] Substituting formula (2) into (3), we obtain the equation concerning Z. e The formula for a quadratic equation in one variable is as follows:

[0030] (t 2 ×a 2 ×X n 2 +t 2 ×Y n 2 +1)×Z e 2 +2×d×Z e +d 2 -R 2 =0 (4)

[0031] Solve formula (4) to obtain the Z-axis coordinate value Z of the object point P on the sphere. e Then, according to Z e The X-axis coordinate X of point P of the object is calculated using formula (3). e and Y-axis coordinate value Y e .

[0032] Furthermore, the texture mapping unit is configured as follows:

[0033] Obtain the rotation angle and rotation sequence information of the sphere from the 3D settings parameters, and let the rotation angle of the sphere around the X, Y, and Z axes be γ respectively. x γ y γ z The corresponding rotation matrices are M, respectively. x M y M z The calculation formula is as follows:

[0034]

[0035]

[0036]

[0037] The total rotation matrix is ​​calculated as M based on the rotation sequence of the sphere. 3D M 3D Let M be the rotation matrix. x M y and M z The product of M, where M x M y and M z The order of multiplication matches the order of rotation; according to M 3D Calculate the rotation inverse matrix M 3D -1 Then, calculate the coordinates (X0) of point P in the initial attitude. m Y0 m Z0 m The calculation formula is as follows:

[0038]

[0039] Among them, X e Y e Z e These represent the X-axis, Y-axis, and Z-axis coordinates of point P, respectively.

[0040] And, according to the aforementioned (X0) m Y0 m Z0 m The polar coordinates of point P under the initial attitude are calculated using formula (9) based on the polar coordinate equation. as follows,

[0041]

[0042] According to the aforementioned polar coordinates Calculate the texture coordinates (X) corresponding to point P. t Y t ),as follows,

[0043]

[0044] Among them, W t H t These represent the width and height in pixels of the texture image, respectively.

[0045] Furthermore, the texture interpolation unit is configured as follows:

[0046] Based on the aforementioned texture coordinates (X) t Y t Let P be the point on the texture image corresponding to this coordinate. t Obtain the texture image with respect to point P. tThe four nearest neighbor pixel values ​​P0 t P1 t P2 t and P3 t After obtaining the pixel values ​​T0, T1, T2, and T3, P is calculated by combining them with preset interpolation weights. t The pixel value T of the point is as follows.

[0047] T = [T0 × (1 - w)] x )+T1×w x ]×(1-w y )+[T2×(1-w x )+T3×w x ]×w y (11)

[0048] That is, the pixel value of screen point P' on the texture image; where, w x and w y These represent the preset interpolation weights in the horizontal and vertical directions, respectively, and are calculated using the following formulas.

[0049]

[0050] in, X represents t The integer function, representing less than or equal to X. t The largest integer; Y represents t The integer function, representing less than or equal to Y. t The largest integer.

[0051] Furthermore, obtain the Z-axis coordinate value Z of the object point P on the sphere. e The steps are as follows:

[0052] Judgment about Z e Does the quadratic equation in one variable have a solution?

[0053] When a solution is determined to exist, two solutions are obtained, namely Z. e0 and Z e1 Based on the occlusion relationship, discard points that are far from the projection plane, that is, let Z e =MIN(Z) e0 Z e1 ), where MIN(·) means returning the minimum value within the parentheses; continue judging Z. e Whether it lies between the far plane distance and the near plane distance; when Z e When the object point is located between the far plane distance and the near plane distance, it is determined to be a visible point. Then, the aforementioned Z... e Substituting the value into formula (3), the X-axis coordinate value X of object point P is calculated. e and Y-axis coordinate value Ye When Z e If the object point is not located between the far plane distance and the near plane distance, it is determined to be an invisible point. The pixel value of the screen point P' corresponding to the object point P is configured to the preset background color, and the pixel adjustment process of the screen point P' ends.

[0054] If no solution is found, the pixel value of the screen point P' corresponding to the object point P is directly configured to the preset background color, and the pixel adjustment process for the screen point P' ends.

[0055] This invention also provides a 3D special effects display processing method suitable for edge domain chips, comprising the following steps:

[0056] The software layer uses special effects to capture user commands for special effects operations on target 3D objects on the screen display area.

[0057] The special effects acceleration module driver receives special effects operation instructions transmitted by the special effects application software, converts the special effects operation instructions into specific special effects processing tasks, and sends them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target three-dimensional object.

[0058] The operating system launches a special effects acceleration module located at the hardware layer. This module can perform projection transformation, texture mapping, and texture interpolation operations on the target 3D object based on the aforementioned 3D setting parameters, according to the viewpoint and projection plane, to obtain a real-time projected image of the target 3D object. It then updates the display result of the screen display area based on the real-time projected image. When the special effects operation command includes adjustment commands for multiple states of the target 3D object, the projected images of the target 3D object in different states are displayed on the screen display area.

[0059] Furthermore, the special effects acceleration module performs projection transformation, texture mapping, and texture interpolation calculations as follows:

[0060] Calculate normalized screen coordinates: For any screen point P' on the screen display area, transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane.

[0061] Based on perspective projection, calculate the coordinates of the object point corresponding to the screen point: For the aforementioned screen point P' on the projection plane, obtain the coordinate position of the object point P corresponding to the screen point P' based on perspective projection.

[0062] Calculate the texture coordinates corresponding to the object point: Obtain the rotation angle and rotation order information set by the user, calculate the rotation inverse matrix based on the aforementioned rotation angle and rotation order information, and obtain the coordinates of the object point P rotated to the initial posture through the rotation inverse matrix; Based on the preset texture mapping algorithm, determine the texture coordinates of the aforementioned object point P on the texture image under the initial posture;

[0063] Based on the aforementioned texture coordinates, the pixel value of the screen point is obtained through interpolation: After determining the texture coordinates, the pixel value at the texture coordinate position is calculated using an interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates. This pixel value is the pixel value of the screen point P' on the texture image.

[0064] Compared with the prior art, the present invention, by adopting the above technical solution, has the following advantages and positive effects: The 3D special effects display processing system provided by the present invention does not require the use of GPU hardware, but instead uses a dedicated hardware module to accelerate graphics calculation. It has high performance and does not occupy CPU resources. It has the advantages of low cost, high performance and fast speed, and is especially suitable for configuration on edge domain chips.

[0065] The special effects acceleration algorithm provided by this invention is used to accelerate graphics computing. The algorithm is simple to implement, does not require caching of additional intermediate data, and consumes little system bandwidth resources.

[0066] On the other hand, there is no need to purchase GPU hardware IP or integrate GPU hardware into the chip. While ensuring graphics processing performance, the device cost is reduced, and the chip area and power consumption are reduced, which meets the design requirements of small chip area and low power consumption in the edge domain.

[0067] On the other hand, it uses dedicated special effects acceleration hardware for graphics computing acceleration, which is high-performance and does not consume CPU resources; at the same time, since it does not require OpenGL ES graphics interface library support, the software interface is simple. Attached Figure Description

[0068] Figure 1 This is a logic block diagram of a 3D special effects image display and processing system provided in the prior art.

[0069] Figure 2 A logical block diagram of another 3D special effects image display processing system provided in the prior art.

[0070] Figure 3 The present invention provides a logic block diagram of a 3D special effects display processing system suitable for edge domain chips.

[0071] Figure 4 This is a schematic diagram of the processing flow of the special effects acceleration module provided by the present invention.

[0072] Figure 5 A perspective projection diagram of a sphere provided for an embodiment of the present invention.

[0073] Figure 6 for Figure 5 A perspective projection diagram of the YOZ section.

[0074] Figure 7 This is a schematic diagram of the display area coordinate system provided in an embodiment of the present invention.

[0075] Figure 8 This is a schematic diagram of the normalized projected coordinate system provided in an embodiment of the present invention.

[0076] Figure 9 This is a schematic diagram of the effective projection area provided in an embodiment of the present invention.

[0077] Figure 10 This is a schematic diagram of the posture of a sphere during projection, provided in an embodiment of the present invention.

[0078] Figure 11 This is a schematic diagram of the initial posture of the sphere provided in an embodiment of the present invention.

[0079] Figure 12 This is a schematic diagram of the texture coordinate system provided in an embodiment of the present invention.

[0080] Figure 13 This is an example diagram illustrating the point position relationship when performing bilinear interpolation on a texture image, as provided in an embodiment of the present invention. Detailed Implementation

[0081] The 3D special effects display processing system and its application for edge domain chips disclosed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated; they can be combined with each other to achieve better technical effects. In the accompanying drawings of the following embodiments, the same reference numerals appearing in each drawing represent the same features or components, which can be applied to different embodiments. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0082] It should be noted that the structures, proportions, sizes, etc., illustrated in the accompanying drawings are merely for illustrative purposes and to aid those skilled in the art in understanding and reading the invention. They are not intended to limit the conditions under which the invention can be implemented. Any modifications to the structure, changes in proportions, or adjustments to size, provided they do not affect the effectiveness or purpose of the invention, should fall within the scope of the technical content disclosed in the invention. The scope of the preferred embodiments of the present invention includes other implementations, wherein functions may be performed not in the order stated or discussed, including substantially simultaneously or in reverse order, depending on the functions involved. This should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0083] Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0084] Example

[0085] See Figure 3 As shown, this embodiment provides a 3D special effects display processing system suitable for edge domain chips.

[0086] The system includes a central processing unit and special effects acceleration module at the hardware layer, and special effects application software, special effects acceleration module drivers, and an operating system at the software layer. The operating system includes system programs for handling various basic system services and performing hardware-related tasks, enabling various basic business operations and handling hardware-based tasks.

[0087] The special effects application software is user-configurable and used to collect user commands for special effects operations on target 3D objects on the screen display area. The special effects acceleration module is responsible for calculations such as projection transformation, texture mapping, and texture interpolation. The special effects acceleration module driver, as the device driver for the special effects acceleration module, provides an interface for upper-level software to call and serves as a communication program between the upper-level software and the device control system.

[0088] In this embodiment, the special effects acceleration module driver is located between the special effects application software and the operating system. It is used to receive special effects operation instructions transmitted by the special effects application software, convert the special effects operation instructions into specific special effects processing tasks, and then send them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target three-dimensional object. The operating system starts the special effects acceleration module to execute the aforementioned special effects processing tasks.

[0089] The special effects acceleration module is configured to: perform projection transformation, texture mapping, and texture interpolation operations on the target 3D object based on the aforementioned 3D setting parameters, using the viewpoint and projection plane, to obtain a real-time projected image of the target 3D object, and update the display result of the screen display area based on the real-time projected image. When the special effects operation command includes adjustment commands for multiple states of the target 3D object, the projected images of the target 3D object in different states are displayed through the screen display area.

[0090] Preferably, if the special effects operation command is to continuously adjust the state of the target 3D object based on preset rules (i.e., including a series of special effects operation commands), the system can continuously and systematically adjust the 3D setting parameters of the target 3D object based on the aforementioned preset rules. Correspondingly, the special effects acceleration module is configured to: update the display result of the target 3D object in real time according to the adjusted 3D setting parameters, thereby outputting the 3D animation effect of the target 3D object.

[0091] At this point, a series of special effects operation instructions can be implemented using special effects scripts. Specifically, after receiving the special effects operation information set by the user, the special effects application software can generate a special effects script corresponding to the aforementioned special effects operation information, and then pass the special effects script to the special effects acceleration driver, that is, pass the special effects operation information to the special effects acceleration driver in the form of a scripting language. After parsing the script, the special effects acceleration driver obtains the specific special effects processing task.

[0092] In this embodiment, the special effects acceleration module may specifically include a projection transformation unit, a texture mapping unit, and a texture interpolation operation unit. The following is in conjunction with... Figure 4 This describes the information processing process of each unit.

[0093] First, for any point P (i.e., the object point) on the surface of the target 3D object, assume that its screen point on the screen display area is P'.

[0094] The projection transformation unit is configured to: calculate normalized screen coordinates, and calculate the coordinates of the object point corresponding to screen point P' based on perspective projection relationship.

[0095] In practice, the steps for calculating normalized screen coordinates can be as follows: For any screen point P' on the current screen display area, transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane.

[0096] Subsequently, the coordinates of the object point corresponding to screen point P' are calculated based on perspective projection relationships. The specific steps are as follows: For the aforementioned screen point P' on the projection plane, the coordinates of the object point P corresponding to screen point P' are obtained based on perspective projection relationships.

[0097] The texture mapping unit is configured to calculate the texture coordinates corresponding to object point P.

[0098] The specific steps can be as follows: obtain the rotation angle and rotation sequence information of the target 3D object in the 3D setting parameters, calculate the rotation inverse matrix based on the aforementioned rotation angle and rotation sequence information, obtain the coordinates of object point P rotated to the initial posture through the rotation inverse matrix, and determine the texture coordinates of the aforementioned object point P on the texture image under the initial posture based on the preset texture mapping algorithm.

[0099] The texture interpolation unit is configured to obtain the pixel value of the screen point by interpolation based on the aforementioned texture coordinates.

[0100] The specific steps can be as follows: After determining the aforementioned texture coordinates, the pixel value at the aforementioned texture coordinates is calculated using an interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates. This pixel value is the pixel value of screen point P' on the texture image.

[0101] In this embodiment, the target three-dimensional object can be a sphere, hemisphere, cylinder, or polyhedron, such as a regular tetrahedron or a regular hexahedron. The 3D model construction of each three-dimensional object can refer to existing three-dimensional object models, and will not be elaborated further here.

[0102] The 3D setting parameters are related to the selected viewpoint and projection plane. Specifically, the 3D setting parameters may include parameters such as near plane distance, far plane distance, center position of the 3D object, field of view, rotation angle of the 3D object, and rotation sequence information.

[0103] The near plane represents the near section of the perspective projection (perspective projection parameter near), and the distance to the near plane is the distance between the near plane and the viewpoint. The far plane represents the far section of the perspective projection (perspective projection parameter far), and the distance to the far plane is the distance between the far plane and the viewpoint. The settings for the near and far planes can be configured by the user or by system default settings.

[0104] The technical solution provided by this invention can customize various 3D effects for various three-dimensional objects on the edge domain chip, such as 3D effects for spheres, hemispheres and other three-dimensional objects. It can realize real-time perspective projection and texture mapping calculation of three-dimensional objects according to the user's operation instructions on the effects of three-dimensional objects - such as the viewpoint change of three-dimensional objects, object movement and / or object rotation operation instructions, so as to update the two-dimensional projection image (or two-dimensional screen image) of the corresponding three-dimensional object in the screen display area.

[0105] As a preferred typical approach, the special effects display processing flow of this embodiment is described in detail below, taking the target three-dimensional object as a sphere as an example.

[0106] For ease of description, see Figure 5 and Figure 6 As shown, the settings are:

[0107] The viewpoint coordinate system is XYZ, and the viewpoint is located at the origin of the viewpoint coordinate system;

[0108] Object coordinate system X m Y m Z m The center of the sphere is located at the origin of the object's coordinate system;

[0109] The X-axis of the object coordinate system is parallel to the X-axis of the viewpoint coordinate system, the Y-axis of the object coordinate system is parallel to the Y-axis of the viewpoint coordinate system, and the Z-axis of the object coordinate system coincides with the Z-axis of the viewpoint coordinate system.

[0110] The projection plane is perpendicular to the Z-axis of the viewpoint coordinate system and its center intersects the Z-axis at (0, 0, -n) in the viewpoint coordinate system. The width and height of the projection plane are W pixels and H pixels, respectively.

[0111] The near plane coincides with the projection plane, is perpendicular to the Z-axis, and its center intersects the Z-axis at the viewpoint coordinate system (0, 0, -n); at this time, the distance between the near plane and the viewpoint is also n, the distance between the far plane and the viewpoint is f, and the field of view angle is fov;

[0112] The center of the sphere is located at (0, 0, -d) in the viewpoint coordinate system;

[0113] The radius of the sphere is R;

[0114] P is a point on the surface of a sphere, and its position is P' on the projection plane after perspective projection.

[0115] When setting the sphere projection, it has already been rotated γ around the X-axis of the object coordinate system. x Y-axis rotation γ y Z-axis rotation γ z .

[0116] At this time, the 3D setting parameters include near plane distance n, far plane distance f, sphere center position d, field of view fov, sphere rotation angle and rotation sequence, etc., which can obtain projected images in different states; when one or more parameters in the 3D setting parameters are changed continuously and regularly, the special effects acceleration module can update the display results of the screen display area in real time according to the adjustment of the above parameters, thereby obtaining the 3D special effects animation effect of the sphere.

[0117] In practice, the normalized screen coordinates are calculated first.

[0118] For a screen point P'(x, y) on a screen display area (viewport) with a width of W pixels and a height of H pixels, the following formula is used:

[0119]

[0120] Point the screen from Figure 7 The example displays the coordinate system (xoy) of the display area, which is then transformed to... Figure 8 The normalized projected coordinate system (X) in the example n OY n ), obtain the projection coordinates (X, X) of screen point P' on the projection plane. n Y n ( , -n). Where n is the distance between the projection plane and the viewpoint. The projection plane is perpendicular to the Z-axis of the viewpoint coordinate system and its center intersects the Z-axis at (0, 0, -n) in the viewpoint coordinate system. The width and height of the projection plane are the same as the width and height of the screen display area, which are W pixels and H pixels, respectively.

[0121] Then, based on the projection coordinates (X) of screen point P' on the projection plane... n Y n The coordinates (X, -n) of the object point P corresponding to the screen point P' can be obtained using the spherical equation formula (2) and the perspective projection relationship formula (3). e Y e Z e ).

[0122] According to the equation of the sphere, we have:

[0123] X e 2 +Y e 2 +(Z e +d) 2 =R 2 (2)

[0124] According to perspective projection relationships, we have:

[0125]

[0126] in, This represents the aspect ratio of the projection plane, where W and H are the width and height of the projection plane, respectively. fov represents the field of view angle. Substituting equation (2) into equation (3), we obtain the relationship between Z and Z. e The formula for a quadratic equation in one variable is as follows:

[0127] (t 2 ×a 2 ×X n 2 +t 2 ×Y n 2 +1)×Z e 2 +2×d×Z e +d2 -R 2 =0 (4)

[0128] By solving formula (4), the Z-axis coordinate Z of point P on the sphere can be obtained. e Then, according to Z e The X-axis coordinate X of point P of the object is calculated using formula (3). e and Y-axis coordinate value Y e .

[0129] Specifically, solve for Z e A quadratic equation in one variable may have the following two cases.

[0130] Case 1: The equation has no solution.

[0131] At this point, it is determined that screen point P' is not a projection point of any point on the sphere. Therefore, screen point P' is filled with the background color, that is, the pixel value of screen point P' is configured as the pixel value of the background color; at this point, the processing of this point is completed.

[0132] Case 2: The equation has a solution.

[0133] At this point, let the two solutions to the equation be Z. e0 and Z e1 (Z e0 =Z e1 (This indicates that there is only one solution). Based on the occlusion relationship, points far from the projection plane are discarded, and the Z coordinates of the effective projection points are obtained. e That is, Z e =MIN(Z) e0 Z e1 ).

[0134] Determine the Z-axis coordinate value of the object point. e After determining the value of Z, it is also necessary to consider... e Determine the visibility of the point on the object.

[0135] In this embodiment, it is achieved by determining Z. e Whether an object point is visible is determined by whether it lies between the distances from the far plane and the near plane. When Z... e When the object point is located between the far plane distance and the near plane distance, it is determined to be a visible point; when Z... e If the object point is not located between the distances of the far plane and the near plane, it is determined to be an invisible point.

[0136] Specifically, based on the near-plane distance n and far-plane distance f in the aforementioned 3D settings parameters, the following logic can be executed: determine if -f ≤ Z e Does ≤-n hold true?

[0137] If -f≤Ze When ≤-n is true, the object point is determined to be within the valid projection area. See [link to relevant documentation]. Figure 9 As shown, this point on the object is a visible point. For visible points, the aforementioned Z... e Substituting the value into formula (3), the X-axis coordinate value X of object point P is calculated. e and Y-axis coordinate value Y e Then, subsequent texture mapping and texture interpolation operations are performed.

[0138] If -f≤Z e When ≤-n does not hold, i.e., Z e <-f or Z e When the value is greater than -n, the object point is determined to be outside the effective projection area and is therefore invisible. For invisible points, the background color is filled, that is, the pixel value of the screen point P' is configured to the pixel value of the background color; at this point, the processing of that point is complete.

[0139] Determine the coordinates (X, Y) of point P on the sphere. e Y e Z e After that, the texture coordinates corresponding to object point P can be calculated.

[0140] Specifically, first calculate the rotation inverse matrix based on the set rotation angle and rotation sequence, and then use the rotation inverse matrix to rotate point P to the position of the initial posture.

[0141] Let the rotation angles of the sphere about the X, Y, and Z axes be γ respectively. x γ y γ z For the attitude during spherical projection, please refer to Figure 10 As shown, the corresponding rotation matrices at this time are M. x M y M z The M x M y M z The calculation formula is as follows:

[0142]

[0143]

[0144]

[0145] The total rotation matrix is ​​calculated as M based on the rotation sequence of the sphere. 3D M 3D Let M be the rotation matrix. x M y and M z The product of M, where M x My and M z The order of multiplication matches the order of rotation; specifically, the order of multiplication is the reverse of the order of rotation.

[0146] For example, let's say the ball first rotates γ around the X-axis. x Then rotate γ around the Y-axis y Finally, rotate γ around the Z-axis z Then the corresponding rotation matrix M 3D =M zyx M zyx =M z M y M x .

[0147] For example, the ball first rotates γ around the Y-axis. y Then rotate γ around the X-axis x Finally, rotate γ around the Z-axis z Then the corresponding rotation matrix M 3D =M zxy M zxy =M z M x M y .

[0148] With M 3D =M zyx For example, according to M zyx Calculate the rotation inverse matrix M zyx -1 Then, the coordinates (X0) of point P in the initial attitude can be calculated. m Y0 m Z0 m The calculation formula is as follows:

[0149]

[0150] Among them, X e Y e Z e These represent the X-axis, Y-axis, and Z-axis coordinates of point P, respectively.

[0151] See the initial posture of the sphere. Figure 11 As shown. Figure 11 In this context, phi represents the azimuth angle, and theta represents the elevation angle.

[0152] Next, based on the mapping relationship, the texture coordinates of point P on the texture image under the initial pose are determined. The texture image is a user- or system-preset image with known pixel information. The shape of the texture image and the texture mapping algorithm can be adaptively selected according to actual application requirements; these are existing technologies and will not be elaborated upon here.

[0153] The following example uses a rectangular texture image, and the preferred mapping algorithm is as follows for texture mapping.

[0154] For the preset rectangular texture image, see [link / reference]. Figure 12 As shown, let the width be W. t pixels, height H t For each pixel, the texture coordinates (X) corresponding to point P are calculated using the following steps. t Y t ).

[0155] First, based on the aforementioned (X0) m Y0 m Z0 m The polar coordinates of point P in the initial posture are calculated using formula (9) based on the polar coordinate equation. as follows,

[0156]

[0157] Based on the aforementioned polar coordinates Calculate the texture coordinates (X) corresponding to point P. t Y t ),as follows,

[0158]

[0159] Among them, W t H t These represent the width and height in pixels of the texture image, respectively.

[0160] Finally, the pixel value of the aforementioned screen point P' is obtained by interpolation based on the texture coordinates. That is, after determining the texture coordinates, the texture value of the target point is calculated using an interpolation algorithm.

[0161] Preferably, a bilinear interpolation algorithm is used. Specifically, based on the aforementioned texture coordinates (X... t Y t Let P be the point on the texture image corresponding to this coordinate. t It can be used with P t The four nearest points P0 t P1 t P2 t and P3 t P is obtained by calculating the pixel value. t See pixel values ​​for points. Figure 13 As shown.

[0162] Because of P t Coordinates are (X) t Y t If P0 is the first of the three, then P0 is the second. t P1t P2 t and P3 t The coordinates are respectively in, X represents t The floor function (or floor function) represents the value less than or equal to X. t The largest integer; Y represents t The integer function, representing less than or equal to Y. t The largest integer; X represents t The ceiling function, representing greater than X. t The smallest integer; Y represents t The ceiling function, representing greater than Y t The smallest integer. Let P0 t P1 t P2 t and P3 t Given pixel values ​​T0, T1, T2, and T3, P can be calculated using the following formula. t The value T of the point:

[0163] T = [T0 × (1 - w)] x )+T1×w x ]×(1-w y )+[T2×(1-w x )+T3×w x ]×w y (11)

[0164] Among them, w x and w y The interpolation weights for the horizontal and vertical directions are calculated using the following formula:

[0165] Among them, w x and w y These are the interpolation weights in the horizontal and vertical directions, respectively.

[0166]

[0167] Thus, we obtain the pixel value T of screen point P′ on the texture image, which is the texture value.

[0168] Another embodiment of the present invention provides a 3D special effects display processing method suitable for edge domain chips. The method includes the following steps.

[0169] S100 uses software to collect user commands for special effects on target 3D objects on the screen display area through software-layer special effects.

[0170] S200, the special effects acceleration module driver receives special effects operation instructions transmitted by the special effects application software, converts the special effects operation instructions into specific special effects processing tasks, and sends them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target three-dimensional object.

[0171] S300, the operating system starts the special effects acceleration module located at the hardware layer. The special effects acceleration module can perform projection transformation, texture mapping and texture interpolation operations on the target three-dimensional object based on the viewpoint and projection plane according to the aforementioned 3D setting parameters to obtain a real-time projection image of the target three-dimensional object, and update the display result of the screen display area according to the real-time projection image; wherein, when the special effects operation command includes adjustment commands for multiple states of the target three-dimensional object, the projection images of the target three-dimensional object in different states are displayed through the screen display area.

[0172] In step S300 of this embodiment, the steps of the special effects acceleration module performing projection transformation, texture mapping and texture interpolation operations can be as follows.

[0173] S310, Calculate normalized screen coordinates: For any screen point P' on the screen display area, transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane.

[0174] S320, Calculate the coordinates of the object point corresponding to the screen point according to the perspective projection relationship: For the aforementioned screen point P' on the projection plane, obtain the coordinate position of the object point P corresponding to the screen point P' according to the perspective projection relationship.

[0175] S330, Calculate the texture coordinates corresponding to the object point: Obtain the user-set rotation angle and rotation order information, calculate the rotation inverse matrix based on the aforementioned rotation angle and rotation order information, and obtain the coordinates of the object point P rotated to the initial posture through the rotation inverse matrix; Based on the preset texture mapping algorithm, determine the texture coordinates corresponding to the aforementioned object point P on the texture image under the initial posture.

[0176] S340, Based on the aforementioned texture coordinates, the pixel value of the screen point is obtained by interpolation: After determining the texture coordinates, the pixel value at the texture coordinate position is calculated by interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates, which is the pixel value of the screen point P' on the texture image.

[0177] In this embodiment, the target three-dimensional object can be a sphere, hemisphere, cylinder, or polyhedron.

[0178] The 3D setting parameters are related to the selected viewpoint and projection plane. The 3D setting parameters may include near plane distance, far plane distance, center position of the 3D object, field of view, rotation angle of the 3D object, and rotation sequence information.

[0179] Other technical features are described in the preceding embodiments and will not be repeated here.

[0180] In the above description, the disclosure of this invention is not intended to limit itself to these aspects. Rather, within the scope of the objectives of this disclosure, components can be selectively and operationally combined in any number. Furthermore, terms such as “comprising,” “encompassing,” and “having” should be interpreted by default as inclusive or open-ended, rather than exclusive or closed, unless explicitly defined as such. All technical, scientific, or other terms are to be understood by those skilled in the art, unless defined as such. Public terms found in dictionaries should not be interpreted in the context of the relevant technical documents in an overly idealistic or impractical manner, unless explicitly defined as such in this disclosure. Any modifications or alterations made by those skilled in the art based on the foregoing disclosure are within the scope of the claims.

Claims

1. A 3D special effects display processing system suitable for edge-domain chips, comprising a central processing unit at the hardware layer, and special effects application software and an operating system at the software layer, wherein the special effects application software is user-configurable for acquiring user special effects operation commands for target 3D objects on the screen display area, characterized in that: At the hardware layer, there is also a special effects acceleration module that does not use the graphics acceleration engine GPU. Corresponding to the special effects acceleration module, there is a special effects acceleration module driver configured at the software layer. The special effects acceleration module driver provides an interface for upper-layer software to call. The special effects acceleration module driver is located between the special effects application software and the operating system. It is used to receive special effects operation instructions transmitted by the special effects application software, convert the special effects operation instructions into specific special effects processing tasks, and then send them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target 3D object. The operating system starts the special effects acceleration module to execute the aforementioned special effects processing tasks. The special effects acceleration module is configured to: perform projection transformation, texture mapping and texture interpolation operations on the target 3D object based on the viewpoint and projection plane according to the aforementioned 3D setting parameters to obtain a real-time projection image of the target 3D object, and update the display result of the screen display area according to the real-time projection image; when the special effects operation command includes adjustment commands for multiple states of the target 3D object, display the projection images of the target 3D object in different states through the screen display area. The special effects acceleration module includes a projection transformation unit, a texture mapping unit, and a texture interpolation operation unit. The projection transformation unit is configured to: calculate normalized screen coordinates, wherein for any screen point P' on the screen display area, the screen point is transformed from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane; and calculate the coordinates of the object point corresponding to the screen point P' according to the perspective projection relationship, wherein for the aforementioned screen point P' on the projection plane, the coordinate position of the object point P corresponding to the screen point P' is obtained according to the perspective projection relationship. The texture mapping unit is configured to: calculate the texture coordinates corresponding to object point P, wherein the rotation angle and rotation order information of the target three-dimensional object in the 3D setting parameters are obtained, the rotation inverse matrix is ​​calculated based on the aforementioned rotation angle and rotation order information, the coordinates of object point P rotated to the initial posture are obtained through the rotation inverse matrix, and the texture coordinates corresponding to the aforementioned object point P on the texture image under the initial posture are determined based on the preset texture mapping algorithm. The texture interpolation unit is configured to: obtain the pixel value of the screen point by interpolation based on the aforementioned texture coordinates, wherein, after determining the aforementioned texture coordinates, the pixel value at the aforementioned texture coordinates is calculated by interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates, which is the pixel value of the screen point P' on the texture image.

2. The 3D special effects display processing system according to claim 1, characterized in that: The special effects operation command is to continuously adjust the state of the target three-dimensional object based on preset rules. At this time, the 3D setting parameters of the target three-dimensional object are continuously and regularly adjusted based on the aforementioned preset rules. The special effects acceleration module is configured to update the display result of the target 3D object in real time according to the adjusted 3D setting parameters, thereby outputting the 3D animation effect of the target 3D object.

3. The 3D special effects display processing system according to claim 1, characterized in that: The special effects operation instructions adopt special effects scripts. After receiving the special effects operation information set by the user, the special effects application software generates a special effects script corresponding to the aforementioned special effects operation information.

4. The 3D special effects display processing system according to claim 1, characterized in that: The target three-dimensional object is a sphere, hemisphere, cylinder, or polyhedron; The 3D setting parameters are related to the selected viewpoint and projection plane, and include near plane distance, far plane distance, center position of the 3D object, field of view, rotation angle of the 3D object, and rotation sequence information.

5. The 3D special effects display processing system according to claim 4, characterized in that: The target three-dimensional object is a sphere; The projection transformation unit is configured to: for a width of pixels, height The screen point P'(x,y) on the screen display area of ​​a pixel is determined by the following formula (1). （1） Transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the projection coordinates of the screen point P' on the projection plane. Let n be the distance between the projection plane and the viewpoint. The projection plane is perpendicular to the Z-axis of the viewpoint coordinate system, and its center intersects the Z-axis at (0, 0, -n) in the viewpoint coordinate system. The width and height of the projection plane are... Pixels and Pixels; then, based on the coordinates of the projection point P' on the projection plane. Based on the spherical equation formula (2) and the perspective projection relationship formula (3). （2） （3） Find the coordinates of the object point P corresponding to the screen point P'. Where d represents the position of the center of the sphere, which is located at (0,0,-d) in the viewpoint coordinate system; R represents the radius of the sphere. , indicating the aspect ratio of the projection plane; , Indicates the field of view; Substituting formula (2) into (3), we obtain the following about The formula for a quadratic equation in one variable is as follows: （4） Solve formula (4) to obtain the object point P on the sphere. axis coordinates Afterwards, according to The object point P is calculated using formula (3). axis coordinates and axis coordinates .

6. The 3D special effects display processing system according to claim 5, characterized in that, The texture mapping unit is configured as follows: Obtain the rotation angle and rotation sequence information of the sphere from the 3D settings parameters, and make the sphere rotate around... , , The rotation angles of the shafts are as follows: , , The corresponding rotation matrices are as follows: , , The calculation formula is as follows: （5） （6） （7） The total rotation matrix is ​​calculated based on the rotation order of the sphere. , Rotation matrix , and The product of, where , and The order of multiplication matches the order of rotation; according to Calculate the rotation inverse matrix Then, calculate Coordinates of the point in the initial attitude The calculation formula is as follows: （8） in, , , They represent Pointed axis coordinate values axis coordinate values ​​and Axis coordinate values; And, according to the foregoing The initial attitude is calculated using formula (9) based on the polar coordinate equation. polar coordinates of the point ,as follows, （9） According to the aforementioned polar coordinates calculate Texture coordinates corresponding to the point ,as follows, （10） Among them, W t H t These represent the width and height in pixels of the texture image, respectively.

7. The 3D special effects display processing system according to claim 6, characterized in that, The texture interpolation unit is configured as follows: Based on the aforementioned texture coordinates Let P be the point on the texture image corresponding to this coordinate. t Obtain the texture image with respect to point P. t The four nearest neighbor pixel values ​​P0 t P1 t P2 t and P3 t After obtaining the pixel values ​​T0, T1, T2, and T3, the interpolation weights are combined to calculate the result. The pixel value T of the point is as follows. （11） That is, the pixel value of screen point P' on the texture image; where, and These represent the interpolation weights in the horizontal and vertical directions, respectively, and are calculated using the following formulas. （12） in, express The integer function, representing less than or equal to The largest integer; express The integer function, representing less than or equal to The largest integer.

8. The 3D special effects display processing system according to claim 5, characterized in that, Get the object point P on the sphere axis coordinates The steps are as follows: Judge about Does the quadratic equation in one variable have a solution? When a solution is determined to exist, two solutions are obtained: and Discard points that are far from the projection plane based on the occlusion relationship, that is, let ,in, This indicates that the minimum value within the parentheses should be returned; the evaluation continues. Whether it lies between the far plane distance and the near plane distance; when When the object point is located between the far plane distance and the near plane distance, it is determined to be a visible point, and then the aforementioned... Substituting the value into formula (3) yields the value of object point P. axis coordinates and axis coordinates ;when If the object point is not located between the far plane distance and the near plane distance, it is determined to be an invisible point. The pixel value of the screen point P' corresponding to the object point P is configured to the preset background color, and the pixel adjustment process of the screen point P' ends. If no solution is found, the pixel value of the screen point P' corresponding to the object point P is directly configured to the preset background color, and the pixel adjustment process for the screen point P' ends.

9. A 3D special effects display processing method suitable for edge domain chips, characterized in that... Including the following steps: The software layer uses special effects to capture user commands for special effects operations on target 3D objects on the screen display area. The special effects acceleration module driver receives special effects operation instructions transmitted by the special effects application software, converts the special effects operation instructions into specific special effects processing tasks, and sends them to the operating system. The special effects processing tasks are configured with 3D setting parameters of the target three-dimensional object. The operating system launches a special effects acceleration module located at the hardware layer that does not use the graphics acceleration engine (GPU). This module can perform projection transformation, texture mapping, and texture interpolation operations on the target 3D object based on the aforementioned 3D setting parameters, according to the viewpoint and projection plane, to obtain a real-time projected image of the target 3D object. It then updates the display result of the screen display area based on the real-time projected image. When the special effects operation command includes adjustment commands for multiple states of the target 3D object, the projected images of the target 3D object in different states are displayed on the screen display area. The special effects acceleration module performs projection transformation, texture mapping, and texture interpolation calculations as follows: Calculate normalized screen coordinates: For any screen point P' on the screen display area, transform the screen point from the display area coordinate system to the normalized projection coordinate system to obtain the coordinate position of the screen point P' on the projection plane. Based on perspective projection, calculate the coordinates of the object point corresponding to the screen point: For the aforementioned screen point P' on the projection plane, obtain the coordinate position of the object point P corresponding to the screen point P' based on perspective projection. Calculate the texture coordinates corresponding to the object point: Obtain the rotation angle and rotation order information set by the user, calculate the rotation inverse matrix based on the aforementioned rotation angle and rotation order information, and obtain the coordinates of the object point P rotated to the initial posture through the rotation inverse matrix; Based on the preset texture mapping algorithm, determine the texture coordinates of the aforementioned object point P on the texture image under the initial posture; Based on the aforementioned texture coordinates, the pixel value of the screen point is obtained through interpolation: After determining the texture coordinates, the pixel value at the texture coordinate position is calculated using an interpolation algorithm based on the known pixel values ​​of the points near the texture coordinates. This pixel value is the pixel value of the screen point P' on the texture image.

Images