Image rendering method and device of intelligent wearable equipment, computer equipment and medium

Abstract

The invention relates to the technical field of intelligent wearing, and discloses an image rendering method and device for intelligent wearing equipment, computer equipment and a medium, and the method comprises the steps: enabling a main control chip to combine a left rendering projection and a right rendering projection to generate a stacked image, and transmitting the stacked image to an FPGA through one path of MIPI; and analyzing the stacked image through the FPGA, generating a left parallax image and a right parallax image, and pushing the left parallax image and the right parallax image to a display screen through two DSIs. Through the above mode, after the main control chip merges the left side rendering projection and the right side rendering projection of the target object to generate the stacked image, the stacked image is sent to the FPGA through one path of MIPI, the stacked image is subjected to image analysis through the FPGA to generate the left parallax image and the right parallax image, and the left parallax image and the right parallax image are sent to the left display screen and the right display screen through one path of DSI respectively. The requirement for the number of the MIPI interfaces on the main control processor of the intelligent wearable device is reduced, and double-screen image rendering of the intelligent wearable device is achieved under the condition that the number of the MIPI interfaces is insufficient.

Description

Technical Field

[0001] The present application relates to the field of smart wearable technology, and in particular to an image rendering method, device, computer equipment and medium for a smart wearable device. Background Art

[0002] Smart wearable devices can achieve stereoscopic imaging. The main principle is to display images with parallax under the left and right viewing angles of the smart wearable device. Usually, there are two MIPIs (Mobile Industry Processor Interface) on the main control chip to send images to the display for display. In the prior art, dual-screen image rendering has certain requirements on the interface on the main control processor, that is, without two MIPIs, the parallax display of the dual screens cannot be guaranteed. Therefore, how to realize dual-screen image rendering of smart wearable devices when the number of MIPIs is insufficient has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides an image rendering method, apparatus, computer equipment and storage medium for a smart wearable device, which realize dual-screen image rendering of the smart wearable device when the number of MIPIs is insufficient.

[0004] In a first aspect, the present application provides an image rendering method for a smart wearable device, the method comprising:

[0005] Acquire the left rendering projection and the right rendering projection of the target object through the main control chip;

[0006] Merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI;

[0007] The stacked images are parsed by the FPGA to generate a left parallax image and a right parallax image, and the left parallax image and the right parallax image are pushed to the left screen and the right screen respectively through the two DSIs to achieve image rendering of the target object on the left screen and the right screen.

[0008] In a second aspect, the present application further provides an image rendering device for a smart wearable device, the device comprising:

[0009] A rendering projection acquisition module, used to acquire a left rendering projection and a right rendering projection of a target object through the main control chip;

[0010] An image stacking module, used for merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through the MIP;

[0011] A rendering module is used to parse the stacked images through the FPGA to generate a left parallax image and a right parallax image, and push the left parallax image and the right parallax image to the left screen and the right screen through the two DS I respectively, so as to realize the image rendering of the target object on the left screen and the right screen.

[0012] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the image rendering method of the smart wearable device as described above when executing the computer program.

[0013] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method of the smart wearable device as described above.

[0014] The present application discloses an image rendering method, apparatus, computer equipment and medium for a smart wearable device. The image rendering method for the smart wearable device comprises obtaining a left rendering projection and a right rendering projection of a target object through the main control chip; merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIP; parsing the stacked image through the FPGA to generate a left parallax image and a right parallax image, and pushing the left parallax image and the right parallax image to the left screen and the right screen respectively through two DSIs to achieve image rendering of the target object on the left screen and the right screen. Through the above method, the present application combines the left rendering projection and the right rendering projection of the target object through the main control chip to generate a stacked image, and then sends the stacked image to the FPGA through a MIP I. After the stacked image is analyzed by the FPGA, the left parallax image and the right parallax image are generated, and are sent to the left and right display screens respectively through a DSI. The requirement on the number of MIP I interfaces on the main control processor of the smart wearable device is reduced, and dual-screen image rendering of the smart wearable device is realized when the number of MIP I interfaces is insufficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 is a schematic flow chart of an image rendering method for a smart wearable device provided in the first embodiment of the present application;

[0017] Figure 2 It is a module schematic diagram of an image rendering method for a smart wearable device provided in an embodiment of the present application;

[0018] Figure 3 is a schematic flow chart of an image rendering method for a smart wearable device provided in the second embodiment of the present application;

[0019] Figure 4 A schematic flowchart of an image rendering method for a smart wearable device provided in a third embodiment of the present application;

[0020] Figure 5 A schematic flowchart of an image rendering method for a smart wearable device provided in a fourth embodiment of the present application;

[0021] Figure 6 A schematic block diagram of an image rendering device for a smart wearable device provided in an embodiment of the present application;

[0022] Figure 7 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] The embodiments of the present application provide an image rendering method, apparatus, computer device and storage medium for a smart wearable device. The image rendering method for the smart wearable device can be applied to the smart wearable device, and the number of MIPI interfaces on the main control processor of the smart wearable device is reduced, so that dual-screen image rendering of the smart wearable device can be realized when the number of MIPI interfaces is insufficient.

[0028] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 This is a schematic flow chart of an image rendering method for a smart wearable device provided in the first embodiment of the present application. The image rendering method for the smart wearable device can be applied to the smart wearable device, and is used to generate a stacked image by merging the left rendering projection and the right rendering projection of the target object, and then send the stacked image to the FPGA through one MIPI. After the FPGA performs image analysis on the stacked image, a left parallax image and a right parallax image are generated, and they are sent to the display screen through one DSI respectively, which reduces the number of MIPI interfaces on the main control processor of the smart wearable device, and can still realize dual-screen image rendering of the smart wearable device when the number of MIPI interfaces is insufficient.

[0030] like Figure 1 As shown, the image rendering method of the smart wearable device specifically includes steps S10 to S30.

[0031] Step S10, obtaining a left rendering projection and a right rendering projection of the target object through the main control chip;

[0032] In one embodiment, the smart wearable device takes AR (Augmented Reality) glasses as an example. The smart glasses obtain 3D information of a target object within a preset range, and the 3D information may be Mesh (wireless network mesh) information, point cloud information, etc. of the target object.

[0033] Mesh is the mesh of the model, which is composed of lines through points, triangles through lines, and faces through triangles. The mesh surface is then expressed through materials, such as shadows, maps, etc. Generally speaking, mesh information includes vertex coordinates, normals, texture coordinates, triangle sequences, and mesh filters. In particular, when rendering images through Mesh, the shape of the target object must be obtained. The shape of the target object expressed by vertex coordinates, normals, texture coordinates, triangle sequences, and mesh filters is Mesh information.

[0034] Point cloud information refers to a set of vectors in a three-dimensional coordinate system. Each position of the target object is recorded in the form of a point. Each point contains three-dimensional coordinates and can carry other information about the properties of the point, such as color, reflectivity, intensity, etc. Point cloud information is usually obtained by laser scanners, cameras, 3D scanners and other devices, and can be used in applications such as 3D modeling, scene reconstruction, robot navigation, virtual reality and augmented reality. Smart glasses obtain the point cloud information of the target object and can generate an initial image based on the point cloud information, embedding the color, reflectivity and intensity of the target object into the initial image.

[0035] In one embodiment, the smart wearable device takes AR glasses as an example, and passes the 3D information of the target object through a graphics rendering device to construct left and right parallax images of the 3D information.

[0036] Parallax is the difference in direction when observing the same target object from two points at a certain distance. The angle between the two points from the current position of the target object is called the parallax angle of the two points, and the line between the two points is called the baseline. As long as the parallax angle and the length of the baseline are known, the distance between the target and the observer can be calculated. It is understandable that when a user looks at the same target object through the left and right lenses of the AR glasses, the target object will produce parallax in the user's left and right eyes.

[0037] The difference in the direction of the same object when viewed from two different positions. For example, when you put a finger in front of your eyes, close your right eye first and look at it with your left eye; then close your left eye and look at it with your right eye, you will find that the position of the finger relative to the distant object has changed. This is parallax when you look at the same point from different angles.

[0038] Because there is a distance between the left and right eyes, there is a slight difference in the perspective of the two eyes, and this difference will cause the scenes observed by the two eyes to shift a little bit. The reason why humans can produce a three-dimensional visual effect with a sense of space is precisely this displacement, which is called parallax in medicine, and is organically synthesized in the brain.

[0039] In one embodiment, this embodiment provides a method for constructing left and right parallax images of 3D information, and the process is: X = P*V*M*X_w, where X is the matrix after the parallax image is constructed for the 3D information, X_w is the 3D information matrix of the target object, M is the conversion matrix from the target object's own coordinate system to the world coordinate system, V is the (left / right) coordinate system conversion matrix projected from the world coordinate system to the AR glasses, and P is the image coordinate system conversion matrix projecting the 3D information to the 2D. The 3D information of the target object can be the mesh information, point cloud information, etc. provided in the above embodiments.

[0040] Step S20: merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI;

[0041] In one embodiment, the present embodiment merges the left and right images with parallax rendering through the main control chip and sends them to the FPGA end through one MIPI. The process of image merging is the process of image stacking.

[0042] Image stacking can be left-right stacking, top-bottom stacking, and alternating stacking.

[0043] In one embodiment, since there are certain requirements for the interface on the main control processor in the related art, if there are no two-way MIPI, the parallax display of the dual screen cannot be guaranteed. However, this embodiment has no requirements for the number of MIPIs. Through one-way MIPI, an FPGA is added for dual-screen image rendering, and two-way DSI is output to the left and right screens through the FPGA. The MIPI in this embodiment can be CSI (Camera Serial Interface) or DSI (Display Serial Interface). This embodiment increases the choice of the main control chip in the AR glasses. Even if there is not a sufficient number of MIPIs on some low-cost main control chips, dual-screen image rendering can also be achieved.

[0044] It can be understood that the left and right screens in this embodiment are the left and right lenses of the AR glasses, and the AR glasses present images to the user through the left and right lenses.

[0045] Step S30, parsing the stacked images through the FPGA to generate a left parallax image and a right parallax image, and pushing the left parallax image and the right parallax image to the left screen and the right screen through the two DSIs respectively, so as to achieve image rendering of the target object on the left screen and the right screen.

[0046] In one embodiment, since the image stacking methods include left-right stacking, top-bottom stacking, and alternating stacking, the process of analyzing the stacked images by FPGA in this embodiment is the inverse process of image stacking.

[0047] In one embodiment, Figure 2 As shown, Figure 2 A schematic diagram of a module of an image rendering method for a smart wearable device provided in an embodiment of the present application. Figure 2 It can be seen that after the 3D information is rendered and processed by the main control chip of the smart wearable device, the left rendering projection and the right rendering projection generated after rendering are stacked, and the generated stacked image is analyzed by FPGA, that is, the stacking process is reversed to generate left and right parallax images, and the left and right parallax images are sent to the left and right screens respectively through two DSIs.

[0048] The present embodiment discloses an image rendering method for a smart wearable device, which includes obtaining the left rendering projection and the right rendering projection of the target object through the main control chip; merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI; parsing the stacked image through the FPGA to generate a left parallax image and a right parallax image, and pushing the left parallax image and the right parallax image to the left screen and the right screen through two DSIs, respectively, to achieve the image rendering of the target object on the left screen and the right screen. Through the above method, the present application merges the left rendering projection and the right rendering projection of the target object through the main control chip to generate a stacked image, and then sends the stacked image to the FPGA through one MIPI. After the FPGA performs image parsing on the stacked image, the left parallax image and the right parallax image are generated, and are sent to the left and right display screens through one DSI respectively, which reduces the number of MIPI interfaces on the main control processor of the smart wearable device, and realizes the dual-screen image rendering of the smart wearable device when the number of MIPIs is insufficient.

[0049] See also Figure 3 , Figure 3This is a schematic flow chart of an image rendering method for a smart wearable device provided in the second embodiment of the present application. The image rendering method for the smart wearable device can be applied to the smart wearable device, and the main control chip constructs a parallax image according to 3D information, generates a left rendering projection and a right rendering projection, and merges the left rendering projection and the right rendering projection to generate a stacked image, and sends the stacked image to the FPGA through a MIP I, and performs image analysis on the stacked image through the FPGA to generate a left parallax image and a right parallax image, and sends them to the display screen through a DSI respectively, and by rendering and stacking the left and right parallax images, the deviation effect of the parallax on the final image rendering effect is reduced.

[0050] based on Figure 1 The embodiment shown, this embodiment Figure 3 As shown, step S10 includes step S101.

[0051] Step S101 : constructing left and right parallax images according to the 3D information of the target object through the main control chip, and generating the left rendering projection and the right rendering projection corresponding to the 3D information.

[0052] In one embodiment, parallax is the difference in direction when observing the same object from two points at a certain distance. The angle between the two points when viewed from the object is called the parallax angle of the two points, and the line between the two points is called the baseline. As long as the parallax angle and the baseline length are known, the distance between the object and the observer can be calculated. The left and right parallax images are the parallax images obtained by observing the target object from two different points on the left and right on the horizontal plane, that is, the left rendering projection and the right rendering projection in this embodiment.

[0053] In one embodiment, the essence of the left rendering projection and the right rendering projection is a disparity map. A disparity map refers to a two-dimensional image that stores the disparity values ​​of all pixels in a single view after stereo correction. The disparity map has the following characteristics:

[0054] 1. The disparity map is a two-dimensional image, the same size as the original image;

[0055] 2. The disparity value of the pixel at each position in the disparity map is stored in pixels;

[0056] 3. Taking the left view disparity map as an example, the disparity value at the pixel position p is equal to the column coordinates of the matching point of the pixel on the right image minus the column coordinates of the pixel on the left image.

[0057] The present embodiment discloses an image rendering method, device, computer equipment and medium for a smart wearable device, wherein the image rendering method for the smart wearable device includes constructing left and right parallax images according to the 3D information of the target object through the main control chip, and generating the left rendering projection and the right rendering projection corresponding to the 3D information. In the above manner, the present application constructs a parallax image according to the 3D information through the main control chip, generates a left rendering projection and a right rendering projection, and merges the left rendering projection and the right rendering projection to generate a stacked image, sends the stacked image to the FPGA through a MIP I, performs image analysis on the stacked image through the FPGA, generates a left parallax image and a right parallax image, and sends them to the display screen through a DSI respectively, and reduces the deviation effect of the parallax on the final image rendering effect by rendering and stacking the left and right parallax images.

[0058] See also Figure 4 , Figure 4 This is a schematic flow chart of an image rendering method for a smart wearable device provided in the third embodiment of the present application. The image rendering method for the smart wearable device can be applied to a server, and is used to clarify the relative position relationship of each position in the same coordinate system by converting the coordinate system of the target object, the coordinate system of the smart wearable device, and the world coordinate system, so as to facilitate the representation of each position in the same coordinate system, and improve the image rendering effect of the target object in the left screen and the right screen.

[0059] based on Figure 3 The embodiment shown, this embodiment Figure 4 As shown, step S101 includes steps S1011 to S1022.

[0060] Step S1011, obtaining a first conversion matrix, a second conversion matrix and a third conversion matrix;

[0061] The first conversion matrix is ​​a conversion matrix for converting the target object from the target object coordinate system to the world coordinate system, the second conversion matrix is ​​a conversion matrix for converting the target object from the world coordinate system to the smart wearable device coordinate system, and the third conversion matrix is ​​a conversion matrix for projecting the 3D information to the 2D image coordinate system;

[0062] In one embodiment, the coordinate conversion process is usually divided into two steps, firstly, the conversion parameters are solved by the common point coordinates, and then the non-common points are converted by the conversion parameters. The conversion parameters are usually divided into rotation, translation and scale parameters, among which the determination of the rotation parameters is the core of the coordinate conversion, and the rotation parameters are the conversion matrices in this embodiment.

[0063] Step S1012: Generate the left rendering projection and the right rendering projection based on a preset rendering algorithm, the 3D information, the first conversion matrix, the second conversion matrix, and the third conversion matrix.

[0064] The present embodiment discloses an image rendering method, device, computer device and medium of a smart wearable device, wherein the image rendering method of the smart wearable device includes obtaining a first conversion matrix, a second conversion matrix and a third conversion matrix; wherein the first conversion matrix is ​​a conversion matrix for converting the target object from the target object coordinate system to the world coordinate system, the second conversion matrix is ​​a conversion matrix for converting the target object from the world coordinate system to the smart wearable device coordinate system, and the third conversion matrix is ​​a conversion matrix for projecting the 3D information to the 2D image coordinate system; based on a preset rendering algorithm, the 3D information, the first conversion matrix, the second conversion matrix and the third conversion matrix, the left rendering projection and the right rendering projection are generated. In the above manner, the present application clarifies the respective positions and relative position relationships in the same coordinate system according to the conversion relationship between the coordinate systems such as the target object coordinate system, the smart wearable device coordinate system and the world coordinate system, so as to facilitate the representation of the respective positions in the same coordinate system, which is conducive to improving the image rendering effect of the target object in the left screen and the right screen.

[0065] See also Figure 5 , Figure 5 This is a schematic flow chart of an image rendering method for a smart wearable device provided in the fourth embodiment of the present application. The image rendering method for the smart wearable device can be applied to the smart wearable device, and provides a variety of stacking methods for users to determine a more suitable stacking method based on specific target objects and different smart wearable devices, thereby improving the generation quality and efficiency of stacked images.

[0066] based on Figure 1 The embodiment shown, this embodiment Figure 5 As shown, step S20 includes step S201.

[0067] Step S201, stacking the left rendering projection and the right rendering projection left and right, and using the image generated after stacking as the stacked image; or

[0068] The left rendering projection and the right rendering projection are stacked up and down, and an image generated after the stacking is used as the stacked image.

[0069] In one embodiment, a pixel is the smallest unit of image display, which is usually composed of sub-pixels of three colors: red, green, and blue. These three colors can produce a variety of colors through different combinations and proportions. For example, red and green add up to yellow, red and blue add up to purple, and all three colors add up to white. The image we see on the screen is actually composed of countless pixels arranged and combined. The clarity and fineness of the image displayed on the screen depends on the resolution and pixel density of the screen. Resolution refers to the number of pixels in the horizontal and vertical directions on the screen.

[0070] For the same screen, the smaller the screen pixel, the clearer the image details that can be displayed. For the same shape, when using screens with different pixel sizes to display, the pattern will appear more delicate and the viewing experience will be better on the screen with a small pixel size. It can be seen that the ideal way to improve the imaging effect is to stack the pixels of each row or column in the image in a regular manner. In combination with this embodiment, the stacking methods include left-right stacking, top-bottom stacking, and alternating stacking.

[0071] Stacking, also known as focus stacking, is an image processing method on a computer that combines multiple photos taken at different focuses or apertures. The combined frames will produce a sharp photo with a large depth of field, which will highlight all the details in the picture, including the usually blurred background.

[0072] In one embodiment, the left and right stacking takes the case where the pixels of the left rendering projection and the right rendering projection are both 640x480 as an example. When stacking, the first 640 rows of pixels in a row of pixels are the left rendering projection, and the last 640 rows of pixels are the right rendering projection. After stacking, the number of pixels of the stacked image generated is 1280x480.

[0073] In one embodiment, taking the case where the pixels of the left rendering projection and the right rendering projection are both 640x480 as an example, when stacking, the first 480 rows of pixels are the left rendering projection, and the last 480 rows of pixels are the right rendering projection. After stacking, the number of pixels of the stacked image generated is 640x960.

[0074] based on Figure 5 In the embodiment shown, in this embodiment, step S20 further includes:

[0075] The row pixels of the left rendering projection and the row pixels of the right rendering projection are alternately stacked, and an image generated after the stacking is used as the stacked image.

[0076] In one embodiment, row pixels are alternately stacked, taking the case where the pixels of the left rendering projection and the right rendering projection are both 640x480. When stacking, the first row of pixels is the left rendering projection, the second row of pixels is the right rendering projection, the third row of pixels is the left rendering projection, and so on. The number of pixels of the stacked image generated after stacking is 640x960.

[0077] based on Figure 1 In the embodiment shown, in this embodiment, step S30 includes:

[0078] Based on a preset stacking mode of the stacked images, performing image decomposition processing on the stacked images through the FPGA to generate the left parallax image and the right parallax image;

[0079] The preset stacking modes include up-and-down stacking, left-and-right stacking, and alternating stacking.

[0080] In one embodiment, the three stacking methods mentioned above are respectively up and down stacking, left and right stacking and alternating stacking. In this embodiment, the decomposition method of the stacked image can be determined according to the stacking method, that is, the decomposition process of this embodiment is the inverse process of the corresponding stacking process.

[0081] Based on any of the above embodiments, in this embodiment, MIPI includes a camera serial interface CSI or the DSI, and the CSI includes a transmission module; the stacked image is sent to a field programmable gate array FPGA through a mobile industry processor interface MIP I, including:

[0082] In the case where the MIPI is the CSI, a connection between the CSI and the FPGA is established through the transmission module, and the stacked image is sent to the FPGA through the transmission module;

[0083] In the case where the MIPI is the DSI, the stacked image is sent to the FPGA via the DSI.

[0084] In one embodiment, MIPI, as an interface standard, cannot be generally understood as a communication protocol. It includes an application layer, a protocol layer, and a physical layer. The MIPI in this embodiment can be replaced by the traditional DSI with a lower-cost CSI. Among them, CSI includes a receiving module and a transmission module. This embodiment applies the transmission module, and bridges CSI to FPGA through the transmission module to send the stacked image to FPGA through the channel.

[0085] This embodiment uses one MIPI interface, namely DSI in this embodiment or CSI with lower cost, to bridge to FPGA, and then the FPGA transmits the stacked image to the left screen and the right screen respectively through two DSIs, thereby reducing the strict requirements on the number of MIPI interfaces in the related technology. Therefore, this embodiment not only reduces the hardware requirements for image rendering of the target object in the smart wearable device, but also reduces the hardware cost.

[0086] See also Figure 6 , Figure 6 The embodiment of the present application provides a schematic block diagram of an image rendering device of a smart wearable device, wherein the image rendering device of the smart wearable device is used to execute the image rendering method of the smart wearable device described above. The image rendering device of the smart wearable device can be configured on a server.

[0087] like Figure 6 As shown, the image rendering device 400 of the smart wearable device includes:

[0088] A rendering projection acquisition module 410 is used to acquire a left rendering projection and a right rendering projection of a target object through the main control chip;

[0089] An image stacking module 420, configured to merge the left rendering projection and the right rendering projection to generate a stacked image, and send the stacked image from the main control chip to the FPGA via one M IPI;

[0090] The rendering module 430 is used to parse the stacked images through the FPGA to generate a left parallax image and a right parallax image, and push the left parallax image and the right parallax image to the left screen and the right screen through the two DSIs respectively, so as to realize the image rendering of the target object on the left screen and the right screen.

[0091] Furthermore, the rendering projection acquisition module 410 includes:

[0092] The parallax image construction unit is used to construct left and right parallax images according to the 3D information of the target object through the main control chip, and generate the left rendering projection and the right rendering projection corresponding to the 3D information.

[0093] Furthermore, the parallax image construction unit includes:

[0094] A conversion matrix acquisition subunit, used to acquire a first conversion matrix, a second conversion matrix and a third conversion matrix;

[0095] The first conversion matrix is ​​a conversion matrix for converting the target object from the target object coordinate system to the world coordinate system, the second conversion matrix is ​​a conversion matrix for converting the target object from the world coordinate system to the smart wearable device coordinate system, and the third conversion matrix is ​​a conversion matrix for projecting the 3D information to the 2D image coordinate system;

[0096] The rendering projection generating subunit is used to generate the left rendering projection and the right rendering projection based on a preset rendering algorithm, the 3D information, the first conversion matrix, the second conversion matrix and the third conversion matrix.

[0097] Furthermore, the image stacking module 420 includes:

[0098] The left-right stacking unit is used to stack the left-side rendering projection and the right-side rendering projection left-right, and use the image generated after stacking as the stacked image.

[0099] The up-and-down stacking unit is used to stack the left-side rendering projection and the right-side rendering projection up-and-down, and use the image generated after stacking as the stacked image.

[0100] The alternating stacking unit is used to alternately stack the row pixels of the left rendering projection and the row pixels of the right rendering projection, and use the image generated after stacking as the stacked image.

[0101] Furthermore, the rendering module 430 includes:

[0102] A stacked image decomposition unit, configured to perform image decomposition processing on the stacked image through the FPGA based on a preset stacking mode of the stacked image, so as to generate the left parallax image and the right parallax image;

[0103] The preset stacking modes include up-and-down stacking, left-and-right stacking, and alternating stacking.

[0104] Furthermore, the image stacking module 420 includes:

[0105] a CS I unit, configured to establish a connection between the CS I and the FPGA through the transmission module when the MIP I is the CS I, and to send the stacked image to the FPGA through the transmission module;

[0106] A DS I unit is used for sending the stacked image to the FPGA via the DS I when the MIP I is the DS I.

[0107] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0108] The above-mentioned device can be implemented in the form of a computer program. Figure 7 Runs on the computer device shown.

[0109] See also Figure 7 , Figure 7 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0110] See also Figure 7 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0111] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image rendering method of the smart wearable device.

[0112] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0113] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any image rendering method of the smart wearable device.

[0114] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0116] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0117] Acquire the left rendering projection and the right rendering projection of the target object through the main control chip;

[0118] Merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI;

[0119] The stacked images are parsed by the FPGA to generate a left parallax image and a right parallax image, and the left parallax image and the right parallax image are pushed to the left screen and the right screen respectively through the two DSIs to achieve image rendering of the target object on the left screen and the right screen.

[0120] In one embodiment, the left rendering projection and the right rendering projection of the target object are obtained by the main control chip to achieve:

[0121] The main control chip constructs left and right parallax images according to the 3D information of the target object, and generates the left rendering projection and the right rendering projection corresponding to the 3D information.

[0122] In one embodiment, the main control chip constructs left and right parallax images according to the 3D information of the target object, and generates the left rendering projection and the right rendering projection corresponding to the 3D information, so as to achieve:

[0123] Obtain a first conversion matrix, a second conversion matrix, and a third conversion matrix;

[0124] The first conversion matrix is ​​a conversion matrix for converting the target object from the target object coordinate system to the world coordinate system, the second conversion matrix is ​​a conversion matrix for converting the target object from the world coordinate system to the smart wearable device coordinate system, and the third conversion matrix is ​​a conversion matrix for projecting the 3D information to the 2D image coordinate system;

[0125] The left rendering projection and the right rendering projection are generated based on a preset rendering algorithm, the 3D information, the first conversion matrix, the second conversion matrix, and the third conversion matrix.

[0126] In one embodiment, the left rendering projection and the right rendering projection are merged to generate a stacked image, which is used to achieve:

[0127] stacking the left rendering projection and the right rendering projection left and right, and using the image generated after stacking as the stacked image; or

[0128] The left rendering projection and the right rendering projection are stacked up and down, and an image generated after the stacking is used as the stacked image.

[0129] In one embodiment, the left rendering projection and the right rendering projection are merged to generate a stacked image, which is used to achieve:

[0130] The row pixels of the left rendering projection and the row pixels of the right rendering projection are alternately stacked, and an image generated after the stacking is used as the stacked image.

[0131] In one embodiment, the stacked images are parsed by the FPGA to generate a left disparity image and a right disparity image, so as to achieve:

[0132] Based on a preset stacking mode of the stacked images, performing image decomposition processing on the stacked images through the FPGA to generate the left parallax image and the right parallax image;

[0133] The preset stacking modes include up-and-down stacking, left-and-right stacking, and alternating stacking.

[0134] In one embodiment, the M IPI includes a camera serial interface CSI or the DSI, and the CSI includes a transmission module; the stacked image is sent from the main control chip to the FPGA through the MIP I to achieve:

[0135] In the case where the MIPI is the CSI, a connection between the CSI and the FPGA is established through the transmission module, and the stacked image is sent to the FPGA through the transmission module;

[0136] In the case where the MIPI is the DSI, the stacked image is sent to the FPGA via the DSI.

[0137] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement an image rendering method for any smart wearable device provided in an embodiment of the present application.

[0138] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0139] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An image rendering method for a smart wearable device, characterized in that: The smart wearable device includes a main control chip, a mobile industry processor interface MIPI, a field programmable logic gate array FPGA, a display module serial interface DSI, a left screen and a right screen. The image rendering method of the smart wearable device includes: Acquire the left rendering projection and the right rendering projection of the target object through the main control chip; Merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI; The stacked images are parsed by the FPGA to generate a left parallax image and a right parallax image, and the left parallax image and the right parallax image are pushed to the left screen and the right screen respectively through the two DSIs to achieve image rendering of the target object on the left screen and the right screen.

2. The image rendering method of the smart wearable device according to claim 1, characterized in that: The obtaining the left rendering projection and the right rendering projection of the target object through the main control chip includes: The main control chip constructs left and right parallax images according to the 3D information of the target object, and generates the left rendering projection and the right rendering projection corresponding to the 3D information.

3. The image rendering method of the smart wearable device according to claim 2, characterized in that: The constructing left and right parallax images according to the 3D information of the target object by the main control chip to generate the left rendering projection and the right rendering projection corresponding to the 3D information includes: Obtain a first conversion matrix, a second conversion matrix, and a third conversion matrix; The first conversion matrix is ​​a conversion matrix for converting the target object from the target object coordinate system to the world coordinate system, the second conversion matrix is ​​a conversion matrix for converting the target object from the world coordinate system to the smart wearable device coordinate system, and the third conversion matrix is ​​a conversion matrix for projecting the 3D information to the 2D image coordinate system; The left rendering projection and the right rendering projection are generated based on a preset rendering algorithm, the 3D information, the first conversion matrix, the second conversion matrix, and the third conversion matrix.

4. The image rendering method of the smart wearable device according to claim 1, characterized in that: The step of merging the left rendering projection and the right rendering projection to generate a stacked image includes: stacking the left rendering projection and the right rendering projection left and right, and using the image generated after stacking as the stacked image; or The left rendering projection and the right rendering projection are stacked up and down, and an image generated after the stacking is used as the stacked image.

5. The image rendering method of the smart wearable device according to claim 1, characterized in that: The step of merging the left rendering projection and the right rendering projection to generate a stacked image includes: The row pixels of the left rendering projection and the row pixels of the right rendering projection are alternately stacked, and an image generated after the stacking is used as the stacked image.

6. The image rendering method of the smart wearable device according to claim 1, characterized in that: The step of parsing the stacked images by the FPGA to generate a left disparity image and a right disparity image includes: Based on a preset stacking mode of the stacked images, performing image decomposition processing on the stacked images through the FPGA to generate the left parallax image and the right parallax image; The preset stacking modes include up-and-down stacking, left-and-right stacking, and alternating stacking.

7. The image rendering method of the smart wearable device according to any one of claims 1 to 6, characterized in that: The MIPI includes a camera serial interface CSI or the DSI, and the CSI includes a transmission module; the stacked image is sent from the main control chip to the FPGA through the MIPI, including: In the case where the MIPI is the CSI, a connection between the CSI and the FPGA is established through the transmission module, and the stacked image is sent to the FPGA through the transmission module; In the case where the MIPI is the DSI, the stacked image is sent to the FPGA via the DSI.

8. An image rendering device for a smart wearable device, characterized in that: include: A rendering projection acquisition module, used to acquire a left rendering projection and a right rendering projection of a target object through the main control chip; An image stacking module, used for merging the left rendering projection and the right rendering projection to generate a stacked image, and sending the stacked image from the main control chip to the FPGA through one MIPI; A rendering module is used to parse the stacked images through the FPGA to generate a left parallax image and a right parallax image, and push the left parallax image and the right parallax image to the left screen and the right screen through the two DSIs respectively, so as to realize the image rendering of the target object on the left screen and the right screen.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the image rendering method of the smart wearable device according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image rendering method of the smart wearable device according to any one of claims 1 to 7.

Images