Weston-based image processing method, system and equipment and medium

Abstract

The invention discloses an image processing method, system and device based on weston, and a medium. The method comprises the following steps: respectively drawing instrument display layers of a plurality of application programs to surface windows corresponding to a wayland client through OpenG1; the method comprises the following steps of: transmitting a handle of a graphic buffer area in each surface window to a western component; synthesizing the plurality of surface windows to obtain an image synthesis window; when the image synthesis window satisfies a distortion correction condition, adjusting vertex coordinates and texture coordinates of the instrument display image model to obtain an instrument display image after distortion correction; and rendering the instrument display image after distortion correction, and transmitting the instrument display image to an instrument display for display. According to the method, after image synthesis is completed in the weston, unified distortion correction processing and rendering are carried out on the synthesized image, so that the calculation overhead is reduced, and the real-time performance is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a weston-based image processing method, system, device and medium. Background Art

[0002] Distortion correction technology is a method used to correct images or videos on special-shaped display screens to solve the distortion problem that occurs in unconventional-shaped display areas. It is widely used in fields such as automotive HUD head-up displays.

[0003] Traditionally, distortion correction is usually handled at the application level, which increases the complexity of the application at the application level and increases the difficulty of development and maintenance. If the display of the entire system needs to be distorted, then each application needs to perform the corresponding correction operation, which increases the workload and complexity, may cause compatibility and consistency issues between applications, and generate a large amount of repetitive redundant code. In addition, the additional distortion correction algorithm at the application level will occupy a large amount of computing resources, resulting in a decrease in system performance. Especially in application scenarios with high real-time requirements, this additional computing overhead may cause delays or freezes.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide an image processing method, system, device and medium based on weston, aiming to solve the technical problem of how to achieve high efficiency and real-time performance of image processing.

[0006] To achieve the above object, the present invention provides an image processing method based on Weston, and the image processing method based on Weston includes:

[0007] Use OpenGl to draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client;

[0008] Pass the handle of the graphics buffer in each surface window to the weston compositor respectively;

[0009] Based on the weston compositor, multiple surface windows are synthesized to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model;

[0010] When the image synthesis window satisfies the distortion correction condition, adjusting the vertex coordinates and texture coordinates of the instrument display image model to obtain the instrument display image after distortion correction;

[0011] The instrument display image after the distortion correction is rendered, and the rendered instrument display image is transmitted to the instrument display for display.

[0012] Optionally, before the step of drawing the vehicle display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl, the method further includes:

[0013] Establish a connection between the Wayland client and the Weston compositor through the Wayland protocol;

[0014] When the connection is successful, a surface window corresponding to the wayland client is created.

[0015] Optionally, the step of respectively passing the handles of the graphics buffers in each surface window to westoncompositor includes:

[0016] Determine the graphics buffer corresponding to each surface window respectively;

[0017] The handles of each graphics buffer are respectively passed to the weston compositor through the wayland protocol.

[0018] Optionally, after the step of synthesizing the multiple surface windows based on the weston compositor to obtain the image synthesis window, the step further includes:

[0019] Determine the instrument type for the instrument display;

[0020] When the instrument type is HUD, it is determined that the image synthesis window satisfies a distortion correction condition.

[0021] Optionally, after the step of determining the instrument type of the instrument display, the method further includes:

[0022] When the instrument type is MET, it is determined that the image synthesis window does not meet the distortion correction condition;

[0023] The instrument display image model is rendered, and the rendered instrument display image is transmitted to the instrument display for display.

[0024] Optionally, the step of adjusting the vertex coordinates and texture coordinates of the instrument display image model includes:

[0025] Determining a model distortion parameter according to the instrument display image model;

[0026] The vertex coordinates of the instrument display image model are transformed according to the model distortion parameters, and the mapping mode of the texture coordinates corresponding to the transformed vertex coordinates is adjusted.

[0027] In addition, to achieve the above-mentioned purpose, the present invention also proposes an image processing system based on weston, and the image processing system based on weston includes:

[0028] The drawing module is used to draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl;

[0029] The transfer module is used to transfer the handle of the graphics buffer in each surface window to westoncompositor respectively;

[0030] A synthesis module, used for synthesizing multiple surface windows based on the weston compositor to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model;

[0031] A distortion correction module, used for adjusting the vertex coordinates and texture coordinates of the instrument display image model to obtain a distortion-corrected instrument display image when the image synthesis window meets the distortion correction condition;

[0032] The rendering module is used to render the instrument display image after the distortion correction, and transmit the rendered instrument display image to the instrument display for display.

[0033] In addition, to achieve the above-mentioned purpose, the present invention also proposes a weston-based image processing device, which includes: a memory, a processor, and a weston-based image processing program stored in the memory and executable on the processor, wherein the weston-based image processing program is configured to implement the steps of the weston-based image processing method described above.

[0034] In addition, to achieve the above-mentioned purpose, the present invention further proposes a medium on which a weston-based image processing program is stored. When the weston-based image processing program is executed by a processor, the steps of the weston-based image processing method described above are implemented.

[0035] The present invention firstly draws the instrument display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl, and then passes the handles of the graphic buffers in each surface window to westoncompositor respectively, and then synthesizes the multiple surface windows based on weston compositor to obtain an image synthesis window, which includes an instrument display image model. When the image synthesis window meets the distortion correction condition, the vertex coordinates and texture coordinates of the instrument display image model are adjusted to obtain the instrument display image after distortion correction, and finally the instrument display image after distortion correction is rendered, and the rendered instrument display image is transmitted to the instrument display for display. After completing the image synthesis inside weston, the present invention performs a unified distortion correction process on the synthesized image to complete the distortion correction and rendering, thereby providing higher efficiency and better real-time performance. At the same time, it can also simplify the development and maintenance of application programs, reduce repeated correction operations, and improve the consistency and compatibility of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural diagram of a weston-based image processing device in a hardware operating environment involved in an embodiment of the present invention;

[0037] Figure 2 It is a flowchart of the first embodiment of the image processing method based on weston of the present invention;

[0038] Figure 3 This is a diagram showing the principle of distortion correction according to the first embodiment of the image processing method based on Weston of the present invention;

[0039] Figure 4 It is a structural block diagram of the first embodiment of the image processing system based on weston of the present invention.

[0040] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0042] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a weston-based image processing device in the hardware operating environment involved in the embodiment of the present invention.

[0043] like Figure 1As shown, the weston-based image processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RandomAccess Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0044] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the image processing device based on Weston, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0045] like Figure 1 As shown, the memory 1005 as a medium may include an operating system, a network communication module, a user interface module, and a weston-based image processing program.

[0046] exist Figure 1 In the weston-based image processing device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the weston-based image processing device of the present invention can be set in the weston-based image processing device, and the weston-based image processing device calls the weston-based image processing program stored in the memory 1005 through the processor 1001, and executes the weston-based image processing method provided by the embodiment of the present invention.

[0047] The embodiment of the present invention provides an image processing method based on weston, referring to Figure 2 , Figure 2 It is a flowchart of the first embodiment of the image processing method based on Weston of the present invention.

[0048] In this embodiment, the image processing method based on weston includes the following steps:

[0049] Step S10: Draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl.

[0050] It is easy to understand that the execution subject of this embodiment can be a weston-based image processing system with functions such as data processing, network communication and program running, or other computer devices with similar functions, etc., and this embodiment is not limited.

[0051] It should be noted that the multiple applications may be 3DAPPs, 2DAPPs, etc.

[0052] It should be understood that the wayland client needs to be connected to the weston compositor through the wayland protocol. When the connection is successful, weston creates a surface window corresponding to the wayland client. The surface window includes a window for a 2D layer and a window for a 3D layer.

[0053] Each surface window includes multiple instrument display layers, and the instrument display layer can be understood as the layer corresponding to the image that needs to be displayed on the instrument.

[0054] In an embodiment, the 3D corresponding instrument display layer is drawn based on the 3DAPP through OpenGL on the surface corresponding to the 3DAPP (ie, the wayland client), and the 2D corresponding instrument display layer is drawn based on the 2DAPP through OpenGL on the surface corresponding to the 2DAPP (ie, the wayland client).

[0055] Step S20: passing the handles of the graphics buffers in each surface window to the weston compositor respectively.

[0056] Furthermore, the graphics buffer corresponding to each surface window is determined respectively; and the handle of each graphics buffer is transmitted to the weston compositor respectively through the wayland protocol.

[0057] Step S30: synthesizing multiple surface windows based on the weston compositor to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model.

[0058] The instrument display image model is an image model to be displayed on the instrument display.

[0059] In a specific implementation, weston (i.e., weston compositor) synthesizes all application windows (i.e., surface windows) that need to be displayed on the same display to create a new window (i.e., image synthesis window).

[0060] Step S40: When the image synthesis window satisfies the distortion correction condition, the vertex coordinates and texture coordinates of the instrument display image model are adjusted to obtain the instrument display image after distortion correction.

[0061] The distortion correction condition is that the instrument display is a HUD.

[0062] It should be understood that if a display needs to display the layers of multiple applications at the same time, each application needs to be corrected, which will increase the performance consumption during correction. However, after the correction is implemented in weston, even if multiple applications need correction, weston will synthesize the windows of these applications and then perform correction processing uniformly, thus eliminating the problem of each application needing to perform correction operations and simplifying the workload of development and maintenance.

[0063] In this embodiment, weston will determine whether the display screen needs to be corrected for distortion based on actual needs, and then determine the instrument type of the instrument display. When the instrument type is HUD, it is determined that the image synthesis window meets the distortion correction conditions. Then, the model distortion parameters are determined according to the instrument display image model, and the vertex coordinates of the instrument display image model are transformed according to the model distortion parameters, and the mapping method of the texture coordinates corresponding to the transformed vertex coordinates is adjusted.

[0064] It should also be noted that implementing distortion correction in weston can reduce additional computational overhead at the application layer, improve real-time performance, and avoid delays or freezes. Distortion correction is implemented during image rendering by adjusting the vertex positions and texture coordinates of the surface. This ensures that images appear more accurate and natural in display areas with unconventional shapes, providing better image quality.

[0065] The model distortion parameter is a position parameter where distortion exists in the instrument display image model.

[0066] In the specific implementation, refer to Figure 3 , Figure 3This is a distortion correction principle diagram of the first embodiment of the image processing method based on Weston of the present invention. Weston transforms the rectangular vertex positions of the image according to specific distortion models and parameters, adjusts the mapping method of texture coordinates, binds vertex positions and texture coordinates to shader attributes, and achieves distortion correction effects.

[0067] It should also be noted that when the instrument type is MET, it is determined that the image synthesis window does not meet the distortion correction conditions; the instrument display image model is rendered, and the rendered instrument display image is transmitted to the instrument display (ie, MET display) for display.

[0068] Step S50: Rendering the instrument display image after distortion correction, and transmitting the rendered instrument display image to the instrument display for display.

[0069] In a specific implementation, the instrument display image model or the distortion-corrected instrument display image is rendered by weston and finally output to the instrument display (ie, MET display or HUD display).

[0070] It should also be noted that weston is a universal graphics stack that is suitable for a variety of embedded and desktop systems. Whether in embedded devices, virtual reality devices or desktop displays, the distortion correction method can provide consistent results, enhancing cross-platform applicability.

[0071] In this embodiment, the instrument display layers of multiple applications are first drawn to the surface windows corresponding to the wayland client through OpenGl, and then the handles of the graphics buffers in each surface window are respectively passed to the weston compositor, and then the multiple surface windows are synthesized based on the weston compositor to obtain an image synthesis window, which includes an instrument display image model. When the image synthesis window meets the distortion correction condition, the vertex coordinates and texture coordinates of the instrument display image model are adjusted to obtain the instrument display image after distortion correction, and finally the instrument display image after distortion correction is rendered, and the rendered instrument display image is transmitted to the instrument display for display. After completing the image synthesis inside weston, this embodiment performs a unified distortion correction process on the synthesized image to complete the distortion correction and rendering, thereby providing higher efficiency and better real-time performance. At the same time, it can also simplify the development and maintenance of application programs, reduce repeated correction operations, and improve the consistency and compatibility of the entire system.

[0072] Reference Figure 4 , Figure 4 It is a structural block diagram of the first embodiment of the image processing system based on weston of the present invention.

[0073] like Figure 4 As shown, the image processing system based on weston proposed in an embodiment of the present invention includes:

[0074] The drawing module 4001 is used to draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl;

[0075] The transfer module 4002 is used to transfer the handle of the graphics buffer in each surface window to westoncompositor respectively;

[0076] A synthesis module 4003, used for synthesizing multiple surface windows based on the weston compositor to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model;

[0077] The distortion correction module 4004 is used to adjust the vertex coordinates and texture coordinates of the instrument display image model to obtain the instrument display image after distortion correction when the image synthesis window meets the distortion correction condition;

[0078] The rendering module 4005 is used to render the instrument display image after the distortion correction, and transmit the rendered instrument display image to the instrument display for display.

[0079] Other embodiments or specific implementations of the image processing system based on weston of the present invention can refer to the above-mentioned method embodiments, which will not be described in detail here.

[0080] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0081] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0083] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A weston-based image processing method, characterized in that: The image processing method based on weston comprises the following steps: Use OpenGl to draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client; Pass the handle of the graphics buffer in each surface window to the weston compositor respectively; Based on the weston compositor, multiple surface windows are synthesized to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model; When the image synthesis window satisfies the distortion correction condition, adjusting the vertex coordinates and texture coordinates of the instrument display image model to obtain the instrument display image after distortion correction; The instrument display image after the distortion correction is rendered, and the rendered instrument display image is transmitted to the instrument display for display.

2. The method according to claim 1, characterized in that Before the step of drawing the vehicle display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl, the method further includes: Establish a connection between the Wayland client and the Weston compositor through the Wayland protocol; When the connection is successful, a surface window corresponding to the wayland client is created.

3. The method according to claim 2, characterized in that The step of respectively passing the handles of the graphics buffers in each surface window to the weston compositor includes: Determine the graphics buffer corresponding to each surface window respectively; The handles of each graphics buffer are respectively passed to the weston compositor through the wayland protocol.

4. The method according to claim 3, after the step of synthesizing a plurality of surface windows based on the weston compositor to obtain an image synthesis window, further comprising: Determine the instrument type for the instrument display; When the instrument type is HUD, it is determined that the image synthesis window satisfies a distortion correction condition.

5. The method according to claim 4, characterized in that After the step of determining the instrument type of the instrument display, the method further includes: When the instrument type is MET, it is determined that the image synthesis window does not meet the distortion correction condition; The instrument display image model is rendered, and the rendered instrument display image is transmitted to the instrument display for display.

6. The method according to claim 4, characterized in that The step of adjusting the vertex coordinates and texture coordinates of the instrument display image model comprises: Determining a model distortion parameter according to the instrument display image model; The vertex coordinates of the instrument display image model are transformed according to the model distortion parameters, and the mapping mode of the texture coordinates corresponding to the transformed vertex coordinates is adjusted.

7. An image processing system based on weston, characterized in that: The weston-based image processing system comprises: The drawing module is used to draw the instrument display layers of multiple applications to the surface windows corresponding to the wayland client through OpenGl; The transfer module is used to transfer the handle of the graphics buffer in each surface window to westoncompositor respectively; A synthesis module, used for synthesizing multiple surface windows based on the weston compositor to obtain an image synthesis window, wherein the image synthesis window includes an instrument display image model; A distortion correction module, used for adjusting the vertex coordinates and texture coordinates of the instrument display image model to obtain a distortion-corrected instrument display image when the image synthesis window meets the distortion correction condition; The rendering module is used to render the instrument display image after the distortion correction, and transmit the rendered instrument display image to the instrument display for display.

8. An image processing device based on weston, characterized in that The device comprises: a memory, a processor, and a weston-based image processing program stored in the memory and executable on the processor, wherein the weston-based image processing program is configured to implement the steps of the weston-based image processing method according to any one of claims 1 to 6.

9. A medium, characterized in that The medium stores a weston-based image processing program, and when the weston-based image processing program is executed by a processor, the steps of the weston-based image processing method according to any one of claims 1 to 6 are implemented.

Images