GLStudio-based instrument symbol and dynamic background picture superposition method and system

Abstract

The invention discloses a GLStudio-based instrument symbol and dynamic background picture superposition method and system. The method comprises the following steps: S1, continuously monitoring an instrument application process; s2, when a dynamic background picture updating request is monitored, performing screenshot on a background image window of a background application to obtain a background image; s3, replacing the background image to a background image layer preset below the symbol image layer, and superposing and rendering the symbol image layer and the replaced background image layer to form a display picture of the virtual instrument; s4, loading and displaying a display picture of the virtual instrument; meanwhile, the invention further provides a corresponding system. According to the invention, the problems of insufficient integration of the symbol layer and the background layer, poor symbol display quality and the like when the display symbol and the background dynamic texture are superposed and fused in an instrument in the prior art are solved, and the background dynamic picture screenshot is replaced to the background layer below the symbol layer picture; the image display is not shielded by other windows, and the original shape and definition of the symbols can be perfectly maintained.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics processing, and in particular to a method and system for superimposing instrument symbols and dynamic background images based on GLStudio. Background Art

[0002] Instruments are common and important equipment, and are widely used in industrial production, detection, and control. Virtual instruments play an indispensable and key role in the simulation industry, especially in some image display and analysis applications, such as the flight simulation industry, because of their advantages such as intuitive display, flexible functional styles, high system integration, and strong controllability.

[0003] In virtual instruments, not only basic symbols such as characters and shapes need to be displayed, but also two-dimensional maps, infrared images, meteorological cloud maps and other images. Therefore, it involves the superposition display of the upper symbol layer and the lower background layer. As the scene position and viewing angle change, there is usually a demand for dynamic display of the background layer. In many common instrument development tools, the window for superimposing the symbol layer and the background layer does not support multi-layer background textures and dynamic background textures. Further design and development are needed to adaptively fuse the symbol layer window and the dynamic background texture window to achieve the effect of superimposing the instrument symbol and the dynamic background image. The common fusion superposition method is to open two application windows of the same size in the system to render and display symbols and dynamic background images respectively, and to make the background transparent by eliminating pixels on the window displaying the symbol, and move the processed symbol window to the upper layer of the background dynamic image. The two windows are dynamically displayed and superimposed to obtain the final dynamic instrument interface. Because the windows are displayed completely independently, other windows may be inserted between the two windows to block the background window image; the background transparency of the symbol window is achieved by deleting the pixels of the specified color, which will cause jagged edges, blur, and partial background residue on the symbol edge, which seriously affects the image display quality. Chinese patent publication number CN113744174A, publication date December 3, 2021, entitled "A method for layered fusion of comprehensive visual images" discloses a method for fusing and superimposing symbols with background images, including drawing the virtual map image and the symbol image in blocks, extracting the virtual map image and the symbol image; fusing the virtual map image with the video image to obtain a fused image, pixel-segmenting the symbol image, removing its background area and superimposing it on the fused image to obtain a layered fused comprehensive visual image. However, this method still has problems such as symbol jaggedness and blurring due to the pixel segmentation method of the symbol layer. Summary of the invention

[0004] The present invention overcomes the problems of insufficient fusion of symbol layers and background layers, poor symbol display quality, etc. in the prior art when symbols and background dynamic textures are superimposed and fused to display in instruments, and provides a method and system for superimposing instrument symbols and dynamic background images based on GLStudio. The method loads a background dynamic image and a symbol layer image respectively through two applications, and replaces a screenshot of the background dynamic image with the background layer under the symbol layer image. There is no need to make the symbol display window of the instrument application transparent. The symbol layer and the background layer are superimposed and displayed in real time in an application window without being blocked by other windows, and the symbol can perfectly maintain its original shape and clarity.

[0005] In order to achieve the above object, the present invention adopts the following scheme: The method for superimposing instrument symbols and dynamic background images based on GLStudio includes the following steps: S1: continuously monitoring the instrument application process, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of a virtual instrument; S2. When a dynamic background image update request is detected in the instrument application process, a background image window rendered by the background application is captured to obtain a background image, wherein the background application is used to generate a dynamic texture image of the virtual instrument display background; S3: replacing the obtained background image with a background layer of the instrument application, the background layer being preset in a lower layer of the symbol layer, and overlaying and rendering the symbol layer and the replaced background layer to form a display screen of the virtual instrument; S4: Loading the display screen of the virtual instrument.

[0006] Preferably, in step S1, the instrument application process is monitored based on the hook technology, and when the process attempts to call the wglSwapBuffers function to exchange the rendering buffer and update the display content, a screenshot is taken of the background image window generated by the background application to obtain the background image and save it in the buffer.

[0007] Preferably, the program for monitoring the instrument application process is encapsulated into a plug-in module, and the plug-in module code is injected into the instrument application process through the LoadLibrary function in the instrument application initialization code.

[0008] Preferably, in step S3, the background image is replaced with the background layer by obtaining the static texture ID information of the background layer in the instrument program, and using the glTexSubImage2D function to replace the static texture of the background layer with the background image data obtained by the screenshot according to the obtained static texture ID information.

[0009] Preferably, in step S2, the method of taking a screenshot of the background image window in the background application to obtain the background image is: calling the glReadPixels interface to read the pixel information in the background application to obtain the background screenshot, and performing preprocessing operations on the background screenshot, the preprocessing operations including window cropping, resolution matching and size transformation.

[0010] This application also provides an instrument symbol and dynamic background image overlay system based on GLStudio, which includes: A monitoring module, which is used to continuously monitor the process of the instrument application, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of the virtual instrument; A background capture module, which is used to capture a background image window rendered by a background application to obtain a background image when a dynamic background image update request is detected in the instrument application process. The background application is used to generate a dynamic texture image of the virtual instrument display background; A background replacement module, which is used to replace the obtained background image with the background layer of the instrument application, and to overlay and render the symbol layer and the replaced background layer to form a display screen of the virtual instrument, wherein the background layer is preset at the lower layer of the symbol layer obtained by rendering the instrument application; The display module is used to load and display the display screen of the virtual instrument.

[0011] The present invention also provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method described in any one of the above schemes.

[0012] The present invention also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above schemes is implemented.

[0013] The present invention includes at least the following beneficial effects: (1) by capturing the dynamic background image in the background application in real time to replace the static texture in the instrument application, and superimposing and displaying them in an instrument application window, the dynamic superposition effect of the instrument symbol and the background dynamic image is more natural and delicate, and can avoid the occlusion of the background image by other windows; (2) there is no need to crop the area around the symbol in the window, thereby ensuring the accuracy and clarity of the real-time change of the symbol; (3) the method of replacing the background layer is faster and less error-prone than using the application window transparency operation, avoiding the problem of incompatibility of some instrument development systems with window transparency processing programs, and having no requirements for the size, position, and display level of the background application window during operation, thereby reducing maintenance costs and improving usability; (4) by using the plug-in module injection technology to inject the operating program into the instrument application process, without affecting the normal operation of the application source code, the problem of the application's high encapsulation of the instrument rendering code is avoided, so that the instrument application can smoothly perform the layer dynamic superposition operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A principle flow chart of the present invention; DETAILED DESCRIPTION

[0015] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0016] like Figure 1 As shown, the method for superimposing instrument symbols and dynamic background images based on GLStudio provided by the present invention includes the following steps: S1: continuously monitoring the instrument application process, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of a virtual instrument; S2. When a dynamic background image update request is detected in the instrument application process, a background image window rendered by the background application is captured to obtain a background image. The background application is used to generate a dynamic texture image for the virtual instrument display background.

[0017] In the working state, the window of the instrument application will be displayed on the top layer of the instrument screen, and the functional symbol identifier of its symbol layer is used to display the current position, data value, scale ratio and other information. Since the background application in this application is only used to generate and load dynamic background images for the screenshot tool to capture image information, the background application only needs to run normally in the background. The background layer of the instrument application is added to the lower layer of the symbol layer. The content of the background layer is a set static texture. Common instrument development software does not support the mode in which both the symbol layer and the background layer change dynamically. Therefore, this method replaces the static texture of the background layer with a screenshot of the current dynamic image in real time, realizing the dynamic effect of the synchronous change of symbols and background textures. The dynamic background image can optionally include two-dimensional maps, infrared images, meteorological cloud maps and other types according to the actual scene.

[0018] S3: Replace the obtained background image with the background layer of the instrument application, the background layer is preset in the lower layer of the symbol layer, and the symbol layer and the replaced background layer are superimposed and rendered to form a display screen of the virtual instrument; when the instrument application is opened and the real-time dynamic picture display function is enabled, a dynamic picture update request appears in the instrument application process, and the way to respond to this update request is to replace the static texture of the background layer with the image in the buffer, and the image in the buffer is obtained by real-time capture and processing of the dynamic background animation of the background application by the screenshot tool.

[0019] S4: Load the display screen of the virtual instrument. The common display frame rate in virtual instruments is not too high. As an example, the current display frame rate of the virtual instrument is set to 30 Hz. The existing processor can fully adapt to the iterative image capture, processing and replacement at this rate, and the GPU can also be used for accelerated processing. The display delay caused by the image capture, processing and replacement process can be stably controlled below 10ms, which is almost imperceptible from the user's perspective. Because the image replacement rate is consistent with the display frame rate of the instrument, the effect on the continuity of the picture is the same as directly playing the original dynamic picture.

[0020] By capturing the dynamic background image in the background application in real time to replace the static texture in the instrument application, and overlaying them in an instrument application window, the dynamic overlay effect of the instrument symbol and the background dynamic image is more natural and delicate, and can avoid the obstruction of the background image by other windows; the symbol is directly displayed on the upper layer of the background texture, and there is no need to use complex algorithms to crop the area around the symbol in the window, ensuring the accuracy and clarity of the real-time symbol changes; the method of replacing the background layer is faster than the window transparency operation and will not cause the problem of pixel error deletion, avoiding the problem of incompatibility of some instrument development systems with window transparency handlers. There are no requirements for the size, position, and display level of the background application window during operation, which reduces maintenance costs and has better usability.

[0021] In another technical solution, in step S1, the instrument application process is monitored based on the hook technology. When the process attempts to call the wglSwapBuffers function to swap the rendering buffer and update the display content, a screenshot is taken of the background image window generated by the background application to obtain the background image and save it in the buffer. The instrument application is provided with a double buffer. When updating the display content, it is necessary to call the hwglSwapBuffers function in OpenGL to update the content in the double buffer to the display interface. The hook injection technology intercepts and modifies the program behavior by inserting a custom code snippet in the execution path of the image update program. After intercepting the program, the content in the double buffer to be used is replaced with the background image obtained by the interception process, so the background image will be displayed as the replaced background texture image.

[0022] Encapsulate the program that monitors the instrument application process into a plug-in module, and inject the plug-in module code into the instrument application process through the LoadLibrary function in the instrument application initialization code. By using the plug-in module injection technology to inject the operating program into the instrument application process, without affecting the normal operation of the application source code, the problem of the application's high encapsulation of the instrument rendering code is avoided, so that the instrument application can smoothly perform dynamic layer overlay operations. In the plug-in module dll file, you can customize the image processing method and algorithm to meet the needs of more application scenarios and improve scalability.

[0023] In another technical solution, in step S3, the background image is replaced with the background layer by obtaining the static texture ID information of the background layer in the instrument program, and using the glTexSubImage2D function to replace the static texture of the background layer with the background image data obtained by the screenshot according to the obtained static texture ID information. Figure 1 As shown, obtaining static texture ID information and replacing the static texture of the background layer with background image data obtained by screenshot processing can also be completed by the plug-in module.

[0024] In another technical solution, the method of obtaining the background image by taking a screenshot of the background image window in the background application in step S2 is: calling the glReadPixels interface to read the pixel information in the background application to obtain the background screenshot, and performing preprocessing operations on the background screenshot, the preprocessing operations including window cropping, resolution matching and size conversion. Because the dynamic background image has multiple sources and has different resolutions, sizes and other characteristics, it is necessary to perform preprocessing to make it meet the display requirements of the instrument application window and better cooperate with the symbol layer information.

[0025] This application also provides an instrument symbol and dynamic background image overlay system based on GLStudio, including: A monitoring module, which is used to continuously monitor the process of the instrument application, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of the virtual instrument; A background capture module, which is used to capture a background image window rendered by a background application to obtain a background image when a dynamic background image update request is detected in the instrument application process. The background application is used to generate a dynamic texture image of the virtual instrument display background; A background replacement module, which is used to replace the obtained background image with the background layer of the instrument application, and to overlay and render the symbol layer and the replaced background layer to form a display screen of the virtual instrument, wherein the background layer is preset at the lower layer of the symbol layer obtained by rendering the instrument application; The display module is used to load and display the display screen of the virtual instrument.

[0026] By loading and using this system, instruments and equipment can complete the dynamic superposition display of instrument symbols and background images, and achieve the same or similar technical benefits as the method provided in this application.

[0027] The present application also provides an electronic device, comprising: at least one processor, and a memory in communication with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs a method as described in any one of the above schemes. The electronic device may be any terminal device including a mobile phone, a laptop computer, a desktop computer, a tablet computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a vehicle-mounted computer, etc.

[0028] The present application also provides a storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, a method as described in any one of the above schemes is implemented.

[0029] Through the description of the above implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present invention, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0030] It should be noted that, although the above describes the various steps in a specific order, it does not mean that the various steps must be performed in the above specific order. In fact, some of these steps can be performed concurrently or even in a different order, as long as the required functions can be achieved. The number of devices and processing scales described here are used to simplify the description of the present invention, and the application, modification and variation of the present invention are obvious to those skilled in the art.

[0031] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. The instrument symbol and dynamic background image superposition method based on GLStudio is characterized by: The following steps are involved: S1: continuously monitoring the instrument application process, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of a virtual instrument; S2. When a dynamic background image update request is detected in the instrument application process, a background image window rendered by the background application is captured to obtain a background image, wherein the background application is used to generate a dynamic texture image of the virtual instrument display background; S3: replacing the obtained background image with a background layer of the instrument application, the background layer being preset in a lower layer of the symbol layer, and overlaying and rendering the symbol layer and the replaced background layer to form a display screen of the virtual instrument; S4: Loading the display screen of the virtual instrument.

2. The method for superimposing instrument symbols and dynamic background images based on GLStudio according to claim 1, characterized in that: In step S1, the instrument application process is monitored based on the hook technology. When the process attempts to call the wglSwapBuffers function to exchange the rendering buffer and update the display content, a screenshot is taken of the background image window generated by the background application to obtain the background image and save it in the buffer.

3. The method for superimposing instrument symbols and dynamic background images based on GLStudio according to claim 2, characterized in that: The program that monitors the instrument application process is encapsulated into a plug-in module, and the plug-in module code is injected into the instrument application process through the LoadLibrary function in the instrument application initialization code.

4. The method for superimposing instrument symbols and dynamic background images based on GLStudio according to claim 1, characterized in that: In step S3, the background image is replaced with the background layer by obtaining the static texture ID information of the background layer in the instrument program, and using the glTexSubImage2D function to replace the static texture of the background layer with the background image data obtained by the screenshot according to the obtained static texture ID information.

5. The method for superimposing instrument symbols and dynamic background images based on GLStudio according to claim 1, characterized in that: In step S2, the method of taking a screenshot of the background image window in the background application to obtain the background image is: calling the glReadPixels interface to read the pixel information in the background application to obtain the background screenshot, and performing preprocessing operations on the background screenshot, the preprocessing operations including window cropping, resolution matching and size transformation.

6. The instrument symbol and dynamic background image overlay system based on GLStudio is characterized by: include: A monitoring module, which is used to continuously monitor the process of the instrument application, wherein the instrument application is used to load a symbol layer having a dynamic instrument symbol in a display screen of the virtual instrument; A background capture module, which is used to capture a background image window rendered by a background application to obtain a background image when a dynamic background image update request is detected in the instrument application process. The background application is used to generate a dynamic texture image of the virtual instrument display background; A background replacement module, which is used to replace the obtained background image with the background layer of the instrument application, and to overlay and render the symbol layer and the replaced background layer to form a display screen of the virtual instrument, wherein the background layer is preset at the lower layer of the symbol layer obtained by rendering the instrument application; The display module is used to load and display the display screen of the virtual instrument.

7. An electronic device, characterized in that include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.

Images