Drawing graph display method and system, electronic equipment and storage medium
By using OpenGL ES to render data directly to the GPU in multiplayer drawing scenarios, the problem of excessive memory usage and lag is solved, and a smoother drawing experience and performance improvement is achieved.
Patent Information
- Application Number
- CN202311587645.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-24
- Publication Date
- 2025-05-27
AI Technical Summary
The existing technology in the scene of multiple people drawing at the same time leads to excessive memory usage and lag, which cannot effectively reduce the program's occupancy of memory resources.
OpenGL ES is used to directly send data to the GPU for rendering, avoiding the performance consumption caused by CPU calculation and then sending it to the graphics card to render, and extracting the behavioral data to be drawn through preset time intervals, drawing only the current line, and avoiding redrawing all previous lines.
It effectively reduces the program's memory resources when implementing the multi-person drawing function, avoids lag, and improves the fluency and performance of drawing.
Smart Images
Figure CN120047303A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing. Specifically, it relates to a display method, system, electronic device, and storage medium for drawing graphics. Background Art
[0002] In the implementation of existing technologies such as the "Draw and Guess" function, there is usually a drawing time limit. When trying to implement a scenario where multiple users can draw simultaneously, the memory will be heavily occupied, and there are often phenomena such as lag. Therefore, a more efficient implementation method is needed to reduce the memory resources occupied by the program when implementing the multi-user drawing function. Summary of the Invention
[0003] In view of this, the purpose of this application is to provide a display method, system, electronic device, and storage medium for drawing graphics to reduce the memory resources occupied by the program when implementing the multi-user drawing function.
[0004] In a first aspect, this application provides a display method for drawing graphics, which is applicable to a drawing interaction program. The drawing interaction program is used to provide a canvas for multiple users to draw simultaneously. The method includes: for each user's client, updating the display content of the canvas in the following manner: extracting the behavior data to be drawn from the data pool of this user's client at a preset time interval and inputting it into the GPU for rendering, and displaying the graphics drawn by the target user in the canvas; where the data pool includes the behavior data of this user formed based on the drawing operations of this user in the canvas, and the data pool also includes the behavior data of other users sent by the server.
[0005] Preferably, the behavior data at least includes the position coordinates of the track points, the color values of the track points, the diameters of the track points, the track IDs, and the user IDs.
[0006] Preferably, it further includes that each user's client stores the behavior data of this user in the local data pool and uploads the behavior data of this user to the server at the same time.
[0007] Preferably, for the behavior data uploaded by each user's client, the server converts the behavior data of this user in the following manner and sends it to the clients of other users: determining the first screen size information corresponding to this user and determining the second screen size information corresponding to other users; performing screen ratio conversion on the first track point position coordinates in the behavior data of this user based on the first screen size information and the second screen size information to generate second track point position coordinates; replacing the first track point position coordinates in the behavior data of this user based on the second track point position coordinates, and sending the replaced behavior data of this user to the clients of other users.
[0008] Preferably, the server performs screen ratio conversion in the following manner: respectively determine a first height value in the first screen size information, a second height value in the second screen size information, and a first abscissa value in the first trajectory point position coordinates; calculate the ratio between the first abscissa value and the first height value; calculate the product of the ratio and the second height value; and use the product as the second abscissa value in the second trajectory point position coordinates.
[0009] Preferably, for the client of each user, the behavior data to be drawn is rendered in the following manner: process all the trajectory point position coordinates under each user ID according to a preset rule to obtain multiple sets of trajectory point position coordinates under the user ID; for each user ID, input the multiple sets of trajectory point position coordinates under the user ID, the corresponding trajectory point color values, and the trajectory point diameters into the GPU for rendering in sequence.
[0010] Preferably, the client of each user obtains multiple sets of trajectory point position coordinates under each user ID in the following manner: arrange all the trajectory point position coordinates in time sequence; divide every seven consecutive trajectory point position coordinates into a set of trajectory point position coordinates, and use the last three trajectory point position coordinates in the previous set of trajectory point position coordinates as the first three trajectory point position coordinates in the next set of trajectory point position coordinates.
[0011] In a second aspect, the present application provides a display system for drawing graphics, which is applicable to draw an interactive program. The interactive program is used to provide a canvas for multiple users to draw simultaneously. The system includes multiple clients and a server.
[0012] Among them, for the client of each user, the display content of the canvas is updated in the following manner: extract the behavior data to be drawn from the data pool of the client at a preset time interval and render it to display the graphics drawn by the target user on the canvas; where the data pool includes the behavior data of the current user formed based on the drawing operations of the user on the canvas, and the data pool also includes the behavior data of other users sent by the server.
[0013] In a third aspect, the present application further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of a display control method for drawing graphics as described above are executed.
[0014] In a fourth aspect, the present application further provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of a display method for drawing graphics as described above are executed.
[0015] A display method, system, electronic device, and storage medium for drawing graphics provided by the present application are used for drawing an interactive program. The interactive program is used to provide a canvas for multiple users to draw simultaneously. The method includes: for each user's client, updating the display content of the canvas in the following manner: extracting the behavior data to be drawn from the data pool of the client at a preset time interval and rendering it to display the graphics drawn by the target user on the canvas; where the data pool includes the behavior data of the current user formed based on the user's drawing operations on the canvas, and the data pool also includes the behavior data of other users sent by the server side.
[0016] To make the above objects, features, and advantages of the present application more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0018] Figure 1 It is a flowchart of a display method for drawing graphics provided by an embodiment of the present application;
[0019] Figure 2 It is a flowchart of the steps for screen ratio conversion provided by an embodiment of the present application;
[0020] Figure 3 It is a flowchart of a thread lock provided by an embodiment of the present application;
[0021] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only some, rather than all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein generally can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without creative efforts shall fall within the protection scope of this application.
[0023] First, an application scenario applicable to this application is introduced. This application can be applied to optimizing the memory resource occupancy of a drawing interaction program when realizing multi-person drawing interaction.
[0024] The drawing interaction program here can provide a canvas for multiple people to draw simultaneously. Multiple users can access the same room simultaneously through networking and draw on the same canvas based on their respective monitors, or multiple users can draw on the same canvas based on one monitor locally.
[0025] The drawing operations of the users here can be in the form of finger painting or mouse painting, etc.
[0026] It should be noted that when rendering the drawn graphics in the prior art, the commonly used method is to combine CAShapeLayer and UIBezierPath. In a multi-person drawing scenario, such an implementation method will cause an obvious memory increase of more than 75%, resulting in a lag phenomenon. Through investigation, it is found that the steps executed at the system layer in such a method are: redrawing all the lines on the canvas, thus causing a large amount of memory occupancy.
[0027] Based on this, the embodiments of this application provide a method, device, electronic device, and storage medium for displaying drawn graphics.
[0028] Here, OpenGL ES is selected to directly send data to the GPU for rendering, which avoids the high performance consumption caused by calculating from the CPU and then sending it to the graphics card for rendering. At the same time, its drawing principle does not redraw all the previous lines, but draws the current line.
[0029] Please refer to Figure 1 , Figure 1 which is a flowchart of a method for displaying drawn graphics provided by the embodiments of this application. As Figure 1As shown in the figure, a display method for drawing graphics provided by an embodiment of the present application is applicable to an interactive program for providing a canvas for multiple users to draw simultaneously. The interactive program includes a client for each user, and the display content of the canvas is updated in the following manner:
[0030] S10. Extract the behavior data to be drawn from the data pool of the client at a preset time interval and input it to the GPU for rendering, and display the graphics drawn by the target user in the canvas.
[0031] Among them, the data pool includes the behavior data of the current user formed based on the user's drawing operations in the canvas, and the data pool also includes the behavior data of other users sent by the server.
[0032] The behavior data here at least includes the position coordinates of the track points, the color values of the track points, the diameters of the track points, the track IDs, and the user IDs.
[0033] Among them, the position coordinates of the track points are multiple CGPoint coordinates converted from the track of the user's drawing operations in the canvas. The origin of the track point coordinates is the upper left corner (0, 0) of the display of the electronic device, with the right direction as the positive X-axis direction and the downward direction as the positive Y-axis direction.
[0034] The color value of the track point can be configured by the user in the menu bar of the canvas and can adopt the form of a combination of RGB and ALPHA to indicate the color of the drawn line segment.
[0035] The diameter of the track point refers to the thickness of the drawn line segment. Here, the line segment is composed of discontinuous points, and through technical processing, the discontinuous points are smoothly connected to form a line segment. Therefore, the thickness of the line segment depends on the size of the points. Here, we set the point size through a FLOAT (data type used to store single-precision or double-precision floating-point numbers) value.
[0036] The track ID is used to indicate whether the position coordinates of the track points belong to the same line segment. Since the solution in the present application uploads and draws behavior data at time intervals, there will be cases where the position coordinates of the track points in the same line segment are uploaded in two sets of behavior data respectively. To ensure the integrity of the drawing, avoid data loss, and at the same time for the convenience of the user's subsequent undo and delete operations, the same track ID label is set for the position coordinates of the track points in the same complete line segment.
[0037] For example, when user A issues an undo instruction for track A034C, then all the position coordinates of the track points marked with A034C can be deleted from the data pools of all clients.
[0038] The user ID is used to indicate the correspondence between the position coordinates of each trajectory point and the user. For example, in "A034C", "A" can be the user ID.
[0039] Specifically, each client can extract the behavior data to be plotted from the local data pool at a frequency of once every 200 ms.
[0040] For each client, in addition to the behavior data of the local user, the local data pool can also include the behavior data of other remote users sent by the server.
[0041] Specifically, the client of each user stores the behavior data of that user in the local data pool and uploads the behavior data of that user to the server at the same time.
[0042] That is to say, each client will upload the extracted behavior data to the server, and the server will then synchronize it to other non-local clients.
[0043] Furthermore, since the display sizes of the monitors or program pages of the electronic devices corresponding to different clients are different, in order to make the rendered graphics more accurate, the server can perform screen ratio conversion processing before synchronizing the behavior data to other non-local clients.
[0044] Such as Figure 2 As shown, for the behavior data uploaded by the client of each user, the server can convert the behavior data of that user in the following way and send it to the clients of other users:
[0045] S101. Determine the first screen size information corresponding to that user and determine the second screen size information corresponding to other users;
[0046] S102. Perform screen ratio conversion on the first trajectory point position coordinates in the behavior data of that user based on the first screen size information and the second screen size information to generate second trajectory point position coordinates;
[0047] S103. Replace the first trajectory point position coordinates in the behavior data of that user based on the second trajectory point position coordinates and send the replaced behavior data of that user to the clients of other users.
[0048] Specifically, the server performs screen ratio conversion in the following way:
[0049] Respectively determine the first height value in the first screen size information, the second height value in the second screen size information, and the first abscissa value in the first trajectory point position coordinates. Calculate the ratio between the first abscissa value and the first height value. Calculate the product of the ratio and the second height value. Use the product as the second abscissa value in the second trajectory point position coordinates.
[0050] Exemplarily, the first screen size information may be 600×800, the second screen size information may be 300×400, the first trajectory point position coordinate may be P1(360, 500), and the second trajectory point position coordinate may be P2(360 / 600×300, 500 / 800×400), that is, P2(180, 250).
[0051] In an embodiment of the present application, a clear and undo function for drawing operations is also provided. Clearing refers to clearing all the line segments drawn by the user in the current canvas; undoing refers to deleting the line segments drawn by the user from new to old in chronological order, and each undo operation only deletes one line segment.
[0052] When the target user issues a clear or undo operation through the target client, the server will broadcast it to all other non-local clients associated with the current canvas. The non-local clients of other users will delete the behavior data corresponding to the user ID of the target user according to the instruction and update the display interface of the canvas.
[0053] Further, when the user terminal of the target user first connects to the target canvas, the server will synchronize the behavior data generated by other users in the target canvas to the user terminal of the target user. When the user terminal of the target user leaves, the user terminal of the target user will delete all the behavior data saved in the data pool.
[0054] In an embodiment of the present application, when extracting the behavior data to be drawn each time, alignment smoothing processing can be considered to avoid the problem of discontinuous line segments caused by rendering the behavior data at intervals.
[0055] Specifically, for the user terminal of each user, the behavior data to be drawn is rendered in the following manner:
[0056] Process all the trajectory point position coordinates under each user ID according to a preset rule to obtain multiple trajectory point position coordinate sets under the user ID.
[0057] Specifically, the user terminal of each user obtains multiple trajectory point position coordinate sets under each user ID in the following manner: arrange all the trajectory point position coordinates in chronological order; divide every seven consecutive trajectory point position coordinates into a trajectory point position coordinate set, and the last three trajectory point position coordinates in the previous trajectory point position coordinate set are used as the first three trajectory point position coordinates in the next trajectory point position coordinate set.
[0058] For each user ID, input the multiple trajectory point position coordinate sets, the corresponding trajectory point color values, and the trajectory point diameters under the user ID into the GPU for rendering.
[0059] Exemplarily, when the position coordinates of the trajectory points corresponding to user A in the behavior data to be drawn can be A = {P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11, P12, P13}, the corresponding sets of position coordinates of the trajectory points divided are A1 = {P1, P2, P3, P4, P5, P6, P7}, A2 = {P5, P6, P7, P8, P9, P10, P11}, and A3 = {P9, P10, P11, P12, P13, P13, P13}. For A1, A2, and A3, each line segment is drawn separately.
[0060] In an embodiment of the present application, a display system for drawing a graph is provided, which is applicable to drawing an interactive program. The interactive program is used to provide a canvas for multiple users to draw simultaneously. The system includes multiple user terminals and a server terminal.
[0061] Among them, for each user's user terminal, the display content of the canvas is updated in the following manner: extracting the behavior data to be drawn from the data pool of the user terminal at a preset time interval and rendering it to display the graph drawn by the target user on the canvas. Among them, the data pool includes the behavior data of the user formed by the user's drawing operation on the canvas of the current user terminal, and the data pool also includes the behavior data of other users sent by the server terminal.
[0062] Specifically, in order to make the interactive drawing program have the characteristics of high performance, multi-terminal collaboration, and no time limit, the following improvements have been made in the development stage:
[0063] (1) In terms of high performance
[0064] (1) Reduce memory occupation:
[0065] Memory is a limited resource. If too much memory is occupied, it will cause the APP to freeze and even risk being terminated by the system.
[0066] When developing the drawing function, multiple methods can be adopted. The first one is to use the combination of the object CAShapeLayer provided by the system and UIBezierPath. CAShapeLayer belongs to the QuartzCore framework and inherits from CALayer. CAShapeLayer is used to draw Bezier curves within the coordinate system. By drawing Bezier curves and setting the path of the shape, various regular and irregular graphics can be drawn. Therefore, CAShapeLayer needs to be used together with UIBezierPath. The UIBezierPath class allows you to draw and render paths composed of straight lines and curves in a custom view. You can directly specify a geometric figure for UIBezierPath during initialization. That is, UIBezierPath is used to specify the drawing path of the figure, and CAShapeLayer is used to draw the figure according to the path.
[0067] After testing, it is found that when using the combination of CAShapeLayer and UIBezierPath for development, obvious memory growth of more than 75% will occur after drawing two thousand line segments, resulting in a lag phenomenon. Through investigation, it is found that using this method to draw not only draws the currently needed lines for rendering but also draws all the lines. Therefore, when there are operations to add or subtract lines, the memory will increase, and the memory will decrease after the drawing is completed.
[0068] The specific way to troubleshoot is as follows: when the memory is too high, analyze the detailed memory usage by viewing the Memory Graph Hierarchy. Specifically, the Memory Graph Hierarchy can be exported to a.memgraph file, and tools such as vmmap, leaks, heap, and malloc_history can be used to analyze this file in the terminal. Use vmmap to obtain an overview of the current a.memgraph file: vmmap-summary a.memgraph. Use leaks to check for memory leaks: leaks a.memgraph. Use heap to display the objects allocated on the process heap: heap -sortBySize a.memgraph. Use malloc_history to check the creation process of a certain memory: malloc_history a.memgraph [address]. Then the code that causes a large amount of memory occupation can be located.
[0069] Therefore, the solution of this application adopts an implementation method based on OpenGL ES. The system can directly send image data to the GPU for rendering using OpenGLES, thus avoiding the high performance consumption caused by calculating on the CPU and then sending it to the graphics card for rendering. At the same time, its drawing principle does not redraw all the previous lines, but draws the current line. Through memory analysis, the memory occupancy rate of this method basically does not increase, and the memory performance is improved by more than 95% compared with CAShapeLayer.
[0070] (2) Reduce battery consumption
[0071] When a large amount of computation is required during code execution, a large amount of power input is needed, which is also the reason for the heating of the battery and motherboard components. Excellent code needs to complete its requirements by occupying as few resources as possible. Using the OpenGL ES framework for drawing, only the current line is drawn each time. Even if a large number of lines are drawn, the performance occupied is the same as that of drawing a single line segment. When drawing lines, the GPU rendering is fully utilized and the CPU performance is released. At the same time, many data of OpenGLES are directly stored in the rendering context. The OpenGL ES implementation can convert the data into the format most suitable for the graphics hardware to improve performance. For example, some data that does not change frequently can be stored in dedicated memory (such as VBO) applied on the GPU.
[0072] (3) Save initialization time
[0073] Using the OpenGL ES framework for drawing is closer to the underlying drawing method of the system. The object types provided by the system are more for the convenience of developers, and the underlying rendering method has been encapsulated multiple times. Although it is convenient to use, it inevitably requires more code, which leads to more initialization operations.
[0074] The initialization operation of an object is a time-consuming operation. Therefore, lazy loading (loading the object when it is used) is adopted here, and at the same time, a more underlying method is used to avoid excessive code and initialization objects. The reusable resources are put into the cache in advance to reduce unnecessary I / O operations.
[0075] (4) Improve execution speed
[0076] Reducing unnecessary initializations and code volume are important ways to improve execution speed. Another way to reflect execution speed is to respond to user operations. By drawing a line segment every seven points and using the last three points of each segment as the first three points of the next segment, a complete line has good continuity and smoothness. At the same time, we broadcast changes to the line segment to the drawing board on the same channel every 200ms when drawing points, so that the line can be updated smoothly on the drawing boards of the same channel. The drawing of user A can be displayed more smoothly on the device, and it is also smoothly and quickly synchronized on the device of user B.
[0077] (5) Improve response speed
[0078] To achieve a fast response speed, we use OpenGL ES framework for drawing, which can better mobilize the performance of the GPU. At the same time, we use asynchronous operations to upload data, so that the UI operations on the interface and the data storage and broadcasting operations of the user do not affect each other and are synchronized. For example, when the content drawn by user A is synchronously displayed on the device of user B, user B is also drawing synchronously. At this time, multiple lines will be rendered simultaneously on device B, and there will be no problem of response delay for the graffiti operation of user B.
[0079] (6) Enhance security
[0080] Since the data we transmit over the network includes the user ID and device ID, etc., to better improve security performance, in the embodiments of this application, the data will be encrypted when transmitted. For example, the user ID is processed by MD5, and a fixed value can be added to the user point data.
[0081] The security of code operation is also part of security. For example, regarding the execution order of data processing during operation, if the data processing cannot mix the operation data of each user and cannot mix the drawing order of line segments. For operation data, for example, the line segment drawn by user A cannot be regarded as that of user B, or the data of line segment one cannot be added to line segment two. To prevent these problems, the data we process each time is in chronological order and is saved according to the user ID and line segment ID, so that the lines and users are in one-to-one correspondence.
[0082] To ensure drawing security, the method of thread lock is also used. When drawing, a single thread is used for drawing. The unprocessed line data before will be temporarily stored in the data pool first, and the thread pool will be traversed all the time, and the data stored first will be taken out first. Because the data stored in chronological order is the position coordinate points at an earlier time, each time the behavior data is extracted from the data pool after traversing the data pool, the behavior data will be deleted from the data pool until there is no data in the data pool, which means that the lines to be drawn have been rendered. This process is extremely fast in the mobile phone system processing, which also makes the drawing smooth and the security is guaranteed.
[0083] (2) Multi - terminal collaboration
[0084] To ensure high performance, it is possible to consider separating data processing and interface rendering. This requires strong concurrent processing capabilities. Specifically, GCD can be used to create a new thread, and the line data sent by users at different ends of the same channel can be processed in this thread. However, multi - threading also gives rise to data security issues, that is, data anomalies may occur when multiple threads access the same piece of data. To prevent this problem, a thread lock will be introduced when operating on data (such as Figure 3 ). That is, when a thread accesses the target data, other threads cannot access it, and only after this thread finishes accessing can other threads continue to access. This can ensure both the efficiency of multi - threading and the security of data.
[0085] Data intercommunication means that the data of a drawing board in the same channel is shared, and the operation data of different users will be broadcast to other users in this channel in a timely manner. To ensure this implementation, we use the form of HTTP long - connection to receive data.
[0086] After the user performs operations such as graffiti drawing or undo / clear, the program will send these instruction data to the server through the long - connection interface. The server broadcasts them to other users in the same channel according to the instructions. Through testing, it can be known that this process can achieve a delay of milliseconds and can well implement the ability of real - time data intercommunication.
[0087] (3) Unlimited time
[0088] In daily tests, it is found that many graffiti drawings limit the user's drawing time, which can ensure that there will not be too many line drawings within the specified time, and there will be no phenomenon of performance degradation and lag.
[0089] To solve this problem, drawing is carried out based on the OpenGL ES framework here. OpenGL ES is a powerful graphics library, a cross - platform graphics API, and a simplified version of OpenGL. The system can use OpenGL ES to directly send image data to the GPU for rendering, which avoids the high performance consumption caused by calculating from the CPU and then sending it to the graphics card for rendering, and can bring better usage effects and user experiences.
[0090] In an embodiment of the present application, for the target client, the following steps can be used for rendering:
[0091] Import the header file #import <GLKit / GLKit.h>; GLKit.h uses OpenGL ES at the bottom layer. Importing it is equivalent to automatically importing OpenGL ES;
[0092] @property(nonatomic, strong) NSMutableArray *allLineMutableArray; / / Records all the traced line segments, stored in chronological order, for the user's undo and clear actions;
[0093] @property(nonatomic, strong) NSMutableDictionary *receiveCopyDictionary; / / Records the points received within two hundred milliseconds;
[0094] @property(nonatomic, strong) NSMutableArray *sendPointArray; / / Sends the array of points generated within two hundred milliseconds;
[0095] @property(nonatomic, strong) NSMutableArray *updLines; / / Lines that need to be updated.
[0096] textureInfo_t brushTexture; / / The texture of the brush;
[0097] glBindBuffer(GL_ARRAY_BUFFER, vboId); / / Binds the vertex buffer;
[0098] glUseProgram(program[PROGRAM_POINT].id); / / Enables the shader;
[0099] ywGlueCreateProgram(vsrc, fsrc, attribCt, (const GLchar **)&attribUsed[0], attrib, NUM_UNIFORMS, &uniformName[0], program[i].uniform, &program[i].id) / / Creates the shader.
[0100] The shader here is usually used to process texture objects and render the processed texture objects to the frame buffer, and then display them on the screen. By extracting texture information, operations such as vertex coordinate space conversion and texture color adjustment (filter effect) can be processed.
[0101] Shaders are divided into vertex shaders and fragment shaders. The shader only needs to be created once. The vertex shader is used to determine the shape of the graph, and the fragment shader is used to determine the rendering color of the graph. The entire usage process of the shader is as follows:
[0102] 1. Create a shader object glCreateShader;
[0103] 2. Associate the source code with each shader object glShaderSource;
[0104] 3. Compile the shader;
[0105] 4. Create a program glCompileShader;
[0106] 5. Attach the shader to the program object glAttachShader;
[0107] 6. Link the program glCreateProgram;
[0108] 7. Activate and deactivate the shader program glUseProgram;
[0109] 8. Delete the shader and the program glDeleteProgram.
[0110] A texture is a cache for storing the element values of an image color. Rendering refers to the process of generating an image from data. Texture rendering is the process of generating an image from data such as color values stored in memory. Texture mapping is a rather complex process, and the basic steps are as follows:
[0111] 1. Create texture memory (GLubyte*)calloc(width*height*4,sizeof(GLubyte));
[0112] 2. Bind the texture memory CGBitmapContextCreate;
[0113] 3. Let the texture store the picture data CGContextDrawImage;
[0114] 4. Apply for a texture ID glGenTextures;
[0115] 5. Bind the texture ID glBindTexture;
[0116] 6. Clear the texture glUniform1i.
[0117] Among them, texture mapping can only be performed in the RGBA mode.
[0118] Part of OpenGL ES runs on the CPU and part runs on the GPU. To coordinate the data exchange between these two parts, the concept of buffers is defined. Both the CPU and the GPU have their own independently controlled memory areas. Buffers can avoid data replication between these two memory areas and improve efficiency. Here, vertex buffers can be bound, configured with data, and then the buffer data can be given to the vertex shader and the shader can be started to begin drawing.
[0119] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 4 shown in the figure, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.
[0120] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 runs, the processor 410 communicates with the memory 420 through the bus 430. When the machine-readable instructions are executed by the processor 410, the steps of the display method for drawing graphics in the above embodiments can be executed. The specific implementation manner can refer to the method embodiments and will not be elaborated here.
[0121] An embodiment of the present application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the display method for drawing graphics in the above embodiments can be executed. The specific implementation manner can refer to the method embodiments and will not be elaborated here.
[0122] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.
[0123] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.
[0124] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0125] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist physically alone for each unit, or two or more units may be integrated in one unit.
[0126] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.
[0127] Finally, it should be noted that: the above-described embodiments are only specific implementation manners of the present application, used to illustrate the technical solutions of the present application, rather than limiting it. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A method for displaying a drawn graphic, It is characterized in that Applicable to a drawing interactive program, the drawing interactive program is used to provide a canvas for multiple users to draw simultaneously, the method comprising: For each user's client, the displayed content of the canvas is updated in the following manner: Extracting the behavior data to be drawn from the data pool of the user terminal at a preset time interval, inputting the data into the GPU for rendering, and displaying the graphics drawn by the target user in the canvas; The data pool includes the behavior data of the user formed by the current user terminal based on the drawing operation of the user on the canvas, and the data pool also includes the behavior data of other users sent by the server terminal.
2. The method according to claim 1, It is characterized in that The behavior data at least includes the track point position coordinates, the track point color value, the track point diameter, the track ID and the user ID.
3. The method according to claim 2, It is characterized in that Also includes: The user end of each user stores the user's behavior data in a local data pool, and uploads the user's behavior data to the server end.
4. The method according to claim 3, It is characterized in that For each user's behavior data uploaded by the client, the server converts the user's behavior data in the following manner and sends it to other users' clients: Determine first screen size information corresponding to the user and determine second screen size information corresponding to other users; Performing screen ratio conversion on the first track point position coordinates in the user's behavior data based on the first screen size information and the second screen size information to generate second track point position coordinates; Based on the second track point position coordinates, the first track point position coordinates in the behavior data of the user are replaced, and the replaced behavior data of the user is sent to the user terminals of other users.
5. The method according to claim 4, It is characterized in that The server performs screen ratio conversion in the following ways: respectively determining a first height value in the first screen size information, a second height value in the second screen size information, and a first horizontal coordinate value in the first track point position coordinates; Calculating a ratio between the first horizontal coordinate value and the first height value; Calculating the product between the ratio and the second height value; The product is used as the second horizontal coordinate value in the position coordinates of the second trajectory point.
6. The method according to claim 1, It is characterized in that For each user's client, the behavior data to be drawn is rendered in the following way: Processing all the track point position coordinates under each user ID according to a preset rule to obtain a set of multiple track point position coordinates under the user ID; For each user ID, a plurality of track point position coordinate sets under the user ID and corresponding track point color values and track point diameters are sequentially input into the GPU for rendering.
7. The method according to claim 6, It is characterized in that Each user's client obtains multiple track point location coordinate sets under each user ID in the following way: Arrange the position coordinates of all trajectory points in time sequence; Every seven consecutive trajectory point position coordinates are divided into a trajectory point position coordinate set, and the last three trajectory point position coordinates in the previous trajectory point position coordinate set are used as the first three trajectory point position coordinates in the next trajectory point position coordinate set.
8. A display system for drawing graphics, It is characterized in that The system is suitable for a drawing interactive program, wherein the drawing interactive program is used to provide a canvas so that multiple users can draw at the same time. The system includes multiple user terminals and a server terminal. For each user's client, the displayed content of the canvas is updated in the following manner: Extracting the behavior data to be drawn from the data pool of the user terminal at a preset time interval and rendering the data to be drawn, so as to display the graphics drawn by the target user in the canvas; The data pool includes the behavior data of the user formed by the current user terminal based on the drawing operation of the user on the canvas, and the data pool also includes the behavior data of other users sent by the server terminal.
9. An electronic device, It is characterized in that include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for displaying a drawing graphic as described in any one of claims 1 to 7.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for displaying a drawing graphic as claimed in any one of claims 1 to 7 are executed.