A motion state recognition method, device, storage medium and electronic device

Abstract

The present application discloses a motion state recognition method, device, storage medium and electronic device, wherein the method comprises: obtaining a target graphic drawing instruction for a target application, obtaining a first camera position matrix for an application element in a first frame in the target graphic drawing instruction; obtaining a second camera position matrix for the application element in a recorded second frame, wherein the second frame is a frame obtained before the first frame based on the target application, and the number of frames separated from the first frame is less than or equal to a preset number of frames; determining the motion state of the application element based on the first camera position matrix and the second camera position matrix, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be judged by the camera position matrix between different frames during the operation of the target application.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a motion state recognition method, device, storage medium and electronic device. Background Art

[0002] With the development of computer technology, various game applications have been developed to enrich people's entertainment life. With the continuous iteration and upgrading of various game applications and the promotion of large-scale game applications, the hardware configuration required for the operation of many popular game applications is getting higher and higher. In the case of insufficient hardware configuration, the operation effect of the game application will be poor and the user experience will be poor.

[0003] At present, under the condition of limited hardware configuration conditions, improving the running effect of game applications and enhancing the user's gaming experience has become another research direction for related companies. Summary of the invention

[0004] The embodiments of the present application provide a motion state recognition method, device, storage medium and electronic device. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a motion state recognition method, the method comprising:

[0006] Obtain a target graphics drawing instruction for a target application, and obtain a first camera position matrix for an application element in a first frame in the target graphics drawing instruction;

[0007] Acquire a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame acquired before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0008] A motion state of the application element is determined based on the first camera position matrix and the second camera position matrix.

[0009] In a second aspect, an embodiment of the present application provides a motion state recognition device, the device comprising:

[0010] A first matrix acquisition module is used to acquire a target graphics drawing instruction for a target application, and acquire a first camera position matrix for an application element in a first frame in the target graphics drawing instruction;

[0011] A second matrix acquisition module, used to acquire a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame acquired before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0012] A motion state determination module is used to determine the motion state of the application element based on the first camera position matrix and the second camera position matrix.

[0013] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0014] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0015] In one or more embodiments of the present application, by obtaining a target graphics drawing instruction for a target application, obtaining a first camera position matrix for an application element in a first frame in the target graphics drawing instruction, and then obtaining a second camera position matrix for the application element in a recorded second frame, the second frame being a frame obtained before the first frame based on the target application, and having a frame number less than or equal to a preset number of frames separated from the first frame, and finally determining the motion state of the application element based on the first camera position matrix and the second camera position matrix, it is achieved that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be determined by the camera position matrix between different frames during the operation of the target application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of a motion state recognition method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a motion state recognition method provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a motion state recognition method provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application;

[0022] Figure 6 A schematic diagram of the structure of a motion state determination module provided in an embodiment of the present application;

[0023] Figure 7 A structural schematic diagram of an image quality adjustment module provided in an embodiment of the present application;

[0024] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present application, it should be noted that, unless otherwise clearly specified and limited, "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are an "or" relationship.

[0027] In the prior art, during the operation of a game application, a user terminal can obtain the motion state of the game screen of the game application based on information such as the action and time of elements in the game screen sent by a game client, and optimize the operating parameters of the game application in real time according to the current motion state of the game screen to improve the operating effect of the game application.

[0028] However, some game applications are not given the function of sending action and time information from the game client to the user terminal during operation during the development stage. That is, during the game operation, the user terminal cannot know the movement status of the game application game screen, and cannot optimize the operation parameters of the game application in real time according to the current movement status of the game screen.

[0029] Based on this, the present application proposes a motion state recognition method. During the operation of the target application, by obtaining the target graphics drawing instructions for the target application, obtaining the first camera position matrix for the application elements in the first frame in the target graphics drawing instructions, and then obtaining the second camera position matrix of the application elements in the recorded second frame, the second frame is a frame obtained before the first frame based on the target application, and the number of frames separated from the first frame is less than or equal to a preset number of frames. Finally, the motion state of the application elements is determined based on the first camera position matrix and the second camera position matrix, so that when the target application client does not send any information such as the action and time of the application elements, the motion state of the application elements in the target application can be judged by detecting the camera position matrix between different frames during the operation of the target application.

[0030] The following is a detailed description in conjunction with specific embodiments. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily follow the steps shown. For example, some steps are parallel and do not have a strict logical order, so the actual execution order is variable.

[0031] See also Figure 1 , is a flow chart of a motion state recognition method provided in an embodiment of the present application. In a specific embodiment, the motion state recognition method is applied to a motion state recognition device or an electronic device equipped with a motion state recognition device. Figure 2 The process shown is described in detail, and the motion state recognition method may specifically include the following steps:

[0032] S101, obtaining a target graphics drawing instruction for a target application, and obtaining a first camera position matrix for an application element in a first frame in the target graphics drawing instruction;

[0033] It can be understood that during the operation of the target application, the central processor in the electronic device sends a graphics drawing instruction for the target application to the image processor to implement the update of each frame of the application screen of the target application.

[0034] Specifically, during the operation of the target application, the central processor in the electronic device sends a target graphics drawing instruction for the target application to the image processor, and obtains a first camera position matrix of application elements in an application screen of the target application in the target graphics drawing instruction.

[0035] It should be noted that during the operation of the target application, when the target application screen needs to be displayed, the central processing unit will call the graphics API to send graphics drawing instructions to the graphics processor, render the data of each frame of the application screen, thereby obtaining the rendered application screen and displaying it on the display screen. For each frame of the application screen, the central processing unit will send multiple graphics drawing instructions to the graphics processor, and the graphics processor will sequentially obtain the graphics drawing instructions sent by the central processing unit and perform rendering operations. When the graphics processor completely executes the graphics drawing instructions corresponding to a frame of the application screen, the rendered application screen is obtained, and then the frame of the application screen can be displayed on the display screen.

[0036] When the CPU sends a graphics drawing instruction to the GPU, the CPU stores the graphics drawing instruction in a buffer of the GPU in consideration of the speed mismatch, and the GPU sequentially obtains each graphics drawing instruction from the buffer.

[0037] Graphics drawing instructions are graphics API instructions, which are used to call a graphics processor to render and draw the display screen of a target application. Graphics drawing instructions can be instructions of an application programming interface (API) such as OpenGL, OpenGL ES, and DirectX.

[0038] The application element is one of the elements of the application screen of the target application, for example, it can be a person, a car, an animal, or a character.

[0039] The first camera position matrix is ​​one or more matrices in the model view projection matrix used by the shader in the graphics processor when rendering the application element in the first frame, and can represent the motion state of the application element in the application picture.

[0040] The model view projection matrix, also known as the MVP matrix, is a matrix obtained by transforming the model matrix, view matrix, and projection matrix. It can be used in shaders to render and display the application screen after the perspective transformation according to the model view projection matrix. The matrices that make up the model view projection matrix are stored in the data passed by each graphics drawing instruction, among which the first camera position matrix is ​​stored in the data passed by the target graphics drawing instruction.

[0041] In one embodiment, when the target application is run for the first time, the target graphics drawing instruction may be determined in the following manner:

[0042] Before obtaining the target graphics drawing instructions for the target application sent by the central processor to the graphics processor, multiple frames of the target application when the application elements are in different motion states are intercepted, and the differences between the model view projection matrices corresponding to the application elements in different motion states are compared, and the camera position matrix is ​​determined in the multiple matrices constituting the model view projection matrix according to the differences between the model view projection matrices corresponding to the different motion states. Then, the first buffer where the camera position matrix is ​​located is determined in each buffer of the graphics processor, and the first buffer identifier of the first buffer is determined, and the buffer named with the first buffer identifier is used as the target buffer, and the graphics drawing instructions stored in the target buffer are used as the target graphics drawing instructions.

[0043] During the operation of the target application, the graphics drawing instructions for the target application sent by the central processing unit to the graphics processing unit are monitored in real time to determine whether the buffer area where the graphics drawing instructions are stored is the target buffer area named with the first buffer area identifier. If it is determined that the buffer area where the graphics drawing instructions are stored is the target buffer area named with the first buffer area identifier, the graphics drawing instructions are determined to be the target graphics drawing instructions, and a first camera position matrix for the application elements in the first frame of the target graphics drawing instructions is obtained.

[0044] In one embodiment, when the target application is not running for the first time, the target graphics drawing instruction may be determined in the following manner:

[0045] A second buffer having the same size as the first buffer is determined in each buffer of the graphics processor, and a second buffer identifier of the second buffer is determined, the buffer named with the second buffer identifier is used as the target buffer, and the graphics drawing instruction stored in the target buffer is used as the target graphics drawing instruction.

[0046] During the operation of the target application, the graphics drawing instructions sent by the central processing unit to the graphics processing unit are monitored in real time to determine whether the buffer area where the graphics drawing instructions are stored is the target buffer area named with the second buffer area identifier. If it is determined that the buffer area where the graphics drawing instructions are stored is the target buffer area named with the second buffer area identifier, the graphics drawing instructions are determined to be the target graphics drawing instructions, and a first camera position matrix for the application element in the first frame of the target graphics drawing instructions is obtained.

[0047] S102, obtaining a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame obtained based on the target application before the first frame, and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0048] Specifically, a second camera position matrix of the application element in the recorded second frame is obtained, the second frame is a frame before the first frame during the running of the target application, and the number of frames between the second frame and the first frame is less than or equal to a preset number of frames, and the second camera position matrix is ​​a camera position matrix corresponding to the application element in the second frame obtained before obtaining the first camera position matrix of the first frame.

[0049] S103: Determine a motion state of the application element based on the first camera position matrix and the second camera position matrix.

[0050] Specifically, the difference between the first camera position matrix corresponding to the application element in the first frame and the second camera position matrix corresponding to the application element in the second frame is calculated. If the difference between the first camera position matrix and the second camera position matrix is ​​zero, that is, it indicates that the application element has not moved from the second frame to the first frame, then the motion state of the application element is determined to be a stationary state; if the difference between the first camera position matrix and the second camera position matrix is ​​not zero, that is, it indicates that the application element has moved from the second frame to the first frame, then the motion state of the application element is determined to be a moving state.

[0051] In one embodiment, after the motion state of the application element is determined, the screen display quality of the target application may be adjusted based on the motion state of the application element.

[0052] Optionally, if it is determined that the motion state of the application element is a moving state, the picture display quality of the current picture is reduced based on the load reduction technology; if it is determined that the motion state of the application element is a stationary state, the picture display quality of the current picture is kept unchanged.

[0053] It is understandable that when the motion state of the application element is static, the changes in each frame displayed by the target application are small or unchanged. When rendering and displaying each frame, the computing pressure of the central processing unit and the graphics processing unit is small, and there will be no jamming caused by insufficient hardware performance. Therefore, there is no need to reduce the display quality of the target application, and a good display effect can be maintained to meet user needs. When the motion state of the application element is mobile, the frames displayed in the target application change rapidly with the movement of the application element. At this time, the changes in each frame displayed by the target application are large. The computing pressure faced by the central processing unit and the graphics processing unit when rendering and displaying each frame is large, and jamming caused by insufficient hardware performance is prone to occur. Therefore, the display quality of the target application can be reduced based on the load reduction technology to relieve the computing pressure of the central processing unit and the graphics processing unit, and keep the target application running smoothly. When the application element moves quickly, due to the rapid changes in each frame displayed in the target application, the user will not pay too much attention to the display quality of the picture, and the reduction in the display quality of the picture will not have a significant impact on the user experience.

[0054] In one or more embodiments of the present application, during the operation of the target application, a target graphics drawing instruction for the target application sent by the central processing unit to the graphics processing unit is obtained, a first camera position matrix for the application element in the first frame in the target graphics drawing instruction is obtained, and then a second camera position matrix for the application element in the recorded second frame is obtained, where the second frame is a frame obtained before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames, and finally the motion state of the application element is determined based on the first camera position matrix and the second camera position matrix, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be determined by detecting the camera position matrix between different frames during the operation of the target application.

[0055] In one embodiment, before obtaining the target graphics drawing instruction, it is also necessary to determine the target buffer where the target graphics drawing instruction is stored. When the target application is run for the first time, refer to Figure 2 , is a flow chart of a motion state recognition method provided in an embodiment of the present application. Figure 2 As shown, the motion state recognition method may include the following steps:

[0056] S201, intercepting multiple frames of images when the application element in the target application is in different motion states, and obtaining difference data between the model view projection matrix of each frame in the multiple frames;

[0057] Specifically, when the target application is run for the first time, multiple frames of the target application are captured when the application elements are in different motion states, and the differences between the model view projection matrices corresponding to the application elements in each frame are compared to obtain the difference data between the model view projection matrices.

[0058] For example, capture a frame in which an application element in the target application is in a stationary state and capture a frame in which an application element in the target application is in a running state, compare the model view projection matrices corresponding to the application elements in the two frames, and obtain the difference data between the two model view projection matrices.

[0059] S202, determining a camera position matrix among a plurality of matrices constituting a model view projection matrix based on the difference data;

[0060] It can be understood that when the application element is in different motion states, the model view projection matrix used by the shader in rendering and displaying the target element is different. By comparing the differences between the model view projection matrices of the application element in different motion states, the difference data between the model view projection matrices are obtained. Based on the difference data, the camera position matrix used to represent the motion state of the application element can be determined from the multiple matrices that constitute the model view projection matrix.

[0061] S203, determining a first buffer in a buffer set where a camera position matrix is ​​located, wherein each buffer in the buffer set is used to store a graphics drawing instruction for a target application;

[0062] S204, using the first buffer as a target buffer, and using the graphics drawing instructions stored in the target buffer as target graphics drawing instructions;

[0063] Each buffer in the buffer set is a buffer in the graphics processor, and is used to store graphics drawing instructions for a target application sent by the central processing unit to the graphics processor.

[0064] Specifically, in step S203 to step S204, after determining the camera position matrix used to represent the motion state of the application element from among the multiple matrices constituting the model view projection matrix, the first buffer where the camera position matrix is ​​located is determined from among the buffers of the graphics processor, the first buffer is used as the target buffer, and the graphics drawing instructions stored in the target buffer are used as the target graphics drawing instructions. The target graphics drawing instructions are graphics drawing instructions carrying the camera position matrix.

[0065] It should be understood that the camera position matrix used to represent the motion state of the application element is not obvious during the first run, that is, the electronic device running the target application does not know which matrix can represent the motion state of the application element at the initial stage, and the camera position matrix is ​​obtained by comparing the differences between the projection matrices of the model views of the application element in different motion states. For the target element in the same target application, the camera position matrix used to represent the motion state of the application element will not change again, so it is only necessary to perform the steps of determining the camera position matrix in sequence when the target application is run for the first time.

[0066] The camera position matrix is ​​the data transmitted in the graphics drawing instructions sent by the central processing unit to the graphics processing unit. After determining the camera position matrix, it is necessary to locate the first buffer where the camera position matrix is ​​stored, and use the first buffer as the target buffer, and use the graphics drawing instructions stored in the target buffer as the target graphics drawing instructions, so as to monitor the target graphics drawing instructions stored in the buffer during the operation of the target application.

[0067] It should be noted that, from the start of the target application to the end of the target application, in a complete operation process, the buffer in which the target graphics drawing instructions carrying the camera position matrix are stored will not change. That is, after the target buffer is determined, the graphics drawing instructions stored in the target buffer are the target graphics drawing instructions carrying the camera position matrix.

[0068] S205, monitoring the graphics drawing instructions for the target application in real time, and determining whether the buffer area where the graphics drawing instructions are stored is the target buffer area;

[0069] S206, if it is determined that the buffer area where the graphics drawing instruction is stored is the target buffer area, then determining the graphics drawing instruction as the target graphics drawing instruction;

[0070] Specifically, steps S205 to S206 monitor the graphics drawing instructions sent by the central processing unit to the graphics processing unit in real time, and determine whether the buffer identifier of the buffer area where the graphics drawing instructions are stored is the same as the target buffer identifier of the target buffer. If they are the same, the buffer area where the graphics drawing instructions are stored is determined to be the target buffer, and the graphics drawing instructions are determined to be the target graphics drawing instructions carrying the camera position matrix.

[0071] The buffer identifier may be a buffer name identifier or a buffer size identifier.

[0072] Optionally, if it is determined that the buffer area where the graphics drawing instruction is stored is not the target buffer area, then it is determined that the graphics drawing instruction is not the target graphics drawing instruction.

[0073] S207, obtaining a first camera position matrix for an application element in a first frame in a target graphics drawing instruction;

[0074] Specifically, after determining the target graphics drawing instruction, the first camera position matrix for the application element in the first frame is obtained from the data transmitted by the target graphics drawing instruction based on the HOOK function.

[0075] The first frame is an application screen of the target application rendered and displayed by the graphics processor based on the target graphics drawing instruction.

[0076] S208, obtaining a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame obtained based on the target application before the first frame, and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0077] Specifically, for step S208, please refer to the detailed description of step S102 in another embodiment, which will not be described in detail here.

[0078] S209, calculating the difference between the first camera position matrix and the second camera position matrix;

[0079] S210, if the difference is zero, determining that the motion state of the application element is a stationary state;

[0080] S211, if the difference is not zero, determining that the motion state of the application element is a moving state;

[0081] Optionally, the moving state may include a walking state, a sprinting state, and a driving state. In one embodiment, the moving speed of the application element may also be calculated based on the time interval between the first frame and the second frame and the difference between the first camera position matrix and the second camera position matrix. If the moving speed of the application element is greater than zero and less than a first speed threshold, the motion state of the application element is determined to be a walking state; if the moving speed of the application element is greater than the first speed threshold and less than the second speed threshold, the motion state of the application element is determined to be a sprinting state; if the moving speed of the application element is greater than the second speed threshold, the motion state of the application element is determined to be a driving state.

[0082] S212, if it is determined that the motion state of the application element is a moving state, reducing the picture display quality of the current picture based on a load reduction technology;

[0083] It is understandable that when the motion state of an application element is a moving state, the frames displayed in the target application change rapidly along with the movement of the application element. At this time, the frames displayed by the target application change greatly, and the central processing unit and the graphics processor face great computing pressure when rendering and displaying each frame, which is prone to jamming caused by insufficient hardware performance. Therefore, the display quality of the target application can be reduced based on the load reduction technology to alleviate the computing pressure of the central processing unit and the graphics processor and keep the target application running smoothly. When the application element moves quickly, since the frames displayed in the target application change rapidly, the user will not pay too much attention to the display quality, and the reduction in the display quality will not have a significant impact on the user experience.

[0084] The load reduction technology may be a GPU load reduction technology or a variable resolution rendering technology.

[0085] Optionally, the moving state may include a walking state, a sprinting state, and a driving state. In one embodiment, if the motion state of the application element is determined to be a walking state, the game image quality is reduced to a first resolution based on a load reduction technology; if the motion state of the application element is determined to be a sprinting state, the game image quality is reduced to a second resolution based on a load reduction technology; if the motion state of the application element is determined to be a driving state, the game image quality is reduced to a third resolution based on a load reduction technology, and the first resolution is greater than the second resolution, and the second resolution is greater than the third resolution.

[0086] S213: If it is determined that the motion state of the application element is a stationary state, the image display quality of the current image is kept unchanged.

[0087] It is understandable that when the motion state of the application elements is static, the changes in each frame of the target application display are small or unchanged. When rendering and displaying each frame, the computing pressure of the central processing unit and the graphics processor is relatively small, and there will be no lag caused by insufficient hardware performance. Therefore, there is no need to reduce the display quality of the target application, and a good display effect can be maintained to meet user needs.

[0088] In an embodiment of the present application, when the target application is first run, firstly, multiple frames of the target application when the application element is in different motion states are intercepted, and the camera position matrix is ​​determined based on the difference data between the model view projection matrix of each frame in the multiple frames, and the target buffer corresponding to the camera position matrix is ​​determined, and the graphics drawing instructions stored in the target buffer are the target graphics drawing instructions carrying the camera position matrix. Then, the target graphics drawing instructions stored in the target buffer are monitored in real time, and the camera position matrix for identifying the motion state of the application element is obtained in the target graphics drawing instructions based on the HOOK function, and the motion state of the application element is determined by calculating the difference between the first camera position matrix of the application element in the first frame and the second camera position matrix of the application element in the second frame, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be judged by detecting the camera position matrix between different frames during the operation of the target application, and the screen display quality of the target application can be adjusted according to the motion state of the application element, so as to avoid the jamming problem caused by the rapid movement of the application element and ensure the smooth operation of the target application.

[0089] In one embodiment, when the target application is not running for the first time, the position of the target buffer stored in the camera position matrix used to represent the motion state of the application element may change, but the size of the target buffer will not change. Therefore, when the target application is not running for the first time, the target buffer stored in the camera position matrix needs to be re-determined based on the size of the first buffer determined when the target application is running for the first time. Figure 3 , is a flow chart of a motion state recognition method provided in an embodiment of the present application. Figure 3 The process shown in the figure is described in detail, and the motion state recognition method may include the following steps:

[0090] S301, determining a second buffer having the same size as the first buffer in the buffer set;

[0091] Specifically, when the target application is not running for the first time, a second buffer having the same size as the first buffer is determined among the buffers of the graphics processor based on the size of the first buffer determined when the target application is running for the first time.

[0092] S302, using the second buffer as a target buffer, and using the graphics drawing instruction stored in the target buffer as a target graphics drawing instruction;

[0093] Specifically, for step S302, please refer to the detailed description of step S204 in another embodiment, which will not be described in detail here.

[0094] S303, monitoring the graphics drawing instructions for the target application in real time, and determining whether the buffer area where the graphics drawing instructions are stored is the target buffer area;

[0095] S304, if it is determined that the buffer area where the graphics drawing instruction is stored is the target buffer area, then determining the graphics drawing instruction as the target graphics drawing instruction;

[0096] Specifically, for step S303 and step S304, please refer to the detailed description of step S205 to step S206 in another embodiment, which will not be described in detail here.

[0097] S305, obtaining a first camera position matrix for an application element in a first frame in a target graphics drawing instruction;

[0098] Specifically, after determining the target graphics drawing instruction, the first camera position matrix for the application element in the first frame is obtained from the data transmitted by the target graphics drawing instruction based on the HOOK function.

[0099] The first frame is an application screen of the target application rendered and displayed by the graphics processor based on the target graphics drawing instruction.

[0100] S306, obtaining a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame obtained based on the target application before the first frame, and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0101] Specifically, for step S306, please refer to the detailed description of step S102 in another embodiment, which will not be described in detail here.

[0102] S307, calculating the difference between the first camera position matrix and the second camera position matrix;

[0103] S308, if the difference is zero, determining that the motion state of the application element is a static state;

[0104] S309, if the difference is not zero, determining that the motion state of the application element is a moving state;

[0105] Optionally, the moving state may include a walking state, a sprinting state, and a driving state. In one embodiment, the moving speed of the application element may also be calculated based on the time interval between the first frame and the second frame and the difference between the first camera position matrix and the second camera position matrix. If the moving speed of the application element is greater than zero and less than a first speed threshold, the motion state of the application element is determined to be a walking state; if the moving speed of the application element is greater than the first speed threshold and less than the second speed threshold, the motion state of the application element is determined to be a sprinting state; if the moving speed of the application element is greater than the second speed threshold, the motion state of the application element is determined to be a driving state.

[0106] S310, if it is determined that the motion state of the application element is a moving state, reducing the picture display quality of the current picture based on a load reduction technology;

[0107] It is understandable that when the motion state of an application element is a moving state, the frames displayed in the target application change rapidly along with the movement of the application element. At this time, the frames displayed by the target application change greatly, and the central processing unit and the graphics processor face great computing pressure when rendering and displaying each frame, which is prone to jamming caused by insufficient hardware performance. Therefore, the display quality of the target application can be reduced based on the load reduction technology to alleviate the computing pressure of the central processing unit and the graphics processor and keep the target application running smoothly. When the application element moves quickly, since the frames displayed in the target application change rapidly, the user will not pay too much attention to the display quality, and the reduction in the display quality will not have a significant impact on the user experience.

[0108] Optionally, the moving state may include a walking state, a sprinting state, and a driving state. In one embodiment, if the motion state of the application element is determined to be a walking state, the game image quality is reduced to a first resolution based on a load reduction technology; if the motion state of the application element is determined to be a sprinting state, the game image quality is reduced to a second resolution based on a load reduction technology; if the motion state of the application element is determined to be a driving state, the game image quality is reduced to a third resolution based on a load reduction technology, and the first resolution is greater than the second resolution, and the second resolution is greater than the third resolution.

[0109] S311: If it is determined that the motion state of the application element is a stationary state, the image display quality of the current image is maintained unchanged.

[0110] It is understandable that when the motion state of the application elements is static, the changes in each frame of the target application display are small or unchanged. When rendering and displaying each frame, the computing pressure of the central processing unit and the graphics processor is relatively small, and there will be no lag caused by insufficient hardware performance. Therefore, there is no need to reduce the display quality of the target application, and a good display effect can be maintained to meet user needs.

[0111] In an embodiment of the present application, when the target application is not running for the first time, the size of the first buffer is determined based on the first time the target application is running, and a second buffer with the same size as the first buffer is determined in each buffer of the graphics processor, and the second buffer is determined to be the target buffer corresponding to the camera position matrix, and the graphics drawing instructions stored in the target buffer are the target graphics drawing instructions carrying the camera position matrix. Then, the target graphics drawing instructions stored in the target buffer are monitored in real time, and the camera position matrix for identifying the motion state of the application element is obtained in the target graphics drawing instructions based on the HOOK function, and the motion state of the application element is determined by calculating the difference between the first camera position matrix of the application element in the first frame and the second camera position matrix of the application element in the second frame, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be judged by detecting the camera position matrix between different frames during the operation of the target application, and the screen display quality of the target application can be adjusted according to the motion state of the application element, so as to avoid the jamming problem caused by the rapid movement of the application element and ensure the smooth operation of the target application.

[0112] The following will be combined with the attached Figure 4 , the motion state recognition device provided by the embodiment of the present application is introduced in detail. It should be noted that the attached Figure 4 A motion state recognition device for executing the present application Figure 1 , Figure 2 , Figure 3 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 1 , Figure 2 , Figure 3 The embodiment shown.

[0113] See also Figure 4 , is a schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application. Figure 4 As shown, the motion state recognition device 1 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the motion state recognition device 1 includes a first matrix acquisition module 11, a second matrix acquisition module 12, and a motion state determination module 13, specifically including:

[0114] A first matrix acquisition module 11 is used to acquire a target graphics drawing instruction for a target application, and acquire a first camera position matrix for an application element in a first frame in the target graphics drawing instruction;

[0115] A second matrix acquisition module 12 is used to acquire a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame acquired before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames;

[0116] The motion state determination module 13 is configured to determine the motion state of the application element based on the first camera position matrix and the second camera position matrix.

[0117] Optional, see Figure 5 , is a schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application. Figure 5 As shown, the device further includes a first instruction determining module 14, and the first instruction determining module 14 is specifically configured to:

[0118] Intercepting multiple frames when the application element in the target application is in different motion states, and obtaining difference data between the model view projection matrix of each frame in the multiple frames;

[0119] determining a camera position matrix among a plurality of matrices constituting the model view projection matrix based on the difference data;

[0120] Determine a first buffer in a buffer set where the camera position matrix is ​​located, each buffer in the buffer set being used to store graphics drawing instructions for a target application;

[0121] The first buffer is used as a target buffer, and the graphics drawing instructions stored in the target buffer are used as target graphics drawing instructions.

[0122] Optional, see Figure 5 , is a schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application. Figure 5 As shown, the device further includes a second instruction determining module 15, and the second instruction determining module 15 is specifically configured to:

[0123] Determine a second buffer in the buffer set having the same size as the first buffer;

[0124] The second buffer is used as a target buffer, and the graphics drawing instructions stored in the target buffer are used as target graphics drawing instructions.

[0125] Optionally, the first matrix acquisition module 11 is specifically used to:

[0126] Monitor the graphics drawing instructions for the target application in real time, and determine whether the buffer area where the graphics drawing instructions are stored is the target buffer area;

[0127] If it is determined that the buffer area where the graphics drawing instruction is stored is the target buffer area, then determining the graphics drawing instruction as the target graphics drawing instruction;

[0128] Obtain a first camera position matrix for an application element in a first frame in the target graphics drawing instruction.

[0129] Optional, see Figure 6 , is a schematic diagram of the structure of a motion state determination module provided in an embodiment of the present application. Figure 6 As shown, the motion state determination module 13 includes:

[0130] A difference calculation unit 131, used to calculate the difference between the first camera position matrix and the second camera position matrix;

[0131] A first judging unit 132, configured to determine that the motion state of the application element is a stationary state if the difference is zero;

[0132] The second judgment unit 132 is configured to determine that the motion state of the application element is a moving state if the difference is not zero.

[0133] Optionally, the moving state includes a walking state, a sprinting state and a driving state, and the first judging unit 132 includes:

[0134] Calculating a moving speed of the application element based on a time interval between the first frame and the second frame and the difference;

[0135] If the moving speed of the application element is greater than zero and less than a first speed threshold, determining that the motion state of the application element is a walking state;

[0136] If the moving speed of the application element is greater than the first speed threshold and less than the second speed threshold, determining that the motion state of the application element is a sprinting state;

[0137] If the moving speed of the application element is greater than a second speed threshold, it is determined that the motion state of the application element is a driving state.

[0138] Optional, see Figure 5 , is a schematic diagram of the structure of a motion state recognition device provided in an embodiment of the present application. Figure 5 As shown, the device further includes a picture quality adjustment module 16, and the picture quality adjustment module 16 is specifically used for:

[0139] The screen display quality of the target application is adjusted based on the motion state of the application element.

[0140] Optional, see Figure 7, is a structural diagram of a picture quality adjustment module provided in an embodiment of the present application. Figure 7 As shown, the image quality adjustment module 16 includes:

[0141] The image quality reducing unit 161 is configured to reduce the image display quality of the current image based on a load reduction technology if it is determined that the motion state of the application element is a moving state;

[0142] The image quality maintaining unit 162 is configured to maintain the image display quality of the current image unchanged if it is determined that the motion state of the application element is a stationary state.

[0143] Optionally, the image quality reduction unit 161 is specifically configured to:

[0144] If it is determined that the motion state of the application element is a walking state, reducing the game image quality to a first resolution based on a load reduction technology;

[0145] If it is determined that the motion state of the application element is a sprinting state, reducing the game image quality to a second resolution based on a load reduction technology;

[0146] If it is determined that the motion state of the application element is a driving state, reducing the game image quality to a third resolution based on a load reduction technology;

[0147] The first resolution is greater than the second resolution, and the second resolution is greater than the third resolution.

[0148] In one or more embodiments of the present application, a target graphics drawing instruction for a target application sent by a central processing unit to a graphics processing unit is obtained, a first camera position matrix for an application element in a first frame in the target graphics drawing instruction is obtained, and then a second camera position matrix for the application element in a recorded second frame is obtained, where the second frame is a frame obtained before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames, and finally the motion state of the application element is determined based on the first camera position matrix and the second camera position matrix, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be judged by the camera position matrix between different frames during the operation of the target application, and the screen display quality of the target application can be adjusted according to the motion state of the application element, thereby avoiding the jamming problem caused by the rapid movement of the application element and ensuring the smooth operation of the target application.

[0149] It should be noted that the motion state recognition device provided in the above embodiment only uses the division of the above functional modules as an example when executing the motion state recognition method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the motion state recognition device provided in the above embodiment and the motion state recognition method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0150] It should be noted that the motion state recognition device provided in the above embodiment only uses the division of the above functional modules as an example when executing the motion state recognition method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the motion state recognition device provided in the above embodiment and the motion state recognition method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0151] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0152] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1 to Figure 3 The motion state recognition method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 to Figure 3 The specific description of the illustrated embodiment will not be repeated here.

[0153] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1 to Figure 3 The motion state recognition method of the embodiment shown in the figure can be specifically executed by referring to Figure 1 to Figure 3 The specific description of the illustrated embodiment will not be repeated here.

[0154] Please refer to Figure 8 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via a bus 150.

[0155] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the terminal 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user pages, and applications; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0156] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium (Non-Transitory Computer-Readable Storage Medium). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems.

[0157] The memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, the requirements for system resources in different application scenarios in the same third-party application are also different. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0158] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0159] The input device 130 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output commands or data, and includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays.

[0160] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0161] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0162] exist Figure 8 In the electronic device shown, the processor 110 can be used to call the motion state recognition program stored in the memory 120 and execute it to implement the motion state recognition method as described in the various method embodiments of the present application.

[0163] In an embodiment of the present application, a target graphics drawing instruction for a target application sent by a central processing unit to a graphics processing unit is obtained, a first camera position matrix for an application element in a first frame in the target graphics drawing instruction is obtained, and then a second camera position matrix for the application element in a recorded second frame is obtained, where the second frame is a frame obtained before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames, and finally the motion state of the application element is determined based on the first camera position matrix and the second camera position matrix, so that when the target application client does not send any information such as the action and time of the application element, the motion state of the application element in the target application can be judged by the camera position matrix between different frames during the operation of the target application, and the screen display quality of the target application can be adjusted according to the motion state of the application element, thereby avoiding the jamming problem caused by the rapid movement of the application element and ensuring the smooth operation of the target application.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0165] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A motion state recognition method, characterized in that: The method comprises: Obtain a target graphics drawing instruction for a target application, and obtain a first camera position matrix for an application element in a first frame in the target graphics drawing instruction; the first camera position matrix is ​​one or more matrices in a model view projection matrix used by a shader in a graphics processor when rendering the application element in the first frame, and represents a motion state of the application element in the application screen; Acquire a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame acquired before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames; A motion state of the application element is determined based on the first camera position matrix and the second camera position matrix.

2. The method according to claim 1, characterized in that When the target application is run for the first time, before obtaining the target graphics drawing instruction for the target application and obtaining the first camera position matrix for the application element in the first frame in the target graphics drawing instruction, the method further includes: Intercepting multiple frames when the application element in the target application is in different motion states, and obtaining difference data between the model view projection matrix of each frame in the multiple frames; determining a camera position matrix among a plurality of matrices constituting the model view projection matrix based on the difference data; Determine a first buffer in a buffer set where the camera position matrix is ​​located, each buffer in the buffer set being used to store graphics drawing instructions for a target application; The first buffer is used as a target buffer, and the graphics drawing instructions stored in the target buffer are used as target graphics drawing instructions.

3. The method according to claim 2, characterized in that When the target application is not running for the first time, before obtaining the target graphics drawing instruction for the target application and obtaining the first camera position matrix for the application element in the first frame in the target graphics drawing instruction, the method further includes: Determine a second buffer in the buffer set having the same size as the first buffer; The second buffer is used as a target buffer, and the graphics drawing instructions stored in the target buffer are used as target graphics drawing instructions.

4. The method according to any one of claims 2 to 3, characterized in that: The step of obtaining a target graphics drawing instruction for a target application and obtaining a first camera position matrix for an application element in a first frame in the target graphics drawing instruction includes: Monitor the graphics drawing instructions for the target application in real time, and determine whether the buffer area where the graphics drawing instructions are stored is the target buffer area; If it is determined that the buffer area where the graphics drawing instruction is stored is the target buffer area, then determining the graphics drawing instruction as the target graphics drawing instruction; Obtain a first camera position matrix for an application element in a first frame in the target graphics drawing instruction.

5. The method according to claim 1, characterized in that The determining the motion state of the application element based on the first camera position matrix and the second camera position matrix includes: Calculating a difference between the first camera position matrix and the second camera position matrix; If the difference is zero, determining that the motion state of the application element is a stationary state; If the difference is not zero, it is determined that the motion state of the application element is a moving state.

6. The method according to claim 5, characterized in that The moving state includes a walking state, a sprinting state and a driving state. If the difference is not zero, determining that the motion state of the application element is a moving state includes: Calculating a moving speed of the application element based on a time interval between the first frame and the second frame and the difference; If the moving speed of the application element is greater than zero and less than a first speed threshold, determining that the motion state of the application element is a walking state; If the moving speed of the application element is greater than the first speed threshold and less than the second speed threshold, determining that the motion state of the application element is a sprinting state; If the moving speed of the application element is greater than a second speed threshold, it is determined that the motion state of the application element is a driving state.

7. The method according to claim 1, characterized in that The method further comprises: The screen display quality of the target application is adjusted based on the motion state of the application element.

8. The method according to claim 7, characterized in that The adjusting the screen display quality of the target application based on the motion state of the application element includes: If it is determined that the motion state of the application element is a moving state, reducing the picture display quality of the current picture based on a load reduction technology; If it is determined that the motion state of the application element is a stationary state, the picture display quality of the current picture is kept unchanged.

9. The method according to claim 8, characterized in that If it is determined that the motion state of the application element is a moving state, reducing the picture display quality of the current picture based on a load reduction technology includes: If it is determined that the motion state of the application element is a walking state, reducing the game image quality to a first resolution based on a load reduction technology; If it is determined that the motion state of the application element is a sprinting state, reducing the game image quality to a second resolution based on a load reduction technology; If it is determined that the motion state of the application element is a driving state, reducing the game image quality to a third resolution based on a load reduction technology; The first resolution is greater than the second resolution, and the second resolution is greater than the third resolution.

10. A motion state recognition device, characterized in that: The device comprises: A first matrix acquisition module is used to acquire a target graphics drawing instruction for a target application, and acquire a first camera position matrix for an application element in a first frame in the target graphics drawing instruction; the first camera position matrix is ​​one or more matrices in a model view projection matrix used by a shader in a graphics processor when rendering the application element in the first frame, and represents a motion state of the application element in the application screen; A second matrix acquisition module, used to acquire a second camera position matrix of the application element in the recorded second frame, where the second frame is a frame acquired before the first frame based on the target application and the number of frames separated from the first frame is less than or equal to a preset number of frames; A motion state determination module is used to determine the motion state of the application element based on the first camera position matrix and the second camera position matrix.

11. A storage medium having a plurality of instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

12. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as described in any one of claims 1 to 9.

Images