User agreement processing method and apparatus, storage medium, and electronic device

Abstract

The present disclosure relates to a user protocol processing method and device, a storage medium and an electronic device, and relates to the technical field of human-computer interaction. The method comprises: obtaining input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal; determining a to-be-switched application scenario corresponding to the input image information according to a first association relationship between preset image information and a preset application scenario; determining a to-be-switched user protocol corresponding to the to-be-switched application scenario according to a second association relationship between the preset application scenario and a preset user protocol; and displaying the to-be-switched user protocol on the display interface. The method realizes automatic switching of multiple user protocols.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of human-computer interaction, and in particular, to a user protocol processing method, a user protocol processing apparatus, a computer readable storage medium, and an electronic device. BACKGROUND

[0002] With the popularity of large-size intelligent terminals, various intelligent terminal devices need to configure different user protocols for different users and / or different application scenarios.

[0003] However, the prior art cannot achieve multi-application scenario switching and / or multi-user protocol switching.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present disclosure is to provide a user protocol processing method, a user protocol processing apparatus, a computer readable storage medium, and an electronic device, thereby at least partially overcoming the problem of being unable to achieve multi-application scenario switching due to the limitations and defects of the related art.

[0006] According to one aspect of the present disclosure, a user protocol processing method is provided, comprising:

[0007] obtaining input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal;

[0008] determining a to-be-switched application scenario corresponding to the input image information according to a first association relationship between preset image information and preset application scenarios;

[0009] determining a to-be-switched user protocol corresponding to the to-be-switched application scenario according to a second association relationship between the preset application scenario and preset user protocols;

[0010] displaying the to-be-switched user protocol on the display interface.

[0011] In an exemplary embodiment of the present disclosure, obtaining input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal comprises:

[0012] determining a starting position node, an intermediate position node, and a terminal position node of the input image information in response to a touch event of a current user acting on a preset coordinate point included in a display panel of the display terminal;

[0013] The input image information is generated according to the start position node, the intermediate position node, the end position node, and a sequence of track points between the start position node, the intermediate position node, and the end position node.

[0014] In an example embodiment of the present disclosure, the application scenario to be switched includes at least one of a normal mode scenario, an office mode scenario, a child / old person mode scenario, a safety mode scenario, and a custom mode scenario.

[0015] Therefore, the user protocol to be switched includes at least one of a first user protocol corresponding to the normal mode scenario, a second user protocol corresponding to the office mode scenario, a third user protocol corresponding to the child / old person mode scenario, a fourth user protocol corresponding to the safety mode scenario, and a fifth user protocol corresponding to the custom mode scenario.

[0016] In an example embodiment of the present disclosure, displaying the user protocol to be switched includes:

[0017] When the application scenario to be switched is the normal mode scenario, and the user protocol to be switched is the first user protocol, the first user protocol is displayed in a first preset display manner in a first preset area of the display interface, and a first preset time required for previewing the first user protocol is displayed in a timing manner in a second preset area of the display interface.

[0018] When the application scenario to be switched is the office mode scenario, and the user protocol to be switched is the second user protocol, the second user protocol is displayed in a second preset display manner in the first preset area of the display interface, and a second preset time required for previewing the second user protocol is displayed in a timing manner in the second preset area of the display interface.

[0019] When the application scenario to be switched is the child / old person mode scenario, and the user protocol to be switched is the third user protocol, the third user protocol is displayed in a third preset manner in the first preset area of the display interface, and the third user protocol is broadcasted.

[0020] When the application scenario to be switched is the safety mode scenario, and the user protocol to be switched is the fourth user protocol, the fourth user protocol is displayed in a fourth preset display manner in the first preset area of the display interface.

[0021] In an example embodiment of the present disclosure, before obtaining the input image information in response to a touch event acting on a preset coordinate point on the display interface of the display terminal, the method for processing the user protocol further includes:

[0022] obtaining a gesture depth map of a gesture to be detected, and calculating current point cloud data possessed by the gesture to be detected according to the gesture depth map;

[0023] matching a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data;

[0024] displaying the preset coordinate point on a display interface of the display terminal when it is determined that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space.

[0025] In an exemplary embodiment of the present disclosure, after the determination that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space, the processing method of the user protocol further includes:

[0026] determining a current application scenario corresponding to the target geometric gesture according to a first association relationship between preset image information and preset application scenarios;

[0027] obtaining a current user protocol corresponding to the current application scenario, and displaying the current user protocol.

[0028] In an exemplary embodiment of the present disclosure, matching a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data includes:

[0029] calculating, based on a preset image processing model, a distance difference between a three-dimensional point coordinate in the current point cloud data and a model surface of a standard geometric gesture included in the preset gesture search space;

[0030] taking a minimum value from the distance difference, and taking the standard geometric gesture corresponding to the minimum value as the target geometric gesture.

[0031] In an exemplary embodiment of the present disclosure, the standard geometric gesture includes a current frame standard geometric gesture, a previous frame standard geometric gesture corresponding to the current frame standard geometric gesture, and a next frame standard geometric gesture corresponding to the current frame standard geometric gesture;

[0032] The calculation of the distance difference between the three-dimensional point coordinate in the current point cloud data and the model surface of the standard geometric gesture included in the preset gesture search space based on the preset image processing model includes:

[0033] initializing a to-be-trained image processing model to obtain to-be-optimized parameters included in the to-be-trained image processing model;

[0034] inputting the current point cloud data of the gesture depth map and hand pose parameters of the last frame of standard geometric gesture into the image processing model to be trained to obtain a predicted distance between the gesture depth map and the last frame of standard geometric gesture;

[0035] constructing a loss function according to an actual distance and the predicted distance between the gesture depth map and the last frame of standard geometric gesture, and optimizing the to-be-optimized parameters according to a preset optimization algorithm and the loss function;

[0036] updating the image processing model to be trained according to the optimized parameters to obtain a trained image processing model, and calculating a distance difference between the current frame of standard geometric gesture and / or the next frame of standard geometric gesture and the gesture depth map by using the trained image processing model.

[0037] In an exemplary embodiment of the present disclosure, the preset optimization algorithm includes a particle swarm optimization algorithm and / or a nearest point optimization algorithm.

[0038] When the preset optimization algorithm is the particle swarm optimization algorithm, optimizing the to-be-optimized parameters according to the preset optimization algorithm and the loss function includes:

[0039] generating a particle swarm according to the to-be-optimized parameters, and randomly setting a starting position and a starting speed of each particle in the particle swarm; wherein each to-be-optimized parameter corresponds to one particle.

[0040] calculating the fitness of each particle according to the loss function, and comparing the fitness of each particle at the current position with the fitness of each particle at the best position; if the fitness of each particle at the current position is better than the fitness of each particle at the best position, the current position is taken as the best position of the individual, otherwise, the best position of the individual remains unchanged.

[0041] comparing the fitness of each particle at the current position with the fitness of the best position of the population; if the fitness of each particle at the current position is better than the fitness of the best position of the population, the current position is taken as the best global position, otherwise, the best global position remains unchanged.

[0042] updating the speed and position of each particle in the particle swarm according to the best position of the individual, the best global position and the starting speed, so as to optimize the to-be-optimized parameters.

[0043] According to an aspect of the present disclosure, a processing device for user agreement is provided, including:

[0044] An input image information generation module is configured to obtain input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal.

[0045] The application scenario to be switched determination module is configured to determine an application scenario to be switched corresponding to the input image information according to a first association relationship between preset image information and preset application scenarios.

[0046] The user protocol to be switched determination module is configured to determine a user protocol to be switched corresponding to the application scenario to be switched according to a second association relationship between the preset application scenario and preset user protocols.

[0047] The user protocol to be switched display module is configured to display the user protocol to be switched on the display interface.

[0048] In an exemplary embodiment of the present disclosure, input image information is obtained in response to a touch event acting on a preset coordinate point on a display interface of a display terminal, and the input image information comprises:

[0049] In response to a touch event of a current user acting on a preset coordinate point included in a display panel of the display terminal, a starting position node, an intermediate position node and a terminal position node of the input image information are determined.

[0050] The input image information is generated according to the starting position node, the intermediate position node, the terminal position node and a track point sequence between the starting position node, the intermediate position node and the terminal position node.

[0051] In an exemplary embodiment of the present disclosure, the application scenario to be switched comprises at least one of a normal mode scenario, an office mode scenario, a child / old person mode scenario, a security mode scenario and a custom mode scenario.

[0052] Therefore, the user protocol to be switched comprises at least one of a first user protocol corresponding to the normal mode scenario, a second user protocol corresponding to the office mode scenario, a third user protocol corresponding to the child / old person mode scenario, a fourth user protocol corresponding to the security mode scenario and a fifth user protocol corresponding to the custom mode scenario.

[0053] In an exemplary embodiment of the present disclosure, displaying the user protocol to be switched comprises:

[0054] When the application scenario to be switched is the normal mode scenario and the user protocol to be switched is the first user protocol, the first user protocol is displayed in a first preset display mode in a first preset area of the display interface, and a first preset time required for previewing the first user protocol is displayed by timing in a second preset area of the display interface.

[0055] When the application scenario to be switched is an office mode scenario, and the user protocol to be switched is a second user protocol, the second user protocol is displayed in a second preset display mode in a first preset area of the display interface, and a second preset time required for previewing the second user protocol is displayed in a timing manner in a second preset area of the display interface.

[0056] When the application scenario to be switched is a child / old person mode scenario, and the user protocol to be switched is a third user protocol, the third user protocol is displayed in a third preset manner in the first preset area of the display interface, and the third user protocol is broadcasted.

[0057] When the application scenario to be switched is a security mode scenario, and the user protocol to be switched is a fourth user protocol, the fourth user protocol is displayed in a fourth preset display mode in the first preset area of the display interface.

[0058] In an example embodiment of the present disclosure, the processing apparatus of the user protocol further includes:

[0059] The current point cloud data calculation module can be configured to obtain a gesture depth map of a gesture to be detected, and calculate current point cloud data possessed by the gesture to be detected according to the gesture depth map.

[0060] The target geometric gesture determination module can be configured to match a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data.

[0061] The preset coordinate point display module can be configured to display the preset coordinate point on a display interface of the display terminal when it is determined that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space.

[0062] In an example embodiment of the present disclosure, the processing apparatus of the user protocol further includes:

[0063] The current application scenario determination module can be configured to determine a current application scenario corresponding to the target geometric gesture according to a first association relationship between preset image information and preset application scenarios.

[0064] The current user protocol display module can be configured to obtain a current user protocol corresponding to the current application scenario, and display the current user protocol.

[0065] In an example embodiment of the present disclosure, matching a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data includes:

[0066] calculate, based on a preset image processing model, a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space;

[0067] extract a minimum value from the distance difference value, and take the standard geometric gesture corresponding to the minimum value as the target geometric gesture.

[0068] In an example embodiment of the present disclosure, the standard geometric gesture includes a current frame standard geometric gesture, a previous frame standard geometric gesture corresponding to the current frame standard geometric gesture, and a next frame standard geometric gesture corresponding to the current frame standard geometric gesture;

[0069] The calculating, based on a preset image processing model, of a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space includes:

[0070] initializing the image processing model to be trained to obtain a to-be-optimized parameter included in the image processing model to be trained;

[0071] inputting the current point cloud data of the gesture depth map and the hand pose parameter of the previous frame standard geometric gesture into the image processing model to be trained to obtain a predicted distance between the gesture depth map and the previous frame standard geometric gesture;

[0072] constructing a loss function according to an actual distance and the predicted distance between the gesture depth map and the previous frame standard geometric gesture, and optimizing the to-be-optimized parameter according to a preset optimization algorithm and the loss function;

[0073] updating the image processing model to be trained according to the optimized parameter to obtain a trained image processing model, and calculating a distance difference value between the current frame standard geometric gesture and / or the next frame standard geometric gesture and the gesture depth map by using the trained image processing model.

[0074] In an example embodiment of the present disclosure, the preset optimization algorithm includes a particle swarm optimization algorithm and / or a nearest point optimization algorithm;

[0075] When the preset optimization algorithm is the particle swarm optimization algorithm, the optimizing the to-be-optimized parameter according to the preset optimization algorithm and the loss function includes:

[0076] generating a particle population according to the to-be-optimized parameter, and randomly setting a starting position and a starting speed of each particle in the particle population; wherein each to-be-optimized parameter corresponds to one particle;

[0077] According to the loss function, fitness of each particle is calculated, and fitness of each particle at current position and fitness of each particle at best position are compared, if fitness of each particle at current position is better than fitness of each particle at best position, current position is taken as individual best position, if not, it is unchanged;

[0078] Fitness of each particle at current position and fitness of population best position are compared, if fitness of each particle at current position is better than fitness of population best position, current position is taken as global best position, if not, it is unchanged;

[0079] According to the individual best position, global best position and the initial speed, speed and position of each particle in the particle swarm are updated, so as to realize optimization of the to-be-optimized parameters.

[0080] According to an aspect of the present disclosure, a computer readable storage medium is provided, and a computer program is stored on the computer readable storage medium, and the computer program is executed by a processor to implement the user protocol processing method of any one of the above.

[0081] According to an aspect of the present disclosure, an electronic device is provided, comprising:

[0082] a processor; and

[0083] a memory for storing executable instructions of the processor;

[0084] The processor is configured to execute the user protocol processing method of any one of the above by executing the executable instructions.

[0085] The user protocol processing method provided by the present disclosure has the following advantages. On the one hand, the input image information can be obtained by responding to the touch event of the preset coordinate point on the display interface of the display terminal. Then, the to-be-switched application scenario corresponding to the input image information is determined according to the first association relationship between the preset image information and the preset application scenario. Then, the to-be-switched user protocol corresponding to the to-be-switched application scenario is determined according to the second association relationship between the preset application scenario and the preset user protocol. Finally, the to-be-switched user protocol is displayed on the display interface, realizing the switching of multiple application scenarios and multiple user protocols, and without manually switching the application scenario and / or the user protocol, the problem that the switching efficiency is low and the accuracy of the switching result cannot be ensured in the prior art is solved. On the other hand, the to-be-switched application scenario is determined according to the input image information, so that the safety during the switching of the application scenario is ensured, and the safety of the user protocol during the switching is ensured.

[0086] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0087] The accompanying drawings, which are incorporated in and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. It is appreciated that the accompanying drawings are only some embodiments of the present disclosure, and other drawings can be obtained by those of ordinary skill in the art without creative effort on the basis of the drawings.

[0088] Figure 1 An example diagram schematically shows a scenario of a user protocol display.

[0089] Figure 2 An example flowchart schematically shows a processing method of a user protocol according to an example embodiment of the present disclosure.

[0090] Figure 3 An example flowchart schematically shows a gesture detection and recognition method according to an example embodiment of the present disclosure.

[0091] Figure 4 An example diagram schematically shows 21 3D key points of a hand according to an example embodiment of the present disclosure.

[0092] Figure 5 An example diagram schematically shows a generation process scenario of a target geometric gesture according to an example embodiment of the present disclosure.

[0093] Figure 6 An example flowchart schematically shows a method of calculating a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space based on a preset image processing model according to an example embodiment of the present disclosure.

[0094] Figure 7 An example diagram schematically shows a calculation process of a specific difference value by an image processing model according to an example embodiment of the present disclosure.

[0095] Figure 8 An example diagram schematically shows a preset coordinate point on a display interface according to an example embodiment of the present disclosure.

[0096] Figure 9 An example diagram schematically shows input image information according to an example embodiment of the present disclosure.

[0097] Figure 10 An example diagram schematically shows an application scenario to be switched according to an example embodiment of the present disclosure.

[0098] Figure 11 An example diagram illustrating displaying a user protocol to be switched according to an example embodiment of the present disclosure is schematically shown.

[0099] Figure 12 A flowchart illustrating another method of processing a user protocol according to an example embodiment of the present disclosure is schematically shown.

[0100] Figure 13 A block diagram illustrating a processing apparatus of a user protocol according to an example embodiment of the present disclosure is schematically shown.

[0101] Figure 14 An electronic device for implementing the above-mentioned method of processing a user protocol according to an example embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0102] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations. In the following description, numerous specific details are provided to give a thorough understanding of example implementations. One skilled in relevant art will recognize, however, that the implementations can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures have not been described in detail to avoid obscuring the understanding of this description. The sole purpose of the description is to enable a person skilled in the art to make, use, and practice the claimed implementations.

[0103] In addition, the drawings are to be regarded as being schematic only. Like references indi- cate similar or like functionalities, so that their description will be omitted here. Some of the block diagrams shown in the drawings are functional entities, which do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor apparatuses and / or microcontroller apparatuses.

[0104] With the popularity of large-size intelligent terminals, various intelligent terminal devices need to configure different user protocols for different users and / or different application scenarios. Therefore, a simple and single user protocol cannot meet multiple different application scenarios. Under the premise that different application scenarios require different user protocols, developers often need to embed specific user protocols into software to solve this problem, and the corresponding user protocol pops up when the software is opened. Moreover, special scene user protocols need to be specially customized. Often, a large amount of manpower and material resources are needed to coordinate the reminder function, and according to the requirements in the actual application process, the user protocol is essential, and its importance is as strong as a mandatory requirement for product instructions.

[0105] Further, in the conventional technical solution, the display of the user protocol can be displayed in the manner shown in Figure 1 However, the conventional method does not combine AI (Artificial Intelligence) functions, cannot adapt to user protocols for multiple application scenarios, cannot meet some special requirements, cannot have different user protocol requirements for multiple scenarios, and has a single user protocol problem.

[0106] Based on this, the present example embodiment first provides a user protocol processing method, which can run on a display terminal. The display terminal can include an intelligent interactive panel, which can be applied to various scenarios such as conferences, teaching, home use, and commercial use, and has interactive functions such as touch, a large size (for example, generally greater than 50 inches), and is installed with software such as handwriting, wireless projection, and remote video conferencing. Of course, those skilled in the art can also run the method of the present disclosure on other platforms according to requirements, and this example embodiment does not specially limit this. Referring to Figure 2 The user protocol processing method can include the following steps:

[0107] Step S210. In response to a touch event acting on a preset coordinate point on a display interface of a display terminal, input image information is obtained;

[0108] Step S220. According to a first association relationship between preset image information and a preset application scenario, a to-be-switched application scenario corresponding to the input image information is determined;

[0109] Step S230. According to a second association relationship between the preset application scenario and a preset user protocol, a to-be-switched user protocol corresponding to the to-be-switched application scenario is determined;

[0110] Step S240. The to-be-switched user protocol is displayed on the display interface.

[0111] In the aforementioned user protocol processing method, on the one hand, input image information can be obtained by responding to touch events acting on preset coordinate points on the display interface of the display terminal; then, based on the first association relationship between the preset image information and the preset application scenario, the application scenario to be switched corresponding to the input image information is determined; then, based on the second association relationship between the preset application scenario and the preset user protocol, the user protocol to be switched corresponding to the application scenario to be switched is determined; finally, the user protocol to be switched is displayed on the display interface, realizing the switching of multiple application scenarios and multiple user protocols, without the need for manual switching of application scenarios and / or user protocols, solving the problem of low switching efficiency and inability to guarantee the accuracy of switching results in the prior art due to manual switching methods; on the other hand, since the process of determining the application scenario to be switched is based on the input image information, the security of the application scenario switching process can be ensured, thereby ensuring the security of the user protocol during the switching process.

[0112] The following will provide a detailed explanation and description of the user agreement processing method in the example embodiments of this disclosure, in conjunction with the accompanying drawings.

[0113] First, the inventive purpose and application scenarios of the exemplary embodiments of this disclosure will be explained and described. Specifically, the user agreement processing method described in the exemplary embodiments of this disclosure realizes a user agreement that adapts to multiple scenarios by combining AI gesture recognition, without having to pop up a special user agreement separately in applications for special scenarios, thus reducing the workload of developing system applications and meeting the user's need for customized user agreements; at the same time, it can also be customized for different needs.

[0114] The user agreement processing method described in the exemplary embodiments of this disclosure can integrate various user agreements of different modes in a manner similar to SDK (Software Development Kit) integration, storing them in the cloud corresponding to the display terminal or in the local memory of the display terminal. Furthermore, since different products require different protocols, this model allows for a single installation package to be compatible with different products and scenarios, eliminating the need for specific design.

[0115] Secondly, the gesture detection and recognition methods described in the exemplary embodiments of this disclosure will be explained and illustrated. Specifically, refer to... Figure 3 As shown, the specific gesture detection and recognition process may include the following steps:

[0116] Step S310: Obtain the gesture depth map of the gesture to be detected, and calculate the current point cloud data of the gesture to be detected based on the gesture depth map.

[0117] In this example embodiment, firstly, a gesture depth map of the gesture to be detected can be acquired using an image acquisition device. This image acquisition device can be a depth image acquisition device, which can be embedded in the display terminal or set up independently; this example does not impose any special restrictions on this. Furthermore, after acquiring the gesture depth map, it can be input into an algorithm (e.g., MediaPipe). The algorithm can output the 3D positions of all key points in the gesture depth map, and obtain the current point cloud data based on the 3D positions of each key point. Here, hand key points can also be understood as the joints of the hand skeleton, typically described using 21 3D key points (see reference for details). Figure 4 (As shown).

[0118] Step S320: Match the target geometric gesture for the gesture to be detected in the preset gesture search space based on the current point cloud data.

[0119] In this example embodiment, firstly, based on a preset image processing model, the distance difference between the 3D point coordinates in the current point cloud data and the model surface of the standard geometric gesture included in the preset gesture search space is calculated; secondly, the minimum value is extracted from the distance difference, and the standard geometric gesture corresponding to the minimum value is taken as the target geometric gesture. That is, in the process of target geometric gesture matching, it can be achieved by calculating the distance difference between the 3D point coordinates in the current point cloud data and the model surface of the standard geometric gesture; wherein, the specific generation of the standard geometric gesture may include: firstly, generating a series of standard geometric gesture models of the hand through hand pose (pose can refer to the hand's pose parameters or node position), and then establishing a search space based on the generated standard geometric gesture models; wherein, in the process of generating the standard geometric gesture model, it can be implemented by linear blend skinning (skeletal skinning animation algorithm), the specific implementation principle of which is: to cover the hand skeleton with a layer of skin and let the skin change with the movement of the skeleton, which is mostly used in the field of animation; in the specific generation process, refer to Figure 5 As shown, the pose can first be converted into the corresponding mesh. Figure 5 The left part of the model is further transformed into a smooth surface model. Figure 5 The right side of the image is used to obtain the standard geometric gesture model. Furthermore, during the generation process, pose can be used as the independent variable. The standard geometric gesture model can be calculated from pose, and the labeled geometric gesture model corresponds one-to-one with pose.

[0120] Further, when the standard geometric gesture is obtained, a specific matching process can be implemented; in the specific matching process, the standard geometric gesture can be divided into a current frame standard geometric gesture, a previous frame standard geometric gesture corresponding to the current frame standard geometric gesture, and a next frame standard geometric gesture corresponding to the current frame standard geometric gesture. Specifically, referring to Figure 6 As shown, based on the preset image processing model, calculating the distance difference value of the three-dimensional point coordinates in the current point cloud data to the model surface of the standard geometric gesture included in the preset gesture search space can include the following steps:

[0121] In step S610, the image processing model to be trained is initialized to obtain the to-be-optimized parameters included in the image processing model to be trained.

[0122] In step S620, the current point cloud data of the gesture depth map and the hand pose parameters of the previous frame standard geometric gesture are input into the image processing model to be trained to obtain the predicted distance between the gesture depth map and the previous frame standard geometric gesture.

[0123] In step S630, a loss function is constructed according to the actual distance and the predicted distance between the gesture depth map and the previous frame standard geometric gesture, and the to-be-optimized parameters are optimized according to the preset optimization algorithm and the loss function.

[0124] In step S640, the image processing model to be trained is updated according to the optimized parameters to obtain a trained image processing model, and the distance difference value between the current frame standard geometric gesture and / or the next frame standard geometric gesture and the gesture depth map is calculated using the trained image processing model.

[0125] Furthermore, the preset optimization algorithm includes particle swarm optimization algorithm and / or nearest point optimization algorithm. Furthermore, when the preset optimization algorithm is a particle swarm optimization algorithm, the optimization of the parameters to be optimized according to the preset optimization algorithm and the loss function can be achieved in the following way: First, a particle swarm is generated according to the parameters to be optimized, and the starting position and starting velocity of each particle in the particle swarm are randomly set; wherein, each parameter to be optimized corresponds to one particle; second, the fitness of each particle is calculated according to the loss function, and the fitness of each particle at the current position is compared with its fitness at the best position. If the fitness of the particle at the current position is better than its fitness at the best position, then the current position is taken as the individual best position; otherwise, it remains unchanged; then, the fitness of each particle at the current position is compared with the fitness of the swarm's best position. If the fitness of the particle at the current position is better than its fitness at the swarm's best position, then the current position is taken as the global best position; otherwise, it remains unchanged; finally, the velocity and position of the particles in the particle swarm are updated according to the individual best position, the global best position, and the starting velocity to achieve the optimization of the parameters to be optimized.

[0126] In one example embodiment, the basic idea of ​​the PSO (Particle Swarm Optimization) algorithm described above is explained and illustrated. Specifically, the basic idea of ​​the PSO algorithm is to accelerate each particle towards its own and the swarm's optimal position, randomly setting the initial position and velocity of particles in space. During the iterative search process, the algorithm records the optimal positions experienced by individual particles and the swarm, as well as the corresponding fitness function values. In this disclosure, it is assumed that the particle search space is n-dimensional, and the entire particle swarm... It contains m particles, and the position of the i-th example is:

[0127] ;

[0128] The particle's velocity at this moment is:

[0129] ;

[0130] The individual extreme value of the particle is:

[0131] ;

[0132] The global extremum of the population is:

[0133] .

[0134] When the particle finds the individual optimal and global optimal position, the speed and position information of itself can be updated according to the following formula (1) and formula (2):

[0135] Formula (1)

[0136] Formula (2)

[0137] wherein, is an inertia weight, determines how much the current speed of the particle is inherited, and are learning factors; is a random number between 0 and 1; and are the speed and position of the particle i in the d dimension in the k iteration, respectively; is the individual extreme value position of a single particle, is the position of the global extreme value of all particles. is the number of cycles, and the iteration termination condition can be set according to a specific problem, and is usually set to reach the maximum iteration number K or the optimal position searched by the population meets the expected value.

[0138] The optimization specific steps of the PSO algorithm are as follows: first, the starting position and the speed of the particle are randomly set, the number of populations is set according to the problem to be solved, and the parameters to be adjusted are set; second, the fitness of each particle is calculated according to the formula of the fitness function, and the fitness of the current position of each particle is compared with the fitness value of the best position , if it is better, the current position is taken as , otherwise remains unchanged; then, the fitness of the current position of each particle is compared with the fitness value of the best position of the population, if it is better, it is taken as the current global best position ; finally, the speed and position of the particle are updated according to the formula; of course, if the termination condition preset by the algorithm is not met, the fitness of each particle is calculated; if the termination condition is met, the loop is ended and the best position information is output.

[0139] It needs to be supplemented here that the particle swarm optimization algorithm is used to optimize the parameters of the image processing model, which can greatly shorten the network training time and solve the local optimal problem of the traditional back propagation (BP) optimization algorithm; wherein the specific calculation process of the specific difference of the image processing model can be referred to as shown in Figure 7 .

[0140] It needs to be further supplemented here that in the process of processing the to-be-trained image processing model, the input of the to-be-trained image processing model is the gesture depth map and the pose of the standard geometric gesture, and the output is the predicted distance between the gesture depth map and the standard geometric gesture. Then the loss function can be constructed according to the predicted distance and the actual distance. The smaller the loss function is, the more similar the input gesture depth map and the standard geometric gesture are. At the same time, in the specific matching process, as long as the pose with the minimum predicted distance value is found in the search space, it is the pose (i.e. the target geometric gesture) to be solved. However, because the search space cannot be written in an analytical form, it is not possible to obtain the minimum distance difference value at one time. Therefore, in the training process, optimization based on the corresponding optimization algorithm is needed to obtain the optimal solution through continuous iteration.

[0141] At the same time, since the iterative numerical solution method usually has high requirements for initialization, if the initialization is not good, it will take a long time to converge through iteration, and it may not be able to converge to the global minimum value (because the loss function is a non-convex function). Therefore, when implementing the algorithm, the pose of the standard geometric gesture of the previous frame is usually used to initialize the standard geometric gesture of the current frame to implement the specific calculation process. The specific optimization process and calculation process are not described here. At the same time, the specific optimization purpose of the ICP optimization algorithm is the same as that of the PSO optimization algorithm, and it is not necessary to describe it here.

[0142] Step S330, when it is determined that the target geometric gesture corresponding to the to-be-detected gesture exists in the preset gesture search space, displaying the preset coordinate point on the display interface of the display terminal.

[0143] At this point, the target geometric gesture has been matched. When the target geometric gesture is matched, the preset coordinate point can be displayed on the display interface. The displayed preset coordinate point can be specifically referred to as shown in Figure 8 When the preset coordinate point is displayed, the gesture password of the application scene to be switched can be drawn to implement application scene switching and / or user agreement switching.

[0144] In an example embodiment, if switching is not needed, the current user agreement can also be directly displayed. The specific display process can include: determining a current application scenario corresponding to the target geometric gesture according to a first association relationship between preset image information and preset application scenarios; obtaining a current user agreement corresponding to the current application scenario, and displaying the current user agreement. It should be noted that the first association relationship between image information and application scenarios is preset; the second association relationship between application scenarios and user agreements is also preset; the user can draw corresponding image information to switch from the current application scenario to a desired application scenario, and then obtain and display the user agreement possessed by the switched application scenario. Through this method, the user experience can be improved on the basis of realizing automatic switching of multiple scenarios and multiple user agreements.

[0145] In the following, the processing method of the user agreement shown in Figures 3-8 will be further explained and described. Specifically: Figure 2

[0146] In step S210, in response to a touch event acting on a preset coordinate point on the display interface of the display terminal, input image information is obtained.

[0147] In the example embodiment, first, in response to a touch event of a current user acting on a preset coordinate point included in the display panel of the display terminal, a starting position node, an intermediate position node, and a termination position node of input image information are determined; second, the input image information is generated according to the starting position node, the intermediate position node, the termination position node, and a sequence of track points between the starting position node, the intermediate position node, and the termination position node. The number of preset coordinate points described herein can be set according to actual needs, and five preset coordinate points are taken as an example for explanation and description, but the present application does not make special limitations thereto.

[0148] Specifically, as shown in Figure 9 , for example, the first preset coordinate point on which the current user acts on the display interface of the display terminal is coordinate point A, that is, the middlemost coordinate point; then, taking the coordinate point A as the starting position node, the touch event input by the current user is continuously received, and then the intermediate position node and the termination position node are determined based on the touch event; the intermediate position node can include one or more, and the termination position node can be the same as or different from the starting position node, and the present example does not make special limitations thereto; meanwhile, the touch event described herein can be a touch event input by multiple fingers, a touch event input by a single finger, and a touch event input by an external device, and the present example does not make special limitations thereto.

[0149] ​Further, in the process of determining the intermediate position node and the terminal position node, the speed of the touch event moving on the display interface and the screen resolution of the display terminal can be determined; wherein the obtained input image information can refer to the following figure Figure 9

[0150] In step S220, according to the first association relationship between the preset image information and the preset application scene, the application scene to be switched corresponding to the input image information is determined.

[0151] Specifically, when the input image information is obtained, the application scene to be switched can be determined. It should be noted that the first association relationship between the preset image information and the preset application scene can be set in advance; for example, image 1 corresponds to the normal mode scene, image 2 corresponds to the office mode scene, image 3 corresponds to the child / old mode scene, image 4 corresponds to the security mode scene, and image 5 corresponds to the custom mode scene; wherein the image information can be set according to the preset coordinate point on the display interface, and the present example does not make special limitation; further, when the first association relationship between the mode scene and the image is obtained, the set first association relationship can be stored in the form of Key-Value, or in the form of list, and the present example does not make special limitation; wherein the application scene to be switched can be the office mode as shown in the following figure Figure 10

[0152] In step S230, according to the second association relationship between the preset application scene and the preset user agreement, the user agreement to be switched corresponding to the application scene to be switched is determined.

[0153] Specifically, when the application scene to be switched is obtained, the user agreement to be switched can be determined. It should be noted that the second association relationship between the preset application scene and the preset user agreement can be set in advance; for example, the normal mode scene corresponds to the user agreement 1, the office mode scene corresponds to the user agreement 2, the child / old mode scene corresponds to the user agreement 3, the security mode scene corresponds to the user agreement 4, and the custom mode scene corresponds to the user agreement 5; further, when the second association relationship between the mode scene and the user agreement is obtained, the set first association relationship can be stored in the form of Key-Value, or in the form of list, and the present example does not make special limitation.

[0154] ​​In step S240, the to-be-switched user protocol is displayed on the display interface; wherein the to-be-switched application scene includes a normal mode scene, an office mode scene, a child / old person mode scene, a safety mode scene, and a custom mode scene; and the to-be-switched user protocol includes a first user protocol corresponding to the normal mode scene, a second user protocol corresponding to the office mode scene, a third user protocol corresponding to the child / old person mode scene, a fourth user protocol corresponding to the safety mode scene, and a fifth user protocol corresponding to the custom mode scene.

[0155] Specifically, when the to-be-switched application scene is the normal mode scene and the to-be-switched user protocol is the first user protocol, the first user protocol is displayed in a first preset display mode in a first preset area of the display interface, and a first preset time required for previewing the first user protocol is displayed in a timing manner in a second preset area of the display interface; when the to-be-switched application scene is the office mode scene and the to-be-switched user protocol is the second user protocol, the second user protocol is displayed in a second preset display mode in the first preset area of the display interface, and a second preset time required for previewing the second user protocol is displayed in a timing manner in the second preset area of the display interface; when the to-be-switched application scene is the child / old person mode scene and the to-be-switched user protocol is the third user protocol, the third user protocol is displayed in a third preset manner in the first preset area of the display interface, and the third user protocol is broadcasted; when the to-be-switched application scene is the safety mode scene and the to-be-switched user protocol is the fourth user protocol, the fourth user protocol is displayed in a fourth preset display mode in the first preset area of the display interface.

[0156] In the following, the specific application scenes of various mode scenes will be explained and described.

[0157] First, the normal mode scene: the normal mode scene can be for normal users and normal entertainment needs; when the to-be-switched scene corresponding to the input image information is the normal mode scene, the normal mode scene is entered, and the first user protocol corresponding to the normal mode scene is obtained, and the whole user protocol is read within a fixed time in a countdown reading manner; specifically, reference can be made to FIG. 1; Figure 11 When the countdown is completed, click Complete to enter the main page;

[0158] Second, the office mode scenario: the office mode scenario can be for ordinary office needs; at the same time, when the input image information corresponds to the to-be-switched scene is the office mode scenario, the office mode scenario can be entered, and the second user agreement corresponding to the office mode scenario is obtained, and the countdown reading mode is adopted to read the overall user agreement in a fixed time; wherein, the second user agreement is mainly for commercial office, and some contents for preventing privacy leakage are mainly reminded, and some key introductions are made for office software in the device, and after entering, there are commonly used software in office environment, and there are one-key screen projection, conference and other functions;

[0159] Further, the children / old people mode scenario: the children / old people mode scenario can be for the old and / or children, and the interface is simple and the operation is simple. At the same time, when the input image information corresponds to the to-be-switched scene is the children / old people mode scenario, the children / old people mode scenario can be entered, and the third user agreement corresponding to the children / old people mode scenario is obtained; at the same time, since the old and / or children often do not understand the user agreement content or the font is too small, the content agreement content is enlarged, and the AI simulated human voice function is used to read the user agreement content, and after reading, the main page is entered, and the corresponding main page content is customized, which is simple and suitable for the needs of the old and children;

[0160] Further, the safety mode scenario: the safety mode scenario can be used for the needs of important occasions such as important meetings. At the same time, when the input image information corresponds to the to-be-switched scene is the safety mode scenario, the safety mode scenario can be entered, and the fourth user agreement corresponding to the safety mode scenario is obtained; at the same time, since the safety of this scene mode is important, the fourth user agreement will focus on explaining the safety, and you need to browse all the contents and slide to the last page to stay in each page content for a long enough time to ensure the reading time length before clicking to complete; and after reading the fourth user agreement, the system security will be upgraded, and only the unique signal remote controller is accepted, other signal generators are shielded, and the played content is encrypted to ensure its safety after unlocking;

[0161] Finally, the custom mode scenario: the custom mode scenario can be for special user agreements. At the same time, when the input image information corresponds to the to-be-switched scene is the custom mode scenario, the custom mode scenario can be entered, and the fifth user agreement corresponding to the custom mode scenario is obtained; at the same time, the custom mode scenario can be input by subscribing to the demand, and the corresponding font, size, regular color and other options can be set according to the demand, and the saved or fixed file format import mode is one-key brushed into the system bottom layer, and the fifth user agreement can be viewed next time after automatic startup.

[0162] Based on the above description, it can be known that the user agreement processing method disclosed in the example embodiments of the present disclosure can switch and display user agreements of multiple scenarios and multiple terminal devices, further solve the problem of different sets of user agreements in multiple devices and multiple scenarios, and greatly shorten the development time efficiency without the need to pop up different user agreements in multiple applications (which can be native applications of the display terminal itself).

[0163] It needs to be further supplemented here that in actual application, the switching of application scenarios and / or user agreements according to input image information is needed to set a password lock for the application scenarios and / or user agreements to be switched, so as to prevent misoperation or directly switch when switching is needed.

[0164] In the following, the user agreement processing method of the example embodiments of the present disclosure will be explained and described in combination with Figure 12 The user agreement processing method of the example embodiments of the present disclosure will be explained and described in combination with Figure 12 The user agreement processing method can include the following steps:

[0165] In step S1201, the gesture depth map of the gesture to be detected is obtained, and the current point cloud data possessed by the gesture to be detected is calculated according to the gesture depth map.

[0166] In step S1202, the target geometric gesture corresponding to the gesture to be detected is matched in the preset gesture search space according to the current point cloud data, and the preset coordinate point is displayed on the display interface of the display terminal when it is determined that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space.

[0167] In step S1203, the input image information is obtained in response to the touch event of the preset coordinate point on the display interface of the display terminal, and the current application scenario corresponding to the target geometric gesture is determined according to the first association relationship between the preset image information and the preset application scenario.

[0168] In step S1204, it is determined whether the current application scenario needs to be switched; if yes, go to step S1206; if no, go to step S1205.

[0169] In step S1205, the current user agreement corresponding to the current application scenario is obtained, and the current user agreement is displayed.

[0170] In step S1206, the input image information is obtained in response to the touch event of the preset coordinate point on the display interface of the display terminal, and the application scenario to be switched corresponding to the input image information is determined according to the first association relationship between the preset image information and the preset application scenario.

[0171] Step S1207: Based on the second association relationship between the preset application scenario and the preset user protocol, determine the user protocol to be switched corresponding to the application scenario to be switched, and display the user protocol to be switched on the display interface.

[0172] Therefore, it can be concluded without a doubt that the user agreement processing method described in the example embodiments of this disclosure can intelligently control and modify different user agreements through camera gesture recognition for different scenarios, so as to adapt to user agreements under multiple scenarios and multiple devices; moreover, it can also adapt to user agreements in multiple scenarios, without having to pop up special user agreements in special scenarios in the app, thus reducing the workload of developing system apps; at the same time, it can also enhance the user experience while meeting the user's need for customized user agreements.

[0173] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0174] This disclosure also provides an example embodiment of a user agreement processing apparatus. Specifically, refer to... Figure 13 As shown, the user protocol processing device may include an input image information generation module 1310, a target application scenario determination module 1320, a target user protocol determination module 1330, and a target user protocol display module 1340. Wherein:

[0175] The input image information generation module 1310 can be used to obtain input image information in response to a touch event acting on a preset coordinate point on the display interface of the display terminal;

[0176] The application scenario determination module 1320 can be used to determine the application scenario to be switched corresponding to the input image information based on the first association relationship between the preset image information and the preset application scenario.

[0177] The user protocol to be switched determination module 1330 can be used to determine the user protocol to be switched corresponding to the application scenario to be switched based on the second association relationship between the preset application scenario and the preset user protocol.

[0178] The user protocol to be switched display module 1340 can be used to display the user protocol to be switched on the display interface.

[0179] In one exemplary embodiment of this disclosure, in response to a touch event acting on a preset coordinate point on the display interface of a display terminal, input image information is obtained, including:

[0180] In response to a touch event of a current user acting on a preset coordinate point included in a display panel of the display terminal, a starting position node, an intermediate position node, and a terminal position node of input image information are determined;

[0181] According to the starting position node, the intermediate position node, the terminal position node, and a track point sequence between the starting position node, the intermediate position node, and the terminal position node, the input image information is generated.

[0182] In an exemplary embodiment of the present disclosure, the application scene to be switched includes at least one of a normal mode scene, an office mode scene, a child / old person mode scene, a safety mode scene, and a custom mode scene.

[0183] Therefore, the user protocol to be switched includes at least one of a first user protocol corresponding to the normal mode scene, a second user protocol corresponding to the office mode scene, a third user protocol corresponding to the child / old person mode scene, a fourth user protocol corresponding to the safety mode scene, and a fifth user protocol corresponding to the custom mode scene.

[0184] In an exemplary embodiment of the present disclosure, displaying the user protocol to be switched includes:

[0185] When the application scene to be switched is the normal mode scene, and the user protocol to be switched is the first user protocol, the first user protocol is displayed in a first preset display manner in a first preset area of the display interface, and a first preset time required for previewing the first user protocol is displayed in a timing manner in a second preset area of the display interface.

[0186] When the application scene to be switched is the office mode scene, and the user protocol to be switched is the second user protocol, the second user protocol is displayed in a second preset display manner in the first preset area of the display interface, and a second preset time required for previewing the second user protocol is displayed in a timing manner in the second preset area of the display interface.

[0187] When the application scene to be switched is the child / old person mode scene, and the user protocol to be switched is the third user protocol, the third user protocol is displayed in a third preset manner in the first preset area of the display interface, and the third user protocol is broadcasted.

[0188] When the application scene to be switched is the safety mode scene, and the user protocol to be switched is the fourth user protocol, the fourth user protocol is displayed in a fourth preset display manner in the first preset area of the display interface.

[0189] In an example embodiment of the present disclosure, the processing apparatus of the user protocol further comprises:

[0190] The current point cloud data calculation module can be configured to obtain a gesture depth map of the gesture to be detected, and calculate current point cloud data possessed by the gesture to be detected according to the gesture depth map.

[0191] The target geometric gesture determination module can be configured to match a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data.

[0192] The preset coordinate point display module can be configured to display the preset coordinate point on the display interface of the display terminal when it is determined that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space.

[0193] In an example embodiment of the present disclosure, the processing apparatus of the user protocol further comprises:

[0194] The current application scenario determination module can be configured to determine a current application scenario corresponding to the target geometric gesture according to a first association relationship between preset image information and a preset application scenario.

[0195] The current user protocol display module can be configured to obtain a current user protocol corresponding to the current application scenario, and display the current user protocol.

[0196] In an example embodiment of the present disclosure, matching a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data comprises:

[0197] Calculating, based on a preset image processing model, a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space;

[0198] Taking a minimum value from the distance difference value, and taking the standard geometric gesture corresponding to the minimum value as the target geometric gesture.

[0199] In an example embodiment of the present disclosure, the standard geometric gesture comprises a current frame standard geometric gesture, a previous frame standard geometric gesture corresponding to the current frame standard geometric gesture, and a next frame standard geometric gesture corresponding to the current frame standard geometric gesture.

[0200] The calculating, based on a preset image processing model, of a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space comprises:

[0201] Initialize the image processing model to be trained to obtain the to-be-optimized parameters included in the image processing model to be trained;

[0202] Input the current point cloud data of the gesture depth map and the hand pose parameters of the standard geometric gesture of the previous frame into the image processing model to be trained to obtain the predicted distance between the gesture depth map and the standard geometric gesture of the previous frame;

[0203] Construct a loss function according to the actual distance and the predicted distance between the gesture depth map and the standard geometric gesture of the previous frame, and optimize the to-be-optimized parameters according to the preset optimization algorithm and the loss function;

[0204] Update the image processing model to be trained according to the optimized parameters to obtain a trained image processing model, and calculate the distance difference between the current frame of standard geometric gesture and / or the next frame of standard geometric gesture and the gesture depth map by using the trained image processing model.

[0205] In an exemplary embodiment of the present disclosure, the preset optimization algorithm includes a particle swarm optimization algorithm and / or a nearest point optimization algorithm;

[0206] When the preset optimization algorithm is the particle swarm optimization algorithm, optimizing the to-be-optimized parameters according to the preset optimization algorithm and the loss function includes:

[0207] Generate a particle population according to the to-be-optimized parameters, and randomly set the initial position and initial speed of each particle in the particle population; wherein each to-be-optimized parameter corresponds to one particle;

[0208] Calculate the fitness of each particle according to the loss function, and compare the fitness of each particle at the current position with the fitness of each particle at the best position, if the fitness of each particle at the current position is better than the fitness of each particle at the best position, then the current position is taken as the best position of the individual, if not, then it remains unchanged;

[0209] Compare the fitness of each particle at the current position with the fitness of the best position of the population, if the fitness of each particle at the current position is better than the fitness of the best position of the population, then the current position is taken as the best global position, if not, then it remains unchanged;

[0210] Update the speed and position of the particles in the particle population according to the best position of the individual, the best global position and the initial speed, so as to optimize the to-be-optimized parameters.

[0211] The specific details of each module in the user agreement processing apparatus have been described in detail in the corresponding user agreement processing method, and therefore will not be repeated here.

[0212] It should be noted that although several modules or units of the devices for action execution are mentioned in the above detailed description, such division is not mandatory. Indeed, features and functionalities of two or more modules or units described above can be embodied in one module or unit according to embodiments of the present disclosure. Conversely, features and functionalities of one module or unit described above can be further divided into multiple modules or units.

[0213] Furthermore, although various steps of the methods in the present disclosure are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in that particular order, or that all of the steps must be performed to achieve the desired results. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, one step can be split into multiple steps, etc.

[0214] In exemplary embodiments of the present disclosure, an electronic device capable of implementing the above-described methods is also provided.

[0215] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a whole hardware embodiment, a whole software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0216] The electronic device 1400 according to this embodiment of the present disclosure will be described below with reference to Figure 14 Figure 14 The electronic device 1400 shown is merely an example and should not limit the function and use range of the embodiments of the present disclosure.

[0217] As shown in Figure 14 The components of the electronic device 1400 can include, but are not limited to, the at least one processing unit 1410 described above, the at least one storage unit 1420 described above, a bus 1430 connecting different system components (including the storage unit 1420 and the processing unit 1410), and a display unit 1440.

[0218] The storage unit stores program codes which can be executed by the processing unit 1410, so that the processing unit 1410 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section of the present specification. For example, the processing unit 1410 can perform the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section of the present specification. Figure 2 ​The step S210 shown in the middle obtains input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal; a step S220: determining a to-be-switched application scene corresponding to the input image information according to a first association relationship between preset image information and a preset application scene; a step S230: determining a to-be-switched user protocol corresponding to the to-be-switched application scene according to a second association relationship between the preset application scene and a preset user protocol; and a step S240: displaying the to-be-switched user protocol on the display interface.

[0219] The storage unit 1420 can include a readable medium in the form of volatile storage such as random access memory (RAM) 14201 and / or cache memory 14202, and also can include a non-volatile storage such as read-only memory (ROM) 14203.

[0220] The storage unit 1420 can also include a program / utility 14204 having a set (at least one) of program modules 14205 such as an operating system, one or more application programs, other program modules, and program data, each of which gives the electronic device 1400 its functionality, or some combination thereof.

[0221] The bus 1430 can represent one or more of several types of bus structures, including an storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit bus, or a local bus using any of a variety of bus architectures.

[0222] The electronic device 1400 can also communicate with one or more external devices 1500 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices that enable a user to interact with the electronic device 1400; and / or one or more devices that enable the electronic device 1400 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 1450. Still yet, the electronic device 1400 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via a network adapter 1460. As depicted, the network adapter 1460 communicates with the other components of the electronic device 1400 via the bus 1430. It should be appreciated that although the network adapter 1460 is depicted as a single component, the network adapter 1460 can include a plurality of components such as a plurality of network adapters 1460. Similarly, the input / output interface 1450 can include a plurality of input / output interfaces 1450. It should also be appreciated that the electronic device 1400 can include a plurality of buses, which can be interconnected in any fashion. It should be appreciated that these components can be implemented in hardware, software, or a combination thereof, and can be located in either the electronic device 1400 or elsewhere, such as in another computing device (e.g., a server) accessed via a network (e.g., the Internet) by the electronic device 1400.

[0223] Those skilled in the art can easily understand from the above description of the embodiments that the example embodiments described herein can be implemented by software or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or on a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0224] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, on which a program product capable of implementing the above-mentioned method of the present disclosure is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing a terminal device to perform the steps according to various example embodiments of the present disclosure described in the above-mentioned “example method” section of the present specification when the program product is run on the terminal device.

[0225] The program product for implementing the above-mentioned method according to the embodiments of the present disclosure can adopt a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited to this, and in this document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device, or apparatus.

[0226] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0227] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that can be

[0228] Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0229] Computer program code for carrying out operations for aspects of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device

[0230] In addition, the flow diagrams illustrate the processes of the method according to the exemplary embodiments of the present disclosure, but are not limited to these processes. It is readily understood that some of the processes of the flow diagrams can be performed at the same time, or in a different order than those of flow diagrams. Also, it is readily understood that some of the processes of the flow diagrams can be performed synchronously or asynchronously, for example, in multiple modules.

[0231] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features of the disclosure as set forth herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. A method of processing a user protocol, characterized by, The method comprises the steps of: in response to a touch event acting on a preset coordinate point on a display interface of a display terminal, obtaining input image information; wherein the preset coordinate point is determined by the following method: obtaining a gesture depth map of a gesture to be detected, and calculating current point cloud data possessed by the gesture to be detected according to the gesture depth map; matching a target geometric gesture for the gesture to be detected in a preset gesture search space according to the current point cloud data; when it is determined that there is a target geometric gesture corresponding to the gesture to be detected in the preset gesture search space, displaying a preset coordinate point on the display interface of the display terminal; determining a to-be-switched application scenario corresponding to the input image information according to a first association relationship between preset image information and preset application scenarios; wherein the to-be-switched application scenario comprises at least one of a normal mode scenario, an office mode scenario, a child / old person mode scenario, a safety mode scenario, and a custom mode scenario; determining a to-be-switched user protocol corresponding to the to-be-switched application scenario according to a second association relationship between the preset application scenario and a preset user protocol; displaying the to-be-switched user protocol on the display interface.

2. The method of claim 1, wherein, In response to a touch event acting on a preset coordinate point on a display interface of a display terminal, obtaining input image information, comprising: in response to a touch event acting on a preset coordinate point on a display panel of the display terminal by a current user, determining a starting position node, an intermediate position node, and a termination position node of the input image information; generating the input image information according to the starting position node, the intermediate position node, the termination position node, and a sequence of track points between the starting position node, the intermediate position node, and the termination position node.

3. The method of claim 1, wherein, The to-be-switched user protocol comprises at least one of a first user protocol corresponding to a normal mode scenario, a second user protocol corresponding to the office mode scenario, a third user protocol corresponding to the child / old person mode scenario, a fourth user protocol corresponding to the safety mode scenario, and a fifth user protocol corresponding to the custom mode scenario.

4. The method of claim 3, wherein, Displaying the to-be-switched user protocol comprises: when the to-be-switched application scenario is a normal mode scenario and the to-be-switched user protocol is a first user protocol, displaying the first user protocol in a first preset display area of the display interface in a first preset display manner, and displaying a first preset time required for previewing the first user protocol in a second preset area of the display interface; when the to-be-switched application scenario is an office mode scenario and the to-be-switched user protocol is a second user protocol, displaying the second user protocol in a first preset display area of the display interface in a second preset display manner, and displaying a second preset time required for previewing the second user protocol in a second preset area of the display interface; When the application scene to be switched is a child / old person mode scene, and the user protocol to be switched is a third user protocol, the third user protocol is displayed in a third preset manner in the first preset area of the display interface, and the third user protocol is broadcasted; When the application scene to be switched is a safety mode scene, and the user protocol to be switched is a fourth user protocol, the fourth user protocol is displayed in a fourth preset display manner in the first preset area of the display interface.

5. The method of claim 1, wherein, After the determination that the target geometric gesture corresponding to the gesture to be detected exists in the preset gesture search space, the processing method of the user protocol further includes: According to a first association relationship between preset image information and a preset application scene, a current application scene corresponding to the target geometric gesture is determined; A current user protocol corresponding to the current application scene is obtained, and the current user protocol is displayed.

6. The method of claim 1, wherein, According to the current point cloud data, the target geometric gesture is matched for the gesture to be detected in the preset gesture search space, including: Based on a preset image processing model, a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space is calculated; The minimum value is taken from the distance difference value, and the standard geometric gesture corresponding to the minimum value is taken as the target geometric gesture.

7. The method of claim 6, wherein, The standard geometric gesture includes a current frame standard geometric gesture, a previous frame standard geometric gesture corresponding to the current frame standard geometric gesture, and a next frame standard geometric gesture corresponding to the current frame standard geometric gesture; Based on a preset image processing model, a distance difference value of a three-dimensional point coordinate in the current point cloud data to a model surface of a standard geometric gesture included in the preset gesture search space is calculated, including: The image processing model to be trained is initialized to obtain a to-be-optimized parameter included in the image processing model to be trained; The current point cloud data of the gesture depth map and the hand pose parameter of the previous frame standard geometric gesture are input into the image processing model to be trained to obtain a predicted distance between the gesture depth map and the previous frame standard geometric gesture; A loss function is constructed according to an actual distance and a predicted distance between the gesture depth map and the previous frame standard geometric gesture, and the to-be-optimized parameter is optimized according to a preset optimization algorithm and the loss function; The image processing model to be trained is updated according to the optimized parameter to obtain a trained image processing model, and the distance difference value between the current frame standard geometric gesture and / or the next frame standard geometric gesture and the gesture depth map is calculated by using the trained image processing model.

8. The method of claim 7, wherein, The preset optimization algorithm includes a particle swarm optimization algorithm and / or a nearest point optimization algorithm; When the preset optimization algorithm is the particle swarm optimization algorithm, the to-be-optimized parameter is optimized according to the preset optimization algorithm and the loss function, including: Generating a particle population according to the parameters to be optimized, and randomly setting a starting position and a starting speed of each particle in the particle population; wherein each parameter to be optimized corresponds to one particle; Calculating the fitness of each particle according to the loss function, and comparing the fitness of each particle at a current position with the fitness of each particle at a best position, if the fitness of each particle at the current position is better than the fitness of each particle at the best position, then the current position is taken as the best position of the individual, if not, then it remains unchanged; Comparing the fitness of each particle at the current position with the fitness of the best position of the population, if the fitness of each particle at the current position is better than the fitness of the best position of the population, then the current position is taken as the global best position, if not, then it remains unchanged; Updating the speed and position of each particle in the particle population according to the best position of the individual, the global best position and the starting speed, so as to realize the optimization of the parameters to be optimized.

9. An apparatus for processing a user protocol, characterized by Comprise: An input image information generation module, configured to obtain input image information in response to a touch event acting on a preset coordinate point on a display interface of a display terminal; A to-be-switched application scenario determination module, configured to determine a to-be-switched application scenario corresponding to the input image information according to a first association relationship between preset image information and a preset application scenario; wherein the preset coordinate point is determined by the following manner: obtaining a gesture depth map of a to-be-detected gesture, and calculating current point cloud data possessed by the to-be-detected gesture according to the gesture depth map; matching a target geometric gesture for the to-be-detected gesture in a preset gesture search space according to the current point cloud data; and displaying the preset coordinate point on the display interface of the display terminal when it is determined that the target geometric gesture corresponding to the to-be-detected gesture exists in the preset gesture search space; A to-be-switched user protocol determination module, configured to determine a to-be-switched user protocol corresponding to the to-be-switched application scenario according to a second association relationship between the preset application scenario and a preset user protocol; wherein the to-be-switched application scenario comprises at least one of a normal mode scenario, an office mode scenario, a child / old person mode scenario, a safety mode scenario and a custom mode scenario; A to-be-switched user protocol display module, configured to display the to-be-switched user protocol on the display interface.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the user protocol processing method of any one of claims 1-8.

11. An electronic device, comprising: Comprise: A processor; And A memory for storing executable instructions of the processor; Wherein the processor is configured to execute the user protocol processing method of any one of claims 1-8 by executing the executable instructions.

Images