Playing control method and device, electronic equipment and storage medium

Abstract

The invention provides a playing control method and device, electronic equipment and a storage medium, and the method comprises the steps: inputting collected gesture collection information into a set recognition model; wherein after the set recognition model outputs a gesture prediction result corresponding to the gesture acquisition information, gesture recognition is carried out based on the gesture acquisition information; under the condition that the set recognition model outputs a gesture prediction result, controlling a preloading player to load a monitoring video stream based on the gesture prediction result; and under the condition that the set recognition model outputs a gesture recognition result, controlling the playing of the monitoring video stream based on the gesture recognition result and the preloading player. According to the method and the device, the user can realize switching of the monitoring video stream through the gesture, a more convenient playing control mode is provided for the user, and the required video is loaded in advance through the gesture prediction, so that the time required for switching the monitoring video stream is shortened, the switching of the monitoring video stream is smoother, and the use experience is further improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of video playing, and particularly relates to a playing control method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of technology, intelligent electronic devices have been widely applied in various fields. In a smart screen with a camera, a user can view monitoring video streams in real time through a full-screen video monitoring picture. However, in the related art, the switching of monitoring video streams is generally controlled by a mouse and a keyboard or a simple remote controller, and the use of a remote controller increases the dependence on a device, and the loss of a remote controller is common in life, which brings great inconvenience to users. The playing control of the monitoring video streams is relatively inconvenient, and there is a lag when the monitoring video streams are switched. Therefore, how to more conveniently, intuitively and smoothly switch and load monitoring video streams is a problem faced by the current technology. SUMMARY

[0003] In view of this, in order to solve the technical problem of how to more conveniently, intuitively and smoothly load and switch monitoring video streams, the present disclosure provides a playing control method and device, an electronic device and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a playing control method is provided, and the playing control method comprises:

[0005] inputting collected gesture collection information into a set recognition model; wherein the set recognition model is used for gesture prediction and gesture recognition based on the received gesture collection information, and after the set recognition model outputs a gesture prediction result corresponding to the gesture collection information, gesture recognition is performed based on the gesture collection information;

[0006] in a case where the set recognition model outputs a gesture prediction result, controlling a preloading player to load monitoring video streams based on the gesture prediction result;

[0007] in a case where the set recognition model outputs a gesture recognition result, controlling the playing of monitoring video streams based on the gesture recognition result and the preloading player.

[0008] In an optional implementation, the controlling of the preloading player to load monitoring video streams based on the gesture prediction result comprises:

[0009] determining a predicted gesture with a prediction probability greater than a first probability threshold in the gesture prediction result as a target predicted gesture;

[0010] controlling the pre-loading player to load the monitoring video stream based on a position of the current monitoring video stream in the video list and the target predicted gesture.

[0011] In an optional implementation, the controlling the pre-loading player to load the monitoring video stream based on the position of the current monitoring video stream in the video list and the target predicted gesture comprises:

[0012] if the target predicted gesture is a first gesture and the current monitoring video stream is not located at a last position of the video list, controlling the pre-loading player to load a next monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the first gesture is a gesture for playing the next monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0013] if the target predicted gesture is a second gesture and the current monitoring video stream is not located at a first position of the video list, controlling the pre-loading player to load a previous monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the second gesture is a gesture for playing the previous monitoring video stream corresponding to the current monitoring video stream in the video list.

[0014] In an optional implementation, the controlling the pre-loading player to load the monitoring video stream based on the position of the current monitoring video stream in the video list and the target predicted gesture comprises:

[0015] if the gesture recognition result indicates that a valid gesture is recognized, determining a recognized gesture with a recognition probability greater than or equal to a second probability threshold in the gesture recognition result as a target recognized gesture;

[0016] controlling the pre-loading player to load the monitoring video stream based on the pre-loading player and the target recognized gesture;

[0017] or,

[0018] if the gesture recognition result indicates that a valid gesture is not recognized, pausing the pre-loading player to load the monitoring video stream.

[0019] In an optional implementation, the controlling the pre-loading player to load the monitoring video stream based on the pre-loading player and the target recognized gesture comprises:

[0020] if the target recognized gesture is consistent with the target predicted gesture, switching the pre-loading player to a current player to play the monitoring video stream loaded by the pre-loading player.

[0021] In an optional implementation, the controlling the pre-loading player to load the monitoring video stream based on the pre-loading player and the target recognized gesture comprises:

[0022] If the target recognition gesture is inconsistent with the target prediction gesture, then based on the position of the current monitoring video stream in the video list and the target recognition gesture, the playing of the monitoring video stream is controlled.

[0023] In an optional embodiment, the controlling of the playing of the monitoring video stream based on the position of the current monitoring video stream in the video list and the target recognition gesture comprises:

[0024] If the target recognition gesture is a first gesture and the current monitoring video stream is not located at the last position of the video list, then the preloading player is controlled to load the next monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the first gesture is a gesture for playing the next monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0025] If the target recognition gesture is a second gesture and the current monitoring video stream is not located at the first position of the video list, then the preloading player is controlled to load the previous monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the second gesture is a gesture for playing the previous monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0026] If the target recognition gesture is the first gesture and the current monitoring video stream is located at the last position of the video list, then the monitoring video stream loading of the preloading player is paused; or,

[0027] If the target recognition gesture is the second gesture and the current monitoring video stream is located at the first position of the video list, then the monitoring video stream loading of the preloading player is paused.

[0028] According to a second aspect of the embodiments of the present disclosure, a playing control device is provided, which comprises:

[0029] An information collection module is configured to input collected gesture collection information into a set identification model; wherein the set identification model is configured to perform gesture prediction and gesture recognition based on the received gesture collection information, and after the set identification model outputs a gesture prediction result corresponding to the gesture collection information, the set identification model performs gesture recognition based on the gesture collection information again;

[0030] The playing control module is configured to control the preloading player to load the monitoring video stream based on the gesture prediction result when the gesture prediction result is output by the setting recognition model.

[0031] The playing control module is further configured to control the monitoring video stream to be played based on the gesture recognition result and the preloading player when the gesture recognition result is output by the setting recognition model.

[0032] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, and the electronic device comprises:

[0033] a processor;

[0034] a memory for storing instructions executable by the processor;

[0035] The processor is configured to perform the playing control method according to any one of the first aspect.

[0036] According to a fourth aspect of the embodiments of the present disclosure, a storage medium is provided, and the storage medium stores one or at least one program executable by one or at least one processor to implement the playing control method according to any one of the first aspect.

[0037] The technical solutions provided by the embodiments of the present disclosure can have the following beneficial effects: In the present disclosure, a setting recognition model can be used to predict and recognize gestures based on gesture collection information collected by an electronic device such as a smart screen. The setting recognition model can first predict gestures based on the earlier part of the gesture collection information, and then recognize gestures based on more information than the above-mentioned part of information, so as to obtain corresponding gesture prediction result and gesture recognition result. Then, the preloading player is controlled to load the corresponding monitoring video stream based on the gesture prediction result, and after obtaining the gesture recognition result, the monitoring video stream is controlled to be played based on the gesture recognition result and the preloading player. Based on the scheme of the present disclosure, the user can switch the monitoring video stream through gestures, avoiding the complex control mode that the user must operate on the screen or must rely on the mouse, keyboard or remote controller for playing control, providing a more convenient playing control mode for the user. Moreover, the video needed is loaded in advance through gesture prediction, the time required for switching the monitoring video stream is shortened, the switching of the monitoring video stream is smoother, and the use experience is further improved.

[0038] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0040] Figure 1 is a flowchart of a play control method according to an exemplary embodiment.

[0041] Figure 2 is a block diagram of a play control apparatus according to an exemplary embodiment.

[0042] Figure 3 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0043] Other advantages and novel features of the present application will become apparent from the following detailed description of the application when considered in conjunction with the drawings. The application can be put into practice / implemented or applied in other different embodiments, and the details in the specification can be modified or changed based on different views and applications, without departing from the spirit of the application. It should be understood that the preferred embodiments are only for illustrating the present application, and are not intended to limit the scope of protection of the present application.

[0044] It should be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application, and only show the components related to the present application in the drawings, rather than presenting the number, shape and size of the components in actual implementation. The actual implementation of each component can be randomly changed in shape, number and proportion, and the layout pattern of the components can be more complex.

[0045] Other advantages and novel features of the present application will become apparent from the following detailed description of the application when considered in conjunction with the drawings. The application can be put into practice / implemented or applied in other different embodiments, and the details in the specification can be modified or changed based on different views and applications, without departing from the spirit of the application. It should be understood that the preferred embodiments are only for illustrating the present application, and are not intended to limit the scope of protection of the present application.

[0046] Embodiments of the present disclosure provide a playback control method and device, an electronic device, and a storage medium. In the present disclosure, a gesture prediction and recognition of gesture collection information collected by an electronic device such as a smart screen can be performed using a set recognition model. The set recognition model can first perform gesture prediction based on the earlier part of the gesture collection information, and then perform gesture recognition based on more information than the above-mentioned part of information, so as to obtain a gesture prediction result and a gesture recognition result corresponding to each other. Then, based on the gesture prediction result, a preloading player is controlled to load a corresponding monitoring video stream, and after obtaining the gesture recognition result, the monitoring video stream is controlled to be played based on the gesture recognition result and the preloading player. Based on the scheme of the present disclosure, a user can switch the monitoring video stream through gestures, avoiding the complex control mode that the user must operate on the screen or must rely on a mouse, a keyboard, or a remote controller for playback control, providing a more convenient playback control mode for the user. Moreover, the video required is loaded in advance through gesture prediction, the time length required for switching the monitoring video stream is shortened, the switching of the monitoring video stream is smoother, and the use experience is further improved.

[0047] In one example embodiment, a playback control method is provided, which can be applied to an electronic device. Referring to Figure 1 The method can include:

[0048] S110, input the collected gesture collection information into a set recognition model;

[0049] S120, in the case where the set recognition model outputs a gesture prediction result, control a preloading player to load a monitoring video stream based on the gesture prediction result;

[0050] S130, in the case where the set recognition model outputs a gesture recognition result, control the monitoring video stream to be played based on the gesture recognition result and the preloading player.

[0051] In step S110, the set recognition model can be used to process the received gesture collection information to output a gesture prediction result and a gesture recognition result corresponding to each other. The set recognition model can be trained by selecting a model architecture suitable for processing time series data, such as a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a convolutional neural network (CNN), etc.

[0052] The electronic device can be configured with a collection device for collecting gesture collection information. The collection device can be, for example, a sensor or a camera, etc. When the gesture collection information is a video, a camera can be used for collection. When the gesture collection information is sensing information, a sensor can be used for collection.

[0053] In some embodiments,

[0054] The gesture collection information can be a video. In this embodiment, the camera can be used to collect the video in real time, and send the collected video to the setting recognition model. After the setting recognition model receives the above-mentioned video, it can detect the gesture of the video frame, and when five fingers in the video frame are detected, the setting identification can be set for the above-mentioned video frame, and the angle of the gesture in the video frame at this time can be marked. The setting identification can be set according to the actual needs, and the specific content and form thereof can not be limited. Then the gesture prediction process is started, and after the setting recognition model obtains the gesture prediction result based on the received video, the gesture recognition is performed to determine the gesture recognition result corresponding to the gesture prediction result. If the gesture recognition result is not determined within a certain time length (for example, 1.5 seconds), it is considered that no effective gesture is recognized.

[0055] As can be known from the above introduction, the setting recognition model generally determines and outputs the gesture prediction result based on the received gesture collection information, and then determines and outputs the corresponding gesture recognition result based on the gesture collection information, so as to ensure that the continuously output gesture prediction result and gesture recognition result correspond to each other. If the setting recognition model outputs a gesture prediction result, and does not output a gesture recognition result, but outputs another gesture prediction result, it indicates that the setting recognition model does not obtain the gesture recognition result corresponding to the above-mentioned previous gesture prediction result.

[0056] The gesture prediction result and the gesture recognition result can be marked by different result identifiers, so as to be distinguished. The specific content of the result identifier can not be limited.

[0057] In step S120, after the setting recognition model outputs the gesture prediction result, the gesture prediction result is analyzed, and the predicted gesture with a prediction probability greater than the first probability threshold in the gesture prediction result is determined as the target predicted gesture. Then, based on the target predicted gesture and the position of the current monitoring video stream in the video list, the preloading player is controlled to load the corresponding monitoring video stream.

[0058] The gesture prediction result can include a predicted gesture, which can be recorded as an original predicted gesture. It should be noted that the gesture in the present scheme can include a first gesture (for example, a gesture of waving left), a second gesture (for example, a gesture of waving right), or other gestures, such as a gesture of waving up or a gesture of waving down, etc., which are not limited.

[0059] In some embodiments, the gesture prediction result comprises a raw predicted gesture, and the raw predicted gesture can be classified as a left-swing gesture and a right-swing gesture. In this embodiment, after the recognition model outputs the raw predicted gesture, the output raw predicted gesture can be directly determined as the target predicted gesture, so as to facilitate subsequent control of the preloading player to load the corresponding monitoring video stream.

[0060] In some embodiments, the gesture prediction result comprises a raw predicted gesture, and the raw predicted gesture can be classified as a left-swing gesture and a right-swing gesture. In this embodiment, after the recognition model outputs the raw predicted gesture, the output raw predicted gesture can be directly determined as the target predicted gesture, so as to facilitate subsequent control of the preloading player to load the corresponding monitoring video stream.

[0061] In some embodiments, the gesture prediction result comprises a raw predicted gesture, and the raw predicted gesture can be classified as a left-swing gesture and a right-swing gesture. In this embodiment, after the recognition model outputs the raw predicted gesture, the output raw predicted gesture can be directly determined as the target predicted gesture, so as to facilitate subsequent control of the preloading player to load the corresponding monitoring video stream.

[0062] In some embodiments, the gesture prediction result comprises a raw predicted gesture, and the raw predicted gesture can be classified as a left-swing gesture and a right-swing gesture. In this embodiment, after the recognition model outputs the raw predicted gesture, the output raw predicted gesture can be directly determined as the target predicted gesture, so as to facilitate subsequent control of the preloading player to load the corresponding monitoring video stream.

[0063] It should be noted that the first probability threshold can be set according to actual needs, and the specific value thereof can not be limited. For a playback control scene with high accuracy requirements, the first probability threshold can be set to be larger; for a playback control scene with low accuracy requirements, the first probability threshold can be set to be smaller. Moreover, the first probability threshold is greater than 0 and less than 1.

[0064] In this embodiment, if the prediction probability is greater than or equal to the first probability threshold, the corresponding raw predicted gesture can be determined as the target predicted gesture, so as to facilitate subsequent control of the preloading player to load the corresponding monitoring video stream. If the prediction probability is less than the first probability threshold, the gesture prediction result can be ignored, that is, it is considered that no gesture is predicted, and the preloading player does not need to load the monitoring video stream.

[0065] In addition, in this step, the first gesture can be a gesture for playing a next monitoring video stream corresponding to the current monitoring video stream in the video list. That is, when the preloading player is controlled to load the corresponding monitoring video stream based on the target predicted gesture and the position of the current monitoring video stream in the video list, if the target predicted gesture is the first gesture, it indicates that the next monitoring video stream corresponding to the current monitoring video stream in the video list can be played.

[0066] In this case, it can be determined whether the current monitoring video stream (i.e., the currently played monitoring video stream) is located at the last position of the video list. If the current monitoring video stream is located at the last position of the video list, it indicates that there is no next monitoring video stream corresponding to the current monitoring video stream, and thus the preloading player does not need to be controlled to load the monitoring video stream. If the current monitoring video stream is not located at the last position of the video list, it indicates that there is a next monitoring video stream corresponding to the current monitoring video stream in the video list, and the user is likely to want to view the next monitoring video stream in the future. Therefore, the preloading player can be controlled to load the next monitoring video stream in advance, so as to facilitate subsequent fast playing of the next monitoring video stream.

[0067] The second gesture can be a gesture for playing a previous monitoring video stream corresponding to the current monitoring video stream in the video list. That is, when the preloading player is controlled to load the corresponding monitoring video stream based on the target predicted gesture and the position of the current monitoring video stream in the video list, if the target predicted gesture is the second gesture, it indicates that the user is likely to want to play the previous monitoring video stream corresponding to the current monitoring video stream in the video list in the future.

[0068] In this case, it can be determined whether the current monitoring video stream (i.e., the currently played monitoring video stream) is located at the first position of the video list. If the current monitoring video stream is located at the first position of the video list, it indicates that there is no previous monitoring video stream corresponding to the current monitoring video stream, and thus the preloading player does not need to be controlled to load the monitoring video stream. If the current monitoring video stream is not located at the first position of the video list, it indicates that there is a previous monitoring video stream corresponding to the current monitoring video stream in the video list, and the user is likely to want to view the previous monitoring video stream in the future. Therefore, the preloading player can be controlled to load the previous monitoring video stream in advance, so as to facilitate subsequent fast playing of the previous monitoring video stream.

[0069] It should be noted that, in addition to the above gesture, other gestures can also be used to control the preloading player to load the corresponding monitoring video stream, and the present application is not limited in this regard.

[0070] In step S130, after the gesture recognition model outputs the gesture recognition result, if the gesture recognition result indicates that a valid gesture is recognized, the gesture recognition result can be analyzed, and the recognized gesture with a recognition probability greater than or equal to the second probability threshold in the gesture recognition result is determined as the target recognized gesture. Then, based on the target recognized gesture and the preloading player, the playing of the monitoring video stream is controlled.

[0071] The gesture recognition result can include the recognized gesture, can include the recognition probability corresponding to the recognized gesture, or can include other information, which is not limited. It should be noted that the recognition probability can be set to a probability of 0 to 1 according to actual needs, for example, 0.1, 0.3, or 0.9, and the like. The greater the recognition probability, the higher the reliability of the original recognized gesture.

[0072] In some embodiments,

[0073] The gesture recognition result includes the original recognized gesture, which can be divided into a left swinging gesture and a right swinging gesture. In this embodiment, after the identification model outputs the original recognized gesture, the output original recognized gesture can be directly determined as the target recognized gesture, so as to facilitate subsequent monitoring and playing of the video stream based on the preloaded player control.

[0074] In some embodiments,

[0075] The gesture recognition result can include the original recognized gesture and the recognition probability corresponding to the original recognized gesture. The original recognized gesture can be divided into a left swinging gesture and a right swinging gesture. In this embodiment, after the identification model outputs the original recognized gesture and the corresponding recognition probability, it can be determined whether the recognition probability is greater than or equal to a second probability threshold.

[0076] It should be noted that the second probability threshold can be set according to actual needs, and the specific value thereof can not be limited. For a playback control scene with high accuracy requirements, the second probability threshold can be set to be larger; for a playback control scene with low accuracy requirements, the second probability threshold can be set to be smaller. Moreover, the second probability threshold is greater than 0 and less than 1. The second probability threshold can be the same as or different from the first probability threshold in other embodiments, which is not limited.

[0077] In this embodiment, if the recognition probability is greater than or equal to the second probability threshold, the corresponding original recognized gesture can be determined as the target recognized gesture, so as to facilitate subsequent monitoring and playing of the video stream based on the preloaded player control. If the recognition probability is less than the second probability threshold, it is considered that the gesture recognition result represents that no effective gesture is recognized, and the gesture recognition result can be ignored, and the preloaded player is paused to load the monitoring video stream, so as to save resources.

[0078] In addition, in this step, when the target recognized gesture and the corresponding target predicted gesture are consistent, the playing of the monitoring video stream based on the target recognized gesture and the preloaded player control can be performed.

[0079] If the target recognition gesture is consistent with the target prediction gesture, it indicates that the target recognition gesture is used to play the monitoring video stream loaded in the preloading player, and the preloading player can be switched to the current player to play the monitoring video stream loaded in the preloading player.

[0080] In some embodiments,

[0081] In the gesture prediction stage, the target prediction gesture can be the first gesture. If the current monitoring video stream is not located at the last position of the video list, the preloading player is controlled to load the next monitoring video stream of the current monitoring video stream. In the gesture recognition stage, the target recognition gesture is also the first gesture, the current player is paused first, then the current player and the preloading player are interchanged, the playing interface is switched to the new current player after the interchanging, and then the current player is controlled to play the monitoring video stream, at this time, the monitoring video stream played by the current player is the next monitoring video stream, that is, the monitoring video stream that the user wants to play.

[0082] If the target recognition gesture is not consistent with the target prediction gesture, it indicates that the target recognition gesture is used to play the monitoring video stream, which is not the same as the monitoring video stream loaded in the preloading player, and the playing of the monitoring video stream can be controlled based on the position of the current monitoring video stream in the video list and the target recognition gesture.

[0083] If the target prediction gesture is the second gesture and the target recognition gesture is the first gesture, and the current monitoring video stream is not located at the last position of the video list, the preloading player is controlled to load the next monitoring video stream of the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded in the preloading player.

[0084] If the target prediction gesture is the second gesture and the target recognition gesture is the first gesture, and the current monitoring video stream is located at the last position of the video list, the loading of the monitoring video stream of the preloading player is paused. It should be noted that when the current monitoring video stream is located at the last position of the video list, after the target prediction gesture is determined to be the second gesture, the preloading player is controlled to load the previous monitoring video stream of the current monitoring video stream in the video list. After the target recognition gesture is determined to be the first gesture, since the target recognition gesture is not consistent with the target prediction gesture, the loading of the monitoring video stream of the preloading player in progress can be paused.

[0085] If the target prediction gesture is the first gesture and the target recognition gesture is the second gesture, and the current monitoring video stream is not located at the first position of the video list, the preloading player is controlled to load the previous monitoring video stream of the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded in the preloading player.

[0086] If the target predicted gesture is the first gesture, the target recognized gesture is the first gesture, and the current monitoring video stream is located at the first position of the video list, the loading of the monitoring video stream of the preloading player is paused. It should be noted that when the current monitoring video stream is located at the first position of the video list, after it is determined that the target predicted gesture is the first gesture, the preloading player can be controlled to load the current monitoring video stream at the next monitoring video stream corresponding to the video list. After it is determined that the target recognized gesture is the second gesture, since the target recognized gesture is inconsistent with the target predicted gesture, the loading of the monitoring video stream being performed by the preloading player can be paused.

[0087] In addition, in this step, if the gesture recognition result indicates that no valid gesture is recognized, it indicates that although the set recognition model predicts a clear gesture based on the collected video, no clear gesture is recognized, which indicates that the user can not want to perform the play control on the monitoring video stream, and thus the loading of the monitoring video stream of the preloading player can be paused.

[0088] In this method, the gesture prediction and recognition of the gesture collection information collected by the smart screen and other electronic devices can be performed by using the set recognition model. The set recognition model can first perform gesture prediction based on the earlier part of the gesture collection information, and then perform gesture recognition based on more information than the above-mentioned part of information, so as to obtain the gesture prediction result and the gesture recognition result corresponding to each other. Then, based on the gesture prediction result, the preloading player is controlled to load the corresponding monitoring video stream, and after the gesture recognition result is obtained, based on the gesture recognition result and the above-mentioned preloading player, the play of the monitoring video stream is controlled. Based on the scheme of the present disclosure, the user can switch the monitoring video stream through the gesture, which avoids the complex control mode that the user must operate on the screen or must rely on the mouse, keyboard or remote controller for the play control, and provides a more convenient play control mode for the user. Moreover, the video needed is loaded in advance through the gesture prediction, which shortens the time length required for the switching of the monitoring video stream, makes the switching of the monitoring video stream more smooth, and further improves the use experience.

[0089] In one example embodiment, a play control method is provided, which can be applied to an electronic device. In this method, during model training, data collection can be performed first. Among them, the gesture information samples containing the gestures (such as the first gesture and the second gesture) that need to be predicted and recognized can be collected first, which can be realized through sensors (such as accelerometers, gyroscopes) or cameras (such as RGB cameras, depth cameras) according to the type of gesture information samples. For example, when the gesture information sample is a video, it can be collected through a camera.

[0090] After the collection of gesture information samples is completed, a corresponding gesture label (e.g., a first gesture and a second gesture) can be configured for each gesture information. Meanwhile, some gesture information samples that do not include the above-mentioned gestures that need to be predicted and recognized can also be collected to train the model to recognize and ignore these gestures.

[0091] After the collection of gesture information samples is completed, the collected data can be cleaned to remove noise and irrelevant information. If visual data (i.e., video) is used, image preprocessing such as frame difference, background removal, key point extraction, etc. can be performed. After the cleaning of the data is completed, the data can be standardized or normalized.

[0092] When training the recognition model, the model can be trained to set a setting identifier in the input sample. The first part of information (e.g., 10 frames of images) after the setting identifier is used for gesture prediction, and the second part of information (e.g., 60 frames of images) after the setting identifier is used for gesture recognition.

[0093] After the gesture information samples are processed as described above, the gesture information samples can be divided into a training set, a validation set, and a test set.

[0094] When training the model, an original model suitable for processing time series data can be selected, such as a recurrent neural network (RNN), a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a convolutional neural network (CNN), etc. In addition, when the valid gesture categories only include a first gesture and a second gesture, a binary classification model can be designed for training.

[0095] The model can include an input layer, a hidden layer, and an output layer. It should be noted that the input layer can be designed according to the data type of the input sample. For sensor data, it can be a one-dimensional time series; for visual data, it can be a two-dimensional image or a three-dimensional video frame sequence.

[0096] Multiple hidden layers can be designed in the model, such as RNN, LSTM, GRU, or CNN layers. For the output layer, a neuron and a sigmoid activation function can be used to output a probability between 0 and 1, representing the probability that the gesture is a first gesture or a second gesture.

[0097] During the training process, a binary cross-entropy loss function can be used to calculate the error between the model output and the true label. An optimizer such as Adam or SGD can be used to adjust the weights of the model. In addition, the validation set can be used to monitor the performance of the model and prevent overfitting.

[0098] The performance of the model can be evaluated using the test set, and the attention indicators can include accuracy, precision, recall, and F1 score, etc. If the above indicators do not meet the requirements, the model can be adjusted to optimize the above indicators to obtain the required setting recognition model.

[0099] After the setting recognition model is trained, the setting recognition model can be deployed in practical applications, such as intelligent devices, robots, or other electronic devices, etc., and it is ensured that the setting recognition model can receive gesture collection information in real time and make predictions and recognitions based on the gesture collection information to output corresponding gesture prediction results and gesture recognition results.

[0100] It should be noted that the gesture prediction result of the setting recognition model can be divided into a prediction result corresponding to the first gesture and a prediction result corresponding to the second gesture. In addition to being divided into a recognition result corresponding to the first gesture and a recognition result corresponding to the second gesture, the gesture recognition result of the setting recognition model can also include a recognition result representing that no valid gesture is recognized.

[0101] In some embodiments,

[0102] The input sample can be a video, and relevant personnel can label the video under various scenarios based on experience. Among them, based on the video frame in which a 5-finger gesture is recognized, it is labeled as gesture prediction, but if the gesture in the video does not change, it can be continuously labeled as gesture prediction, and the gesture angle at this time is marked as @. Then detect the motion trajectory of the gesture in the labeled video for the next video frame, and when the motion trajectory of the gesture in the video changes by 10 radians from the previously marked gesture angle @, label the gesture (such as a left swinging gesture or a right swinging gesture) in the gesture prediction result; when the motion trajectory of the gesture in the video changes by 60 radians from the gesture angle @, label the gesture (such as a left swinging gesture or a right swinging gesture) in the gesture recognition result. In addition, when the model is trained, if there is no large gesture recognition result within a certain time period (for example, 1.5 seconds) after the gesture prediction is triggered successfully, the gesture recognition is interrupted and ended, and a gesture recognition result representing that no valid gesture is recognized is returned.

[0103] The method can train the required setting recognition model and can well guarantee the reliability of the setting recognition model to improve the reliability of the entire playback control method.

[0104] In one example embodiment, a playback control method is provided, which can be applied to an electronic device. In the method, the setting recognition model can make gesture predictions and recognitions based on the received video, and the gestures it can predict and recognize are divided into left swinging gestures and right swinging gestures.

[0105] The setting recognition model analyzes the input video to obtain a prediction or recognition result, and sends a corresponding event notification. For example, if a target prediction gesture is determined, a gesture prediction swing event notification corresponding to the target prediction gesture is sent; if a target recognition gesture is determined, a gesture swing event notification corresponding to the target recognition gesture is sent.

[0106] It should be noted that the application scenarios of the method are not limited to the video switching scenarios to be mentioned below. As long as the gesture swing direction recognition system is used to trigger events and perform corresponding business processing, it can be applied.

[0107] In this embodiment, the video list of the monitoring video can be displayed in full screen horizontally, and the user can switch to the next or previous monitoring video stream through left or right gestures. When the currently played monitoring video (i.e., the current monitoring video) is at the first position of the video list, the operation of switching to the previous one cannot be performed. When the currently played monitoring video (i.e., the current monitoring video) is at the last position of the video list, the operation of switching to the next one or the left swipe operation cannot be performed.

[0108] The playing of the monitoring video stream involves loading and rendering of the video stream. In code implementation, two players can be constructed and used cyclically. One is named the current player, and the other is named the preloading player. When the monitoring video stream is switched, the two players can be exchanged in identity to achieve the switching.

[0109] Through the training of the model, the setting recognition model can have gesture prediction swing direction detection (i.e., gesture prediction). If the current monitoring video is at the first position of the video list and the target prediction gesture obtained by prediction is a right swing gesture, no action is performed, i.e., the preloading player does not load the monitoring video stream. If the current monitoring video is at the last position of the video list and the target prediction gesture obtained by prediction is a left swing gesture, no action is performed, i.e., the preloading player does not load the monitoring video stream.

[0110] In other cases, if the target prediction gesture is a left swing gesture, the preloading player pulls (i.e., loads) the next monitoring video stream in advance. If the target prediction gesture is a right swing gesture, the preloading player pulls the previous monitoring video stream in advance. However, no specific playing and rendering action is performed.

[0111] After the target recognition gesture is determined, if the target recognition gesture is consistent with the target predicted gesture, the screen switching is executed, that is, the preloaded player is switched to the current player and playback rendering is performed; if the target recognition gesture is inconsistent with the target predicted gesture, the preloaded player is controlled to re-pull the corresponding monitoring video stream based on the target recognition gesture, and then the preloaded player is switched to the current player and playback rendering is performed; when the gesture recognition result indicates that no valid gesture is recognized, it means that the predicted gesture did not actually occur, and the preloaded player pauses pulling the monitoring video stream.

[0112] This method predicts the user's waving direction and pre-loads the surveillance video stream, improving the smoothness of video switching. Furthermore, by identifying the waving direction, it avoids ineffective surveillance video stream loading and conserves system resources. In other words, based on this method, users can switch surveillance video streams with gestures, avoiding the complex control methods that require users to operate on the screen or rely on a mouse, keyboard, or remote control for playback, providing users with more convenient playback control. Furthermore, by pre-loading the required video through gesture prediction, the time required to switch surveillance video streams is shortened, making the switching of surveillance video streams smoother and further improving the user experience.

[0113] In an exemplary embodiment, a playback control device is provided, which can be applied to an electronic device. The device can be used to implement the above-mentioned playback control method. For example, referring to Figure 2 As shown, the device may include an information collection module 10 and a playback control module 20.

[0114] An information collection module 10 is configured to input the collected video into a recognition model; wherein the recognition model is configured to perform gesture prediction and gesture recognition based on the received gesture collection information, and after the recognition model outputs a gesture prediction result corresponding to the gesture collection information, gesture recognition is performed based on the gesture collection information;

[0115] The playback control module 20 is used to control the preloaded player to load the monitoring video stream based on the gesture prediction result when the recognition model is set to output the gesture prediction result;

[0116] The playback control module 20 is further configured to control the playback of the monitoring video stream based on the gesture recognition result and the preloaded player when the recognition model is set to output the gesture recognition result.

[0117] In an exemplary embodiment, a playback control device is provided, which can be applied to an electronic device. Figure 2 As shown, in the device, the playback control module 20 can be used to:

[0118] The predicted gesture with a prediction probability greater than the first probability threshold in the gesture prediction result is determined as a target predicted gesture.

[0119] Based on the position of the current monitoring video stream in the video list and the target predicted gesture, the preloading player is controlled to load the monitoring video stream.

[0120] In one example embodiment, a play control apparatus is provided, which can be applied to an electronic device. Referring to Figure 2 The play control module 20 can be configured to:

[0121] If the target predicted gesture is the first gesture and the current monitoring video stream is not located at the last position of the video list, the preloading player is controlled to load the next monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the first gesture is a gesture for playing the next monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0122] If the target predicted gesture is the second gesture and the current monitoring video stream is not located at the first position of the video list, the preloading player is controlled to load the previous monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the second gesture is a gesture for playing the previous monitoring video stream corresponding to the current monitoring video stream in the video list.

[0123] In one example embodiment, a play control apparatus is provided, which can be applied to an electronic device. Referring to Figure 2 The play control module 20 can be configured to:

[0124] If the gesture recognition result indicates that a valid gesture is recognized, the recognized gesture with a recognition probability greater than or equal to a second probability threshold in the gesture recognition result is determined as a target recognized gesture.

[0125] Based on the preloading player and the target recognized gesture, the play of the monitoring video stream is controlled.

[0126] Alternatively,

[0127] If the gesture recognition result indicates that no valid gesture is recognized, the loading of the monitoring video stream by the preloading player is paused.

[0128] In one example embodiment, a play control apparatus is provided, which can be applied to an electronic device. Referring to Figure 2 The play control module 20 can be configured to:

[0129] If the target recognized gesture is consistent with the target predicted gesture, the preloading player is switched to a current player to play the monitoring video stream loaded by the preloading player.

[0130] In an example embodiment, a play control apparatus is provided, applicable to an electronic device. Referring to Figure 2 In the apparatus, the play control module 20 is configured to:

[0131] If the target identification gesture is inconsistent with the target prediction gesture, the play of the monitoring video stream is controlled based on the position of the monitoring video stream in the video list and the target identification gesture.

[0132] In an example embodiment, a play control apparatus is provided, applicable to an electronic device. Referring to Figure 2 In the apparatus, the play control module 20 is configured to:

[0133] If the target identification gesture is the first gesture and the current monitoring video stream is not located at the last position of the video list, the preloading player is controlled to load the next monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the first gesture is a gesture for playing the next monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0134] If the target identification gesture is the second gesture and the current monitoring video stream is not located at the first position of the video list, the preloading player is controlled to load the previous monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the second gesture is a gesture for playing the previous monitoring video stream corresponding to the current monitoring video stream in the video list; or,

[0135] If the target identification gesture is the first gesture and the current monitoring video stream is located at the last position of the video list, the monitoring video stream loading of the preloading player is paused; or,

[0136] If the target identification gesture is the second gesture and the current monitoring video stream is located at the first position of the video list, the monitoring video stream loading of the preloading player is paused.

[0137] In an example embodiment, an electronic device is provided. The electronic device can be a smart device (such as a smart screen) or a robot provided with a camera, or other devices provided with a camera, without limitation.

[0138] Referring to Figure 3As shown, the electronic device 100 can include at least one processor 101, a memory 102, at least one network interface 104, and other user interfaces 103. The various components of the electronic device 100 are coupled together by a bus system 105, which can include a data bus, a power bus, a control bus, and a state signal bus. For the sake of clarity, the various buses are illustrated in the drawing as the bus system 105.

[0139] The user interface 103 can include a display, a keyboard, or a pointing device (e.g., a mouse, a trackball, a touchpad, or a touchscreen).

[0140] It is to be understood that the memory 102 in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 102 described herein is intended to include, among other things, these and any other suitable types of memory.

[0141] In some embodiments, the memory 102 stores elements, executable units or data structures, or a subset thereof, or an extended set thereof, such as an operating system 1021 and application programs 1022.

[0142] The operating system 1021 contains various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application program 1022 contains various application programs, such as a media player (Media Player), a browser (Browser), and the like, for implementing various application services. The program implementing the method of the embodiments of the present application can be included in the application program 1022.

[0143] In the embodiments of the present application, the processor 101 is configured to execute the method provided by each method embodiment by invoking the program or instruction stored in the memory 102, specifically, the program or instruction stored in the application program 1022.

[0144] The method disclosed in the embodiments of the present application can be applied to the processor 101 or implemented by the processor 101. The processor 101 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 101. The processor 101 can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware code processing processor to execute, or a combination of hardware and software units in the code processing processor to execute. The software unit can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory 102, and the processor 101 reads the information in the memory 102 and combines the hardware to complete the above method.

[0145] It is to be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

[0146] For a software implementation, the techniques described herein can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes can be stored in memory units and executed by processors. The memory unit can be implemented within the processor or external to the processor.

[0147] Embodiments of the present application provide a storage medium (computer-readable storage medium). The storage medium stores one or at least one programs. The storage medium can include a volatile memory, such as a random access memory, and / or can include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive, or a solid-state drive. The storage medium can also include a combination of the above-mentioned memories.

[0148] The one or at least one programs stored in the storage medium can be executed by one or at least one processors. When the storage medium is applied to an electronic device, the above-mentioned method performed by the electronic device can be implemented. The processor is configured to execute a control program of the electronic device stored in the memory, so as to implement the above-mentioned method performed by the electronic device.

[0149] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0150] It should be noted that the terms "one embodiment," "an embodiment," "certain embodiments," certain examples," "some embodiments," and the like, typically refer to one of only a number of particular embodiments or implementations, which implement examples of the present application. This numbering reference is merely to facilitate discussion, and does not imply a particular order of execution or sequence of steps, unless explicitly stated in the claim. Furthermore, the use of these terms in the description is not intended to exclude the technical possibility of several alternative embodiments being configured according to the claims. Moreover, the terms "comprise", "comprising", "include", "including", and the like, specify the presence of stated features, integers, steps, or components, but do not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.

[0151] It should be noted that, in the present document, the terms "first", "second", and the like, merely denote different categories, and do not require or imply any actual relationship or order between such categories. Also, the terms "comprise", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without further constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0152] The above embodiments are only preferred examples of the present application, and the protection scope of the present application is not limited thereto. Any equivalent replacement or transformation of the present application made by those skilled in the art based on the present application is within the protection scope of the present application.

Claims

1. A play control method characterized by comprising: The playing control method comprises: inputting the collected gesture collection information into a set recognition model; wherein the set recognition model is used for gesture prediction and gesture recognition based on the received gesture collection information, and after the set recognition model outputs a gesture prediction result corresponding to the gesture collection information, gesture recognition is performed based on the gesture collection information; in the case that the set recognition model outputs a gesture prediction result, the preloading player is controlled to load the monitoring video stream based on the gesture prediction result; in the case that the set recognition model outputs a gesture recognition result, the playing of the monitoring video stream is controlled based on the gesture recognition result and the preloading player.

2. The play control method according to claim 1, characterized by, The control of the preloading player to load the monitoring video stream based on the gesture prediction result comprises: a predicted gesture with a prediction probability greater than a first probability threshold in the gesture prediction result is determined as a target predicted gesture; the preloading player is controlled to load the monitoring video stream based on the position of the current monitoring video stream in the video list and the target predicted gesture.

3. The play control method according to claim 2, characterized by, The control of the preloading player to load the monitoring video stream based on the position of the current monitoring video stream in the video list and the target predicted gesture comprises: if the target predicted gesture is a first gesture and the current monitoring video stream is not located at the last position of the video list, the preloading player is controlled to load the next monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the first gesture is a gesture used to play the next monitoring video stream corresponding to the current monitoring video stream in the video list; or if the target predicted gesture is a second gesture and the current monitoring video stream is not located at the first position of the video list, the preloading player is controlled to load the previous monitoring video stream corresponding to the current monitoring video stream in the video list; wherein the second gesture is a gesture used to play the previous monitoring video stream corresponding to the current monitoring video stream in the video list.

4. The playback control method according to any one of claims 1 to 3, characterized by, The control of the playing of the monitoring video stream based on the gesture recognition result and the preloading player comprises: if the gesture recognition result represents that a valid gesture is recognized, an identified gesture with an identification probability greater than or equal to a second probability threshold in the gesture recognition result is determined as a target identified gesture; the playing of the monitoring video stream is controlled based on the preloading player and the target identified gesture; or if the gesture recognition result represents that a valid gesture is not recognized, the loading of the monitoring video stream by the preloading player is paused.

5. The play control method according to claim 4, characterized by, The control of the playing of the monitoring video stream based on the preloading player and the target identified gesture comprises: if the target identified gesture is consistent with the target predicted gesture, the preloading player is switched to a current player to play the monitoring video stream loaded by the preloading player.

6. The play control method according to claim 4, characterized by, The control of the playing of the monitoring video stream based on the preloading player and the target identified gesture comprises: If the target recognition gesture is inconsistent with the target prediction gesture, then based on the position of the current monitoring video stream in the video list and the target recognition gesture, the playing of the monitoring video stream is controlled.

7. The play control method according to claim 6, wherein The playing of the monitoring video stream based on the position of the current monitoring video stream in the video list and the target recognition gesture comprises: If the target recognition gesture is a first gesture and the current monitoring video stream is not located at the last position of the video list, then the preloading player is controlled to load the next monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the first gesture is a gesture for playing the next monitoring video stream corresponding to the current monitoring video stream in the video list; or, If the target recognition gesture is a second gesture and the current monitoring video stream is not located at the first position of the video list, then the preloading player is controlled to load the previous monitoring video stream corresponding to the current monitoring video stream in the video list, and the preloading player is switched to the current player to play the monitoring video stream loaded by the preloading player; wherein the second gesture is a gesture for playing the previous monitoring video stream corresponding to the current monitoring video stream in the video list; or, If the target recognition gesture is the first gesture and the current monitoring video stream is located at the last position of the video list, then the monitoring video stream loading of the preloading player is paused; or, If the target recognition gesture is the second gesture and the current monitoring video stream is located at the first position of the video list, then the monitoring video stream loading of the preloading player is paused.

8. A play control apparatus characterized by comprising: The playing control device comprises: an information collection module, configured to input collected gesture collection information into a set identification model; wherein the set identification model is configured to perform gesture prediction and gesture recognition based on the received gesture collection information, and after the set identification model outputs a gesture prediction result corresponding to the gesture collection information, the set identification model performs gesture recognition based on the gesture collection information; a playing control module, configured to, in the case that the set identification model outputs a gesture prediction result, control a preloading player to load a monitoring video stream based on the gesture prediction result; the playing control module is further configured to, in the case that the set identification model outputs a gesture recognition result, control the playing of the monitoring video stream based on the gesture recognition result and the preloading player.

9. An electronic device, comprising: The electronic device comprises: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to perform the playing control method according to any one of claims 1-7.

10. A storage medium, characterized by The storage medium stores one or at least one program, and the one or at least one program is executable by one or at least one processor to implement the playing control method according to any one of claims 1-7.

Images