Video-based behavior recognition method, computing device, and user device

Abstract

The present application provides a video-based behavior recognition method, computing device, and user device. The method includes: determining multiple first video segments in a video to be recognized; obtaining a first feature sequence composed of first segment features corresponding to each of the multiple first video segments; wherein the first feature sequence is obtained by fusing the global feature sequence and the local feature sequence corresponding to the multiple first video segments; performing segment combination processing on the multiple first video segments to obtain multiple candidate video segments; identifying a target video segment that meets preset behavior recognition conditions among the multiple candidate video segments, and determining the target behavior corresponding to the target video segment. The present application improves the accuracy and efficiency of behavior recognition in videos.

Description

Technical Field

[0001] The present application relates to the technical field of electronic devices, and in particular to a video-based behavior recognition method, computing device, and user device. Background Art

[0002] With the rapid development of science and technology, the amount of video is constantly increasing. Finding meaningful videos from this massive amount of video has become a hot topic of research. Video can be a continuous image with more than 24 frames per second. The human eye cannot distinguish individual frames, resulting in a smooth and continuous image.

[0003] In the prior art, methods for analyzing video content generally involve searching for certain key frames or key video clips in the video, and using feature extraction algorithms to extract the features of the character behaviors appearing in the key frames or key video clips to obtain behavioral features, and then classifying the behavioral features to obtain target behaviors of the key frames or key video clips, and using the target behaviors of the key frames to define the behaviors in the video.

[0004] However, the above extraction method using key frames or key video segments may miss some behaviors in the video, resulting in a decrease in the extraction accuracy of video segments in which certain behaviors appear in the video. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a video-based behavior recognition method, a computing device, and a user device to solve the technical problem of low accuracy in behavior recognition in videos in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a video-based behavior recognition method, comprising:

[0007] Determining a plurality of first video segments in a video to be identified;

[0008] Obtaining a first feature sequence composed of first segment features corresponding to the plurality of first video segments respectively; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the plurality of first video segments;

[0009] Performing segment combination processing on the multiple first video segments to obtain multiple candidate video segments;

[0010] A target video segment that meets a preset behavior recognition condition is identified from the multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0011] In a second aspect, an embodiment of the present application provides a video-based behavior recognition method, comprising:

[0012] In response to a request to call a video recognition interface, determining a processing resource corresponding to the video recognition interface;

[0013] The following steps are performed using the processing resources corresponding to the video recognition interface:

[0014] Determining a plurality of first video segments in a video to be identified;

[0015] Obtaining a first feature sequence composed of first segment features corresponding to the plurality of first video segments respectively; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the plurality of first video segments;

[0016] Performing segment combination processing on the multiple first video segments to obtain multiple candidate video segments;

[0017] A target video segment that meets a preset behavior recognition condition is identified from the multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0018] In a third aspect, an embodiment of the present application provides a video-based behavior recognition method, comprising:

[0019] Detecting a video recognition request initiated by a user and a video to be recognized provided by the user;

[0020] Send the video to be recognized to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface, wherein the processing resources are used to perform the following steps:

[0021] Determine a plurality of first video segments in a video to be identified; obtain a first feature sequence composed of first segment features corresponding to each of the plurality of first video segments; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the plurality of first video segments; perform segment combination processing on the plurality of first video segments to obtain a plurality of candidate video segments; identify a target video segment that meets a preset behavior recognition condition among the plurality of candidate video segments, and determine a target behavior corresponding to the target video segment;

[0022] The target video segment and the target behavior corresponding to the target video segment provided by the computing device are obtained.

[0023] In a fourth aspect, an embodiment of the present application provides a computing device, comprising: a storage component and a processing component; wherein the storage component is used to store one or more computer instructions; the one or more computer instructions are called by the processing component for execution;

[0024] The processing component is specifically used for:

[0025] Determine multiple first video segments in a video to be identified; obtain a first feature sequence consisting of first segment features corresponding to the multiple first video segments respectively; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the multiple first video segments; perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments; identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

[0026] In a fifth aspect, an embodiment of the present application provides a user device, comprising: a storage component and a processing component; wherein the storage component is used to store one or more computer instructions; the one or more computer instructions are called by the processing component for execution;

[0027] The processing component is specifically used for:

[0028] Detecting a video recognition request initiated by a user and a video to be recognized provided by the user;

[0029] Send the video to be recognized to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface, wherein the processing resources are used to perform the following steps:

[0030] Determine a plurality of first video segments in a video to be identified; obtain a first feature sequence composed of first segment features corresponding to each of the plurality of first video segments; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the plurality of first video segments; perform segment combination processing on the plurality of first video segments to obtain a plurality of candidate video segments; identify a target video segment that meets a preset behavior recognition condition among the plurality of candidate video segments, and determine a target behavior corresponding to the target video segment;

[0031] The target video segment and the target behavior corresponding to the target video segment provided by the computing device are obtained.

[0032] In an embodiment of the present application, after determining a plurality of first video segments in a video to be identified, a first feature sequence consisting of first segment features corresponding to each of the plurality of first video segments is obtained. The first feature sequence is obtained by fusing a global feature sequence and a local feature sequence between the plurality of first video segments, so that the first feature sequence includes both global features and local features. Compared with the feature information obtained by only performing feature analysis on the video alone, the feature information obtained is more comprehensive. When performing behavioral analysis on a plurality of first video segments in a video to be identified, a plurality of video segments are combined to obtain a plurality of candidate video segments, and the video segments can be further combined and expanded. By adopting feature extraction from multiple angles and combined expansion of video segments, accurate behavioral recognition of more video segments in the video to be processed is achieved, so as to obtain a more comprehensive target video segment with higher recognition accuracy in the video to be identified, and to achieve accurate extraction of behavioral segments. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0034] Figure 1 A flowchart of an embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0035] Figure 2 A flowchart of another embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0036] Figure 3 An example diagram of a video clip provided in an embodiment of the present application;

[0037] Figure 4 A flowchart of another embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0038] Figure 5 A flowchart of another embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0039] Figure 6 A flowchart of another embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0040] Figure 7 A flowchart of another embodiment of a video-based behavior recognition method provided in an embodiment of the present application;

[0041] Figure 8 An example diagram of a video-based behavior recognition method provided in an embodiment of the present application;

[0042] Figure 9 A schematic diagram of the structure of an embodiment of a computing device provided in an embodiment of the present application;

[0043] Figure 10 A schematic diagram of the structure of an embodiment of a user equipment provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in the examples of this application and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two, but does not exclude the inclusion of at least one.

[0046] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0047] As used herein, the words “if” and “if” may be interpreted as “at the time of” or “when” or “in response to determining” or “in response to identifying,” depending on the context. Similarly, the phrases “if it is determined” or “if (stated condition or event) is identified” may be interpreted as “when it is determined” or “in response to determining” or “when identifying (stated condition or event)” or “in response to identifying (stated condition or event),” depending on the context.

[0048] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or system. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or system comprising the element.

[0049] The technical solution of the embodiment of the present application can be applied to behavior recognition in videos. By obtaining multiple video clips in the video, features are extracted using global and local features to obtain accurate features. Candidate video clips are obtained by combining multiple video clips to identify target video clips that meet the recognition conditions, thereby improving the accuracy of behavior recognition results.

[0050] In the prior art, when analyzing video content, one can extract behavioral features by searching for key frames or key video clips in the video, and use a classifier to classify the behavioral features to obtain the target behavior of the key video clip or key video frame. However, the behavior recognition method using key frames or key video clips may miss certain behaviors in the video, resulting in a decrease in the accuracy of extracting video clips containing certain predetermined behaviors in the video.

[0051] In an embodiment of the present application, after determining multiple first video segments in a video to be identified, a first feature sequence consisting of first segment features corresponding to each of the multiple first video segments is obtained. This first feature sequence is obtained by fusing the global feature sequence and the local feature sequence between the multiple first video segments, so that the first feature sequence includes both global and local features, providing more comprehensive feature information than the feature information obtained by simply performing feature analysis on each video. When performing behavioral analysis on multiple first video segments in the video to be identified, the multiple video segments are combined to obtain multiple candidate video segments, which can then be further combined and expanded. After determining the candidate segment features corresponding to the multiple candidate video segments based on the segment features corresponding to the multiple video segments in the target feature sequence, target video segments that meet preset behavioral recognition criteria can be identified from the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments. By employing feature extraction from multiple angles and combining and expanding video segments, accurate behavioral recognition can be achieved for more video segments in the video to be processed according to their corresponding features, thereby obtaining a more comprehensive and more accurately recognized target video segment in the video to be identified, and accurately extracting behavioral segments.

[0052] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0053] like Figure 1 FIG. 1 is a flowchart of an embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0054] 101: Determine multiple first video segments in a video to be identified.

[0055] The video-based behavior recognition provided in the embodiments of the present application can be applied to computing devices, which can be computers, ordinary servers, cloud servers, super personal computers, laptops, tablet computers, etc. The embodiments of the present application do not impose too many restrictions on the specific types of computing devices.

[0056] When the technical solution provided by this application is applied to electronic devices such as computers and laptops, the video to be identified can be collected by the electronic device or provided by the user. When the technical solution provided by this application is applied to a server, the video to be identified can be sent from the user terminal to the server.

[0057] The multiple first video segments are sorted according to their respective segment start times.

[0058] Optionally, a sliding window method for extracting video segments can be used to extract multiple first video segments from the video to be identified. For example, a window length of 2 seconds and a step length of 1 second are used to extract segments to obtain multiple first video segments.

[0059] The multiple first video segments each have a segment start time and a segment end time. The multiple first video segments are sorted by segment start time. Any first video segment can be any segment of the video to be identified. For example, if the video to be identified is 300 seconds long, one of the first video segments can be a segment between 20 and 50 seconds of the video to be identified.

[0060] 102: Obtain a first feature sequence consisting of first segment features corresponding to a plurality of first video segments.

[0061] The first feature sequence is obtained by fusing global feature sequences and local feature sequences corresponding to multiple first video segments. The first feature sequence includes first segment features corresponding to the multiple video segments.

[0062] Optionally, the first segment features corresponding to the plurality of first video segments are sorted according to the segment start times corresponding to the plurality of first video segments, and the sorted plurality of first segment features are combined to form a first feature sequence. That is, the plurality of first segment features in the first feature sequence are arranged according to the segment start times of the corresponding first video segments.

[0063] The segment feature of any first video segment may be a first segment feature. The first feature sequence includes first segment features corresponding to a plurality of first video segments. The feature dimensions of the plurality of first segment features are the same. For example, the first segment feature may be a feature vector with a dimension of 100. The number of sequences in the first feature sequence composed of the plurality of first segment features is equal to the number of segments of the plurality of first video segments. For example, assuming that the number of segments of the plurality of first video segments is 50, the number of sequences in the first feature sequence is 100. Assuming that the first feature vector is a feature vector with a dimension of 100, the first feature sequence may be a feature matrix of 100 (rows) * 50 (columns).

[0064] 103: Perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments.

[0065] The plurality of candidate video segments may be obtained by combining at least one sequentially adjacent video segment. Combining the plurality of video segments to obtain the plurality of candidate video segments may include combining at least one sequentially adjacent first segment among the plurality of first video segments to obtain the plurality of candidate video segments.

[0066] When performing segment combination processing on multiple first video segments, at least one adjacent first video segment can be combined to form a candidate video segment. Therefore, any candidate video segment corresponds to at least one first video segment.

[0067] 104: Identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

[0068] To accurately identify a target video segment, behavior recognition can be performed on each of the candidate video segments to obtain candidate behaviors corresponding to each of the candidate video segments. At least one of the candidate behaviors can then be compared with a pre-set behavior to determine whether the target video segment meets the behavior recognition criteria within the at least one candidate behavior. Furthermore, after identifying and obtaining the candidate behaviors corresponding to each of the candidate video segments, a score can be assigned to the at least one candidate behavior to obtain a behavior score corresponding to the at least one candidate video segment, thereby determining a target behavior score for the target video segment.

[0069] Optionally, a face recognition algorithm can be used to perform face recognition on users who perform target behaviors in target video clips to determine the identity information of users who perform target behaviors, thereby accurately obtaining the behaviors of each user to achieve further user behavior monitoring and have a more effective monitoring effect.

[0070] In an embodiment of the present application, after determining the multiple first video segments in the video to be identified, a first feature sequence consisting of the first segment features corresponding to the multiple first video segments is obtained. The first feature sequence is obtained by fusing the global feature sequence and the local feature sequence between the multiple first video segments, so that the first feature sequence includes both global features and local features. Compared with the feature information obtained by only performing feature analysis on the video alone, the feature information is more comprehensive. When performing behavioral analysis on the multiple first video segments in the video to be identified, the multiple video segments are combined to obtain multiple candidate video segments, and the video segments can be further combined and expanded. By adopting feature extraction from multiple angles and combined expansion of video segments, more candidate video segments are obtained to obtain a more comprehensive target video segment with higher recognition accuracy in the video to be identified, and accurate extraction of behavioral segments is achieved.

[0071] like Figure 2 FIG. 1 is a flowchart of an embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0072] 201: Determine a plurality of first video segments in a video to be identified.

[0073] 202: Based on a preset feature acquisition algorithm, obtain a second feature sequence consisting of second segment features corresponding to a plurality of first video segments.

[0074] Based on a video feature extraction algorithm, second segment features corresponding to each of the multiple first video segments can be obtained. The multiple second segment features are arranged according to the start time of each corresponding first video segment to form a second feature sequence. After determining the multiple first video segments, segment features of the multiple first video segments can be extracted to obtain multiple second segment features.

[0075] 203: Extracting a global feature sequence and a local feature sequence corresponding to the second feature sequence.

[0076] The second feature sequence includes a plurality of second segment features ordered according to segment start times of corresponding video segments. Global features of the second feature sequence can be extracted to obtain a global feature sequence, and local features of the second feature sequence can be extracted to obtain a local feature sequence.

[0077] 204: Fusing the global feature sequence and the local feature sequence to obtain a first feature sequence composed of first segment features corresponding to the plurality of first video segments.

[0078] The first feature sequence is obtained by fusing the global feature sequences and local feature sequences corresponding to the multiple first video segments. The number of sequences in the global feature sequence, the number of sequences in the local feature sequence, and the number of sequences in the first video segment are the same. For example, the global feature sequence can be a 70*50 feature matrix with 50 sequences, the local feature sequence can be a 30*50 feature matrix with 50 sequences, and the first feature sequence can be a 100*50 feature matrix with 50 sequences.

[0079] Optionally, when fusing the global feature sequence and the local feature sequence, the global feature sequence and the local feature sequence may be concatenated to obtain the first feature sequence. In one possible design, the global feature sequence and the local feature sequence may be weighted to obtain the first feature sequence. For example, the global feature sequence and the local feature sequence may be weighted to obtain the first feature sequence.

[0080] 205: Perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments.

[0081] 206: Identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

[0082] Some steps in the examples of this application are Figure 1 Some steps of the illustrated embodiments are the same and will not be repeated here.

[0083] In an embodiment of the present application, multiple first video segments in a video to be identified are determined, and based on a preset feature acquisition algorithm, a second feature sequence consisting of features of the second segments corresponding to each of the multiple first video segments is obtained. A feature extraction algorithm is employed to perform preliminary feature extraction on the second feature sequence consisting of features of the second segments corresponding to each of the multiple first video segments. Feature stratification is then performed using the feature sequence, and local and global feature extraction methods are employed separately to accurately extract global and local feature sequences. This contributes to the accurate extraction of features in the subsequent feature recognition process, thereby improving the accuracy of behavior recognition in the video segments.

[0084] As an embodiment, extracting the global feature sequence and the local feature sequence corresponding to the second feature sequence may include:

[0085] The second feature sequence is divided into a first sub-feature sequence and a second sub-feature sequence according to the feature dimension.

[0086] Based on the first sub-feature sequence, a global feature sequence corresponding to the second feature sequence is extracted.

[0087] Based on the second sub-feature sequence, a local feature sequence corresponding to the second feature sequence is extracted.

[0088] Since the second feature sequence includes multiple second segment features, in practical applications, the second segment features can exist in the form of feature vectors. Therefore, the second feature sequence can be divided into a first sub-feature sequence and a second sub-feature sequence according to the feature dimension. Any second segment feature in the second feature sequence can be divided into a first sub-segment feature and a second sub-segment feature. Taking the second segment feature of the 35th column as a 100-dimensional feature vector as an example, the second segment feature can be divided into a 70-dimensional first sub-segment feature and a 30-dimensional second sub-segment feature according to the feature dimension.

[0089] When the second feature sequence is divided into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension, the first sub-feature sequence may include multiple first sub-segment features, and the second sub-feature sequence may include multiple second sub-segment features. The first sub-segment features and the second sub-segment features corresponding to any sequence may be concatenated according to their respective feature dimensions to form the second segment features corresponding to that sequence.

[0090] When extracting a global feature sequence corresponding to the second feature sequence based on the first sub-feature sequence, feature calculation can be performed on the first sub-feature sequence to obtain a global feature sequence. When extracting a local feature sequence corresponding to the second feature sequence based on the second sub-feature sequence, feature calculation can be performed on the second sub-feature sequence to obtain a local feature sequence.

[0091] In order to extract accurate global and local features, a feature network can be used to extract feature fragments. Generally, the network depth of the feature network is closely related to the accuracy of feature extraction. The feature network needs to reach a certain network depth to accurately extract features and obtain more accurate features. In practical applications, the number of network layers can be set, that is, the number of feature extraction times. The more feature extraction times, the more accurate the extracted features.

[0092] In some embodiments, after dividing the second feature sequence into a first sub-feature sequence and a second sub-feature sequence according to the feature dimension, the method may further include:

[0093] Determine the number of feature extraction times N; where N is a positive integer greater than or equal to 1;

[0094] Extracting a global feature sequence corresponding to the second feature sequence based on the first sub-feature sequence includes:

[0095] Performing N global feature extractions on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence;

[0096] Extracting a local feature sequence corresponding to the second feature sequence based on the second sub-feature sequence includes:

[0097] Perform N local feature extractions on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence.

[0098] As a possible implementation, any global feature extraction step performed on the first sub-feature sequence includes:

[0099] Determine the global convolution operator;

[0100] A global convolution operator is used to perform convolution calculation with the first sub-feature sequence participating in the global convolution calculation to obtain a global feature sequence obtained by the convolution calculation; wherein, the first feature sub-sequence participating in the global convolution calculation is the global feature sequence obtained by the previous convolution calculation, and the global feature sequence obtained by the Nth convolution calculation is the last obtained global feature sequence.

[0101] Optionally, the global convolution operator can simultaneously perform convolution calculations on at least three adjacent first sub-segment features in the first feature subsequence to extract contextual features for each first sub-feature segment in the first sub-feature sequence, so as to obtain global features corresponding to the multiple first sub-segment features in the first feature subsequence. The number of convolution kernels in the global convolution operator is the same as the number of features of the at least three first sub-segment features involved in the convolution calculation. For example, when the number of convolution kernels of the global convolution operator is 3, the three first sub-segment features involved in the convolution calculation are, and the convolution formula can be:

[0102] y t =f(x t-1 , x t , x t+1 )

[0103] Among them, t is the sequence order of the first sub-feature sequence of the current convolution calculation, f is the convolution formula corresponding to the three convolution kernels, and y is the convolution calculation result.

[0104] When using the global convolution operator to perform convolution calculation with the first sub-feature sequence participating in the global convolution calculation, the global convolution operator can be used to perform convolution sliding calculation on multiple first sub-segment features of the first sub-feature sequence participating in the global convolution calculation according to a preset convolution step size, so as to obtain a global feature sequence composed of global sub-features corresponding to the multiple first sub-segment features at the end of the convolution calculation.

[0105] As another possible implementation, any local feature extraction step performed on the second feature subsequence includes:

[0106] Determine the local convolution operator;

[0107] A local convolution operator is used to perform convolution calculation with the second sub-feature sequence participating in the local convolution calculation to determine a local feature sequence obtained by the convolution calculation; wherein the second sub-feature sequence participating in the local convolution calculation is the local feature sequence obtained by the previous convolution calculation, and the local feature sequence obtained by the Nth convolution calculation is the last local feature sequence obtained.

[0108] Optionally, the local convolution operator can only perform a separate convolution calculation on each second sub-segment feature in the second sub-feature sequence to obtain a local feature sequence consisting of local sub-features corresponding to multiple second sub-segment features. The local convolution operator can be a convolution operator with a convolution kernel of 1. When using the local convolution operator to perform convolution calculation with the second sub-feature sequence participating in the local convolution calculation, it can be specifically to use a single local convolution kernel to perform convolution calculation with multiple second sub-segment features in the second sub-feature sequence respectively, to obtain a global convolution sequence consisting of local sub-features corresponding to multiple second sub-segment features. The convolution kernel of the local convolution operator is one. When using the local convolution operator to perform convolution calculation with any second sub-segment feature, the calculation formula used can be: y t =f(x t ).

[0109] In a possible design, fusing the global feature sequence and the local feature sequence to obtain a first feature sequence composed of first segment features corresponding to a plurality of first video segments may include:

[0110] The global feature sequence and the local feature sequence are fused in terms of feature dimension to obtain a first feature sequence composed of first segment features corresponding to the plurality of first video segments.

[0111] Since the first sub-feature sequence and the second sub-feature sequence are obtained by dividing the second feature sequence in the dimension, in order to obtain accurate features in the feature dimension, the first sub-feature sequence and the second sub-feature sequence can be spliced ​​in the feature dimension to obtain the first feature sequence, thereby achieving accurate acquisition of the first feature sequence.

[0112] Optionally, fusing the global feature sequence and the local feature sequence from a feature dimension perspective may specifically include: determining a first feature dimension corresponding to the first sub-feature sequence and a second feature dimension corresponding to the second sub-feature sequence; determining a third feature dimension of the global feature sequence based on the first feature dimension; and determining a fourth feature dimension of the local feature sequence based on the second feature dimension; and concatenating the global feature sequence and the local feature sequence from a feature dimension perspective in sequence order according to the third and fourth feature dimensions to obtain a feature sequence.

[0113] In some embodiments, obtaining a second feature sequence composed of second segment features corresponding to a plurality of first video segments based on a preset feature acquisition algorithm includes:

[0114] Divide the video to be identified into multiple initial video segments;

[0115] Performing segment extension processing on the multiple initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments;

[0116] Determine a plurality of first video segments obtained by sorting the plurality of initial video segments and the extended video segments corresponding to the plurality of initial video segments according to the segment start time;

[0117] A second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments is determined.

[0118] Optionally, the multiple initial video segments can be obtained by dividing the multiple videos to be identified. Specifically, the division step may include determining the segment length and dividing the video to be identified into the multiple initial video segments according to the segment length. For example, if the video to be identified is a 10-minute video segment and the preset segment length is 60 seconds, the video to be identified can be divided into 10 video segments of 60 seconds each.

[0119] The segment start time corresponding to each of the multiple initial video segments is the start time of the video segment in the video to be identified, and the segment end time is the end time of the video segment in the video to be identified. Segment extension processing is performed on each of the multiple initial video segments. For example, segment extension processing can be performed based on the segment length of the video segment or on adjacent video segments to obtain extended video segments corresponding to the video segments. Any initial video segment can correspond to one or more extended videos.

[0120] As a possible implementation, the second segment features corresponding to the plurality of first video segments are obtained by:

[0121] Extracting initial segment features corresponding to the multiple initial video segments based on a preset video feature extraction algorithm;

[0122] Determining, based on the initial segment features corresponding to the multiple initial video segments, the extended segment features of the extended video segments corresponding to the multiple initial video segments;

[0123] Second segment features corresponding to the multiple first video segments are determined based on the initial segment features corresponding to the multiple initial video segments and the extended segment features of the extended video segments corresponding to the multiple initial video segments.

[0124] A plurality of initial video segments and the extended video segments respectively corresponding to the initial video segments may be arranged according to the segment start time to form a plurality of first video segments.

[0125] When extracting features from multiple initial video segments using a video feature extraction algorithm, a common video feature extraction algorithm can be used to extract video features. Examples of such algorithms include Word2vec (Efficient Estimation of Word Representations in Vector Space) and P3D video feature extraction networks (learning spatio-temporal representation with pseudo-3D residual networks). The feature extraction method for video segments is the same as that in the prior art and will not be further described here.

[0126] The segment features of the extended video segment of any initial video segment can be determined according to the initial segment features of the initial video segment.

[0127] In a possible design, based on the initial segment features corresponding to the multiple initial video segments, determining the extended segment features of the extended video segments corresponding to the multiple initial video segments may include:

[0128] Determining, from the multiple initial video segments, associated video segments corresponding to the multiple initial video segments respectively;

[0129] For any initial video segment, the initial segment feature of the initial video segment and the initial segment feature of the associated video segment corresponding to the initial video segment are used to determine the extended segment feature of the extended video segment corresponding to the initial video segment.

[0130] The associated video segment corresponding to any initial video segment can be the initial video segment adjacent to the initial video segment. For example, the associated video segment of any initial video segment can be specified as the next initial video segment adjacent to the initial video segment. The associated video segment corresponding to the last initial video segment is empty.

[0131] Since the initial segment features of any initial video segment have been determined, the segment features of the associated video segment corresponding to any initial video segment have also been determined and can directly participate in the feature calculation of the extended video segment.

[0132] When calculating the extended segment feature of any extended video segment corresponding to an initial video segment, the feature may be calculated based on the association relationship between the initial video segment and the associated video segment corresponding to the initial video segment.

[0133] Further, optionally, for any initial video segment, using the initial segment feature of the initial video segment and the initial segment feature of the associated video segment corresponding to the initial video segment, determining the extended segment feature of the extended video segment corresponding to the initial video segment includes:

[0134] For any initial video segment, determining the proportion of the extended video segment corresponding to the initial video segment to the first segment of the initial video segment and the proportion of the extended video segment to the second segment of the associated video segment corresponding to the initial video segment;

[0135] A feature weighted calculation is performed based on the first segment ratio and the initial video features of the initial video segment, as well as the second segment ratio and the initial video features of the associated video segment corresponding to the initial video segment, to obtain the extended segment features of the extended video segment corresponding to the initial video segment.

[0136] The proportion of the first segment and the proportion of the second segment can be determined based on the number of extended video segments. The proportion of the extended video segment in the first segment of the corresponding initial video segment and the proportion of the second segment in the corresponding associated video segment can be determined in sequence based on the number of extended video segments. For example, if the number of extended video segments of an initial video segment is 2, then it can be determined that the first extended segment accounts for one-third of the initial video segment and two-thirds of the associated video segments of the initial video segment; the second extended segment accounts for two-thirds of the initial video segment and one-third of the associated video segments of the initial video segment.

[0137] As another embodiment, performing segment extension processing on the initial video segments to obtain extended video segments corresponding to the multiple initial video segments may include:

[0138] From the multiple initial video segments, associated video segments corresponding to the multiple initial video segments are determined.

[0139] Based on the preset total amount of segment analysis, the number of segment extensions corresponding to each of the multiple initial video segments is determined.

[0140] For any initial video segment, the initial video segment and the associated video segments corresponding to the initial video segment are used to obtain the extended video segments corresponding to the initial video segment according to the segment extension quantity, so as to obtain the extended video segments corresponding to the multiple initial video segments respectively.

[0141] The total number of fragments analyzed can be set to the number of pre-set fragment sequences. For example, if the total number of fragments analyzed is 50, the number of pre-set fragment sequences is 50.

[0142] When determining the number of segment extensions corresponding to multiple initial video segments based on the preset total segment analysis amount, the initial segment number of the multiple initial video segments can be determined first, and the quotient of the total segment analysis amount and the initial segment analysis amount can be calculated, and the number of segment extensions corresponding to the multiple initial video segments can be obtained after rounding.

[0143] For example, if the total number of segments analyzed is 50 and the number of segments is 15, the quotient of 50 and 15 is 3.333, which is rounded to 3. From this, the original initial video segment (1) is subtracted, resulting in a total number of segment extensions corresponding to each of the multiple initial video segments: 3-1=2. If the total number of the multiple initial video segments and the extended video segments corresponding to the multiple initial video segments does not reach the total number of segments analyzed, segment padding can be performed based on the multiple initial video segments and the extended video segments corresponding to the multiple initial video segments to ensure that the total number of segments participating in the segment analysis meets the required total number of segments, thereby improving the effectiveness and accuracy of segment extension.

[0144] In one possible design, for any initial video segment, using the initial video segment and the associated video segments corresponding to the initial video segment, according to the number of segment extensions, obtaining the extended video segments corresponding to the initial video segment, so as to obtain the extended video segments corresponding to the multiple initial video segments respectively, may include:

[0145] For any initial video segment, a linear interpolation algorithm is used to perform segment interpolation between the initial video segment and the associated video segment corresponding to the initial video segment according to the segment extension number to obtain the extended video segment corresponding to the initial video segment, so as to obtain the extended video segments corresponding to multiple initial video segments.

[0146] Optionally, for any initial video segment, the initial video segment whose segment start time is after multiple initial video segments and whose segment start time difference with the multiple initial video segments is the smallest is determined as the associated video segment of the multiple initial video segments, so as to obtain the associated video segments corresponding to the multiple initial video segments respectively.

[0147] Optionally, for any initial video segment, the segment start time of the extended video segment corresponding to the initial video segment is determined based on the segment start time of the initial video segment and the segment start time of the associated video segment corresponding to the initial video segment, and the segment end time of the extended video segment corresponding to the initial video segment is determined based on the segment end time of the initial video segment and the segment end time of the associated video segment corresponding to the initial video segment.

[0148] Furthermore, optionally, for any initial video segment, the proportion of the extended video segment corresponding to the initial video segment in the first segment of the initial video segment and the proportion of the extended video segment in the second segment of the associated video segment corresponding to the initial video segment can be determined; and a time summation calculation is performed based on the product of the segment start time of the initial video segment and the first segment proportion, and the product of the segment start time of the associated video segment and the second segment proportion, to obtain the segment start time of the extended video segment. Similarly, for any initial video segment, the proportion of the extended video segment corresponding to the initial video segment in the first segment of the initial video segment and the proportion of the extended video segment in the second segment of the associated video segment corresponding to the initial video segment can be determined; and a time summation calculation is performed based on the product of the segment end time of the initial video segment and the first segment proportion, and the product of the segment end time of the associated video segment and the second segment proportion, to obtain the segment end time of the extended video segment.

[0149] refer to Figure 3 , the time axis t of the video to be identified with a duration of 10 minutes can be divided into 5 initial video segments S1 to S5 with a duration of 2 minutes. It is assumed that the associated video segment of each initial video segment is the next initial video segment adjacent to the video segment, that is, the associated video segment of S1 is S2, the associated video segment of S2 is S3, the associated video segment of S3 is S4, and the associated video segment of S4 is S5, and S5 does not have an associated video segment. When based on any initial video segment and the extended video segment corresponding to the initial video segment, it can be determined based on the segment start time of the two video segments. Taking S1 as an example, the segment start time of S1 is 0s, and the segment end time is 120s. The segment start time of S2, the associated video segment of S2, is 121s, and the segment end time is 240s. Assuming that the number of extended video segments of S1 is 2, it can be determined that the extended video segment K1 accounts for 2 / 3 of the first segment of S1, and K1 accounts for 1 / 3 of the second segment of S2. At this time, the segment start time of K1 is 40s (0*2 / 3+121*1 / 3), and the segment end time is 160s (120*2 / 3+240*1 / 3). The extended video segment K2 accounts for 1 / 3 of the first segment of S1, and K2 accounts for 2 / 3 of the second segment of S2. At this time, the segment start time of K2 is 80s (0*1 / 3+121*2 / 3), and the segment end time is 200s (120*1 / 3+240*2 / 3). According to the above method, refer to Figure 3 , the extended video segments K3 and K4 of S2, the extended video segments K5 and K6 of S3, and the extended video segments K7 and K8 of S4 can be determined respectively, and finally a plurality of first video segments consisting of 13 video segments are obtained.

[0150] like Figure 4FIG. 1 is a flowchart of an embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0151] 401: Determine multiple first video segments in a video to be identified.

[0152] It should be noted that some steps in the examples of this application are similar to Figure 1 Some steps of the illustrated embodiments are the same and will not be repeated here.

[0153] 402: Obtain a first feature sequence consisting of first segment features corresponding to a plurality of first video segments.

[0154] The first feature sequence is obtained by fusing global feature sequences and local feature sequences corresponding to multiple first video clips.

[0155] 403: Perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments.

[0156] 404: Determine candidate segment features corresponding to multiple candidate video segments respectively according to the first segment features corresponding to the multiple first video segments in the first feature sequence.

[0157] Optionally, any candidate video segment may correspond to at least one first video segment. The candidate segment feature of any candidate video segment may be obtained by weighted calculation based on the first segment features of the at least one first video segment corresponding to the candidate video segment.

[0158] 405: Identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determine a target behavior of the target video segment.

[0159] To accurately identify a target video segment, behavior recognition can be performed using candidate segment features corresponding to each of the multiple candidate video segments to obtain candidate behaviors corresponding to each of the multiple candidate video segments. At least one candidate behavior can then be compared with a preset behavior to determine whether the target video segment satisfies the behavior recognition criteria within the at least one candidate behavior. Furthermore, after identifying and obtaining the candidate behaviors corresponding to each of the multiple candidate video segments, a score can be assigned to the at least one candidate behavior to obtain a behavior score corresponding to the at least one candidate video segment, thereby determining a target behavior score for the target video segment.

[0160] Optionally, a face recognition algorithm can be used to perform face recognition on users who perform target behaviors in target video clips to determine the identity information of users who perform target behaviors, thereby accurately obtaining the behaviors of each user to achieve further user behavior monitoring and have a more effective monitoring effect.

[0161] In an embodiment of the present application, after determining multiple first video segments in a video to be identified, a first feature sequence consisting of first segment features corresponding to each of the multiple first video segments is obtained. This first feature sequence is obtained by fusing the global feature sequence and the local feature sequence between the multiple first video segments, so that the first feature sequence includes both global and local features, providing more comprehensive feature information than the feature information obtained by simply performing feature analysis on each video. When performing behavioral analysis on multiple first video segments in the video to be identified, the multiple video segments are combined to obtain multiple candidate video segments, which can then be further combined and expanded. After determining the candidate segment features corresponding to the multiple candidate video segments based on the segment features corresponding to the multiple video segments in the target feature sequence, target video segments that meet preset behavioral recognition criteria can be identified from the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments. By employing feature extraction from multiple angles and combining and expanding video segments, accurate behavioral recognition can be achieved for more video segments in the video to be processed according to their corresponding features, thereby obtaining a more comprehensive and more accurately recognized target video segment in the video to be identified, and accurately extracting behavioral segments.

[0162] like Figure 5 FIG. 1 is a flowchart of another embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0163] 501: Determine multiple first video segments in a video to be identified.

[0164] 502: Obtain a first feature sequence consisting of first segment features corresponding to a plurality of first video segments.

[0165] The first feature sequence is obtained by fusing global feature sequences and local feature sequences corresponding to multiple first video clips.

[0166] 503: Determine the number of selected segments and the step length of the selected segments.

[0167] The number of selected segments is a positive integer from 1 to M, M is the total number of segments in the plurality of first video segments, and the segment selection step is a constant.

[0168] 504: Starting from the number of selected segments being 1, based on the segment selection step, a first video segment whose number of segments matches the number of selected segments is selected from multiple first video segments to obtain a candidate video segment, and the selection ends when the number of selected segments is M.

[0169] 505: Obtain multiple candidate video segments obtained when the selection is completed.

[0170] 506: Identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

[0171] Optionally, identifying a target video segment from a plurality of candidate video segments that meets the behavior recognition condition and determining the target behavior corresponding to the target video segment may include: determining candidate segment features corresponding to each of the plurality of candidate video segments based on first segment features corresponding to each of the plurality of first video segments in the first feature sequence. Identifying a target video segment from the plurality of candidate video segments that meets the preset behavior recognition condition based on the candidate segment features corresponding to each of the plurality of candidate video segments, and determining the target behavior corresponding to the target video segment.

[0172] In the embodiment of the present application, when performing segment expansion on multiple first video segments, the segment expansion is performed using a method of selecting a number of segments and a segment selection step size. This ensures that all first video segments can participate in the video segment combination process, ensuring that the candidate video segments can more comprehensively cover video segments of different lengths and time periods, increasing the number of candidate video segments and broadening the scope of segment analysis. Furthermore, the segment features of multiple first video segments can be reused multiple times, eliminating the need to perform feature extraction on each candidate video segment separately. This reduces the complexity of feature calculation and improves the efficiency and accuracy of segment selection.

[0173] As an embodiment, any candidate video segment may be selected and determined from multiple first video segments in the following manner:

[0174] The first video segment corresponding to any candidate video segment and matching the segment selection quantity is obtained to obtain at least one first video segment corresponding to the candidate video segment.

[0175] The minimum segment start time and the maximum segment end time are determined according to the segment start time and the segment end time respectively corresponding to at least one first video segment in the candidate video segments.

[0176] Obtain candidate video segments consisting of the minimum segment start time and the maximum segment end time in the video to be identified.

[0177] Furthermore, optionally, the candidate segment features of the candidate video segment are determined by:

[0178] A feature weighted calculation is performed on first segment features respectively corresponding to at least one first video segment in the candidate video segments to obtain candidate segment features of the candidate video segments.

[0179] As another embodiment, identifying a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on candidate segment features corresponding to the multiple candidate video segments, and determining the target behavior corresponding to the target video segment may include:

[0180] Based on the behavior recognition algorithm, identify the candidate behaviors and behavior scores corresponding to the candidate feature segments corresponding to the multiple candidate video segments;

[0181] According to the candidate behaviors and behavior scores corresponding to the multiple candidate video segments, a target video segment that meets the preset behavior recognition conditions is selected from the multiple candidate video segments, and the target behavior corresponding to the target video segment is determined.

[0182] In one possible design, multiple behavior categories can be preset, and the behavior features corresponding to the multiple behavior categories can be determined. When identifying the candidate behaviors and behavior scores corresponding to the candidate feature segments corresponding to the multiple candidate video segments based on the behavior recognition algorithm, the candidate segment features of any candidate video segment can be respectively matched with the behavior features of the multiple behavior categories to determine the behavior probability values ​​that the candidate video segment may correspond to in the multiple behavior categories, and the behavior category with the highest behavior probability value can be determined as the candidate behavior of the candidate video segment, so as to obtain the candidate behaviors of the multiple candidate video segments. The behavior score of any candidate video segment can be determined based on the highest behavior probability value of the candidate video segment. For example, the probability value can be directly converted to a percentage system to obtain the behavior score.

[0183] In some embodiments, based on the candidate behaviors and behavior scores corresponding to multiple candidate video clips, when selecting a target video clip that meets the preset behavior recognition conditions from multiple candidate video clips, it can be specifically determined whether the maximum probability value corresponding to any candidate video clip is greater than a preset probability threshold. If so, the candidate video clip is determined to be a target video clip that meets the recognition conditions; if not, the candidate video clip is determined to be a target video clip that does not meet the recognition conditions, so as to obtain all target video clips of the multiple candidate video clips.

[0184] As yet another embodiment, the method may further include:

[0185] Outputting a plurality of candidate video segments to the first user, so that the first user can confirm the plurality of candidate video segments;

[0186] Identifying a target video segment that meets a preset behavior recognition condition from multiple candidate video segments and determining a target behavior corresponding to the target video segment includes:

[0187] In response to a confirmation operation performed by the first user on the plurality of candidate video segments, a target video segment that meets a preset behavior recognition condition is identified from the plurality of candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0188] In some embodiments, after outputting multiple candidate video segments to the first user for the first user to confirm the multiple candidate video segments, the method further includes:

[0189] In response to an adjustment request triggered by a first user for a plurality of candidate video segments, obtaining adjustment information of each of the plurality of candidate video segments provided by the first user;

[0190] A plurality of adjusted candidate video segments are determined according to the adjustment information respectively corresponding to the plurality of candidate video segments.

[0191] A target video segment that meets a preset behavior recognition condition is identified from the adjusted multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0192] Optionally, determining the adjusted multiple candidate video segments based on the adjustment information corresponding to each of the multiple candidate video segments may include: performing segment adjustment on any candidate video segment based on the adjustment information of the candidate video segment to obtain the adjusted candidate video segment of the candidate video segment, and performing adjustment processing on the multiple candidate video segments respectively to obtain the adjusted multiple candidate video segments.

[0193] The adjustment information includes adjustment times corresponding to the multiple candidate video segments.

[0194] Optionally, determining the multiple adjusted candidate video segments based on the adjustment information corresponding to the multiple candidate video segments may include: adjusting the multiple candidate video segments based on their respective adjustment information to obtain the multiple adjusted candidate video segments. Specifically, this may include: performing time adjustment on any candidate video segment based on the adjustment time corresponding to the candidate video segment to obtain the adjusted candidate video segment, thereby obtaining the multiple adjusted candidate video segments.

[0195] As yet another embodiment, after identifying a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determining the target behavior corresponding to the target video segment, the method may further include:

[0196] The target video segment and the target behavior corresponding to the target video segment are output to the second user, so that the second user can confirm the target video segment.

[0197] In some embodiments, the method may further include:

[0198] Obtain the condition adjustment request initiated by the second user, adjust the behavior recognition condition, and return to the step of identifying the target video segment that meets the preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determine the target behavior corresponding to the target video segment to continue execution.

[0199] In some embodiments, the method may further include:

[0200] In response to a video recognition request initiated by a second user, a video to be recognized provided by the second user is obtained.

[0201] like Figure 6 FIG. 1 is a flowchart of another embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0202] 601: In response to a request to call a video recognition interface, determine processing resources corresponding to the video recognition interface.

[0203] Use the processing resources corresponding to the video recognition interface to perform the following steps:

[0204] 602: Determine multiple first video segments in the video to be identified.

[0205] 603: Obtain a first feature sequence consisting of first segment features corresponding to a plurality of first video segments.

[0206] The first feature sequence is obtained by fusing global feature sequences and local feature sequences corresponding to multiple first video clips.

[0207] 604: Perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments.

[0208] 605: Identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

[0209] In the embodiment of the present application, some steps in the embodiment of the present application are the same as those in the above-mentioned embodiment, and for the sake of brevity of description, they are not repeated here.

[0210] In an embodiment of the present application, video recognition is provided to users in the form of an interface, so that users can simply implement the behavior recognition function of the video to be recognized by calling the video recognition interface and providing the video to be recognized, providing users with a fast and accurate behavior recognition service and improving the service efficiency of video recognition.

[0211] like Figure 7 FIG. 1 is a flowchart of another embodiment of a video-based behavior recognition method provided by an embodiment of the present application. The method may include the following steps:

[0212] 701: Detecting a video recognition request initiated by a user and a video to be recognized provided by the user.

[0213] 702: Send the video to be recognized to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface. The processing resources are used to perform the following steps:

[0214] Determine multiple first video clips in the video to be identified; obtain a first feature sequence composed of first segment features corresponding to the multiple first video clips; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the multiple first video clips; perform segment combination processing on the multiple first video clips to obtain multiple candidate video clips; identify a target video clip that meets a preset behavior recognition condition among the multiple candidate video clips, and determine a target behavior corresponding to the target video clip.

[0215] 703: Obtain a target video segment and a target behavior corresponding to the target video segment provided by the computing device.

[0216] In the embodiments of this application, the user terminal can call a video recognition interface to provide the user with a video behavior recognition service. By sending the video recognition request initiated by the detected user and the video to be recognized to the corresponding computing device through the video recognition interface, the computing device can call upon the corresponding processing resources to execute the video-based behavior recognition method provided in the embodiments of this application, thereby realizing behavior recognition of the video to be recognized. By directly providing the user with a video-based behavior recognition method, the user's complex operations are reduced, the efficiency of the user's behavior recognition is improved, the scalability of behavior recognition is enhanced, and the complexity of behavior recognition is simplified.

[0217] In one possible design, reference Figure 8The video-based behavior recognition method provided in the embodiment of the present application can be applied to various application scenarios such as video surveillance and self-service assessment. The camera device is a micro camera M1, which is deployed in a room at location A to collect video. The behavior recognition method provided in the embodiment of the present application is set in the cloud server M2, which can be connected to a user device, which can be, for example, a mobile client M3.

[0218] Among them, the micro camera M1 in room A can continuously collect videos in the room and transmit the collected videos to the cloud server M2 801. The cloud server M2 can receive the videos to be identified collected by the micro camera M1.

[0219] Afterwards, cloud server M1 determines 802 multiple first video segments in the video to be identified and obtains 803 a first feature sequence consisting of first segment features corresponding to each of the multiple first video segments. The obtained first feature sequence can be obtained by fusing the global feature sequences and local feature sequences corresponding to the multiple first video segments, so that the first segment features of each first video segment can simultaneously include both local features and global features, resulting in more accurate features. After performing segment combination processing on the multiple first video segments, multiple candidate video segments are obtained 804. Based on the first segment features corresponding to the multiple first video segments in the first feature sequence, candidate segment features corresponding to the multiple candidate video segments are determined 805. Then, based on the candidate segment features corresponding to the multiple candidate video segments, target video segments that meet preset behavior recognition conditions can be identified 806 from the multiple candidate video segments, and the target behavior corresponding to the target video segment can be determined. By combining and expanding the video segments, accurate behavior recognition can be achieved for more video segments in the video to be processed according to their corresponding features, thereby obtaining more comprehensive and more accurately recognized target video segments in the video to be identified and accurately extracting behavior segments.

[0220] To allow the monitoring user, i.e., the user of mobile client M3, to be promptly informed of certain behaviors occurring at location A, the target video clip and the target behavior of the target video clip can be output. Cloud server M2 can transmit the obtained target video clip and the target behavior of the target video clip to mobile client M3 for viewing. In some possible designs, the output of the target video clip and the target behavior of the target video clip can also include a prompt, for example, a verbal output such as "Behavior C occurred at location A at time B," so that the monitoring user can promptly be informed of the behavior occurring at location A.

[0221] like Figure 9FIG. 1 is a schematic diagram of a structure of an embodiment of a computing device provided by an embodiment of the present application. The computing device may include: a storage component 901 and a processing component 902; wherein the storage component 901 is used to store one or more computer instructions; and the one or more computer instructions are called and executed by the processing component 902;

[0222] The processing component 902 may be specifically configured to:

[0223] Determine multiple first video clips in the video to be identified; obtain a first feature sequence composed of first segment features corresponding to the multiple first video clips; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the multiple first video clips; perform segment combination processing on the multiple first video clips to obtain multiple candidate video clips; identify a target video clip that meets a preset behavior recognition condition among the multiple candidate video clips, and determine a target behavior corresponding to the target video clip.

[0224] In an embodiment of the present application, after determining a plurality of first video segments in a video to be identified, a first feature sequence consisting of first segment features corresponding to each of the plurality of first video segments is obtained. The first feature sequence is obtained by fusing a global feature sequence and a local feature sequence between the plurality of first video segments, so that the first feature sequence includes both global features and local features. Compared with the feature information obtained by only performing feature analysis on the video alone, the feature information obtained is more comprehensive. When performing behavioral analysis on a plurality of first video segments in a video to be identified, a plurality of video segments are combined to obtain a plurality of candidate video segments, and the video segments can be further combined and expanded. By adopting feature extraction from multiple angles and combined expansion of video segments, accurate behavioral recognition of more video segments in the video to be processed is achieved, so that a more comprehensive and more accurate target video segment in the video to be identified is identified, and accurate extraction of the target behavioral segment is achieved.

[0225] As an embodiment, the processing component obtains a first feature sequence composed of first segment features corresponding to a plurality of first video segments; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the plurality of first video segments, and specifically may include:

[0226] Based on a preset feature acquisition algorithm, a second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments is acquired;

[0227] Extracting a global feature sequence and a local feature sequence corresponding to the second feature sequence;

[0228] The global feature sequence and the local feature sequence are fused to obtain a first feature sequence composed of first segment features corresponding to the plurality of first video segments.

[0229] As a possible implementation manner, the processing component extracting the global feature sequence and the local feature sequence corresponding to the second feature sequence may specifically include:

[0230] Dividing the second feature sequence into a first sub-feature sequence and a second sub-feature sequence according to the feature dimension;

[0231] Based on the first sub-feature sequence, extracting a global feature sequence corresponding to the second feature sequence;

[0232] Based on the second sub-feature sequence, a local feature sequence corresponding to the second feature sequence is extracted.

[0233] In some embodiments, the processing component may also be used to:

[0234] Determine the number of feature extraction times N; where N is a positive integer greater than or equal to 1;

[0235] Extracting a global feature sequence corresponding to the second feature sequence based on the first sub-feature sequence includes:

[0236] Performing N global feature extractions on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence;

[0237] Extracting a local feature sequence corresponding to the second feature sequence based on the second sub-feature sequence includes:

[0238] Perform N local feature extractions on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence.

[0239] As a possible implementation manner, any global feature extraction process performed by the processing component on the first sub-feature sequence may specifically include:

[0240] Determine the global convolution operator;

[0241] Perform convolution calculation on the first sub-feature sequence participating in the global convolution calculation using the global convolution operator to obtain a global feature sequence obtained by the convolution calculation; wherein the first feature sub-sequence participating in the global convolution calculation is the global feature sequence obtained by the previous convolution calculation, and the global feature sequence obtained by the Nth convolution calculation is the last global feature sequence obtained;

[0242] Any local feature extraction process performed by the processing component on the second feature subsequence may specifically include:

[0243] Determine the local convolution operator;

[0244] A local convolution operator is used to perform convolution calculation with the second sub-feature sequence participating in the local convolution calculation to determine a local feature sequence obtained by the convolution calculation; wherein the second sub-feature sequence participating in the local convolution calculation is the local feature sequence obtained by the previous convolution calculation, and the local feature sequence obtained by the Nth convolution calculation is the last local feature sequence obtained.

[0245] In some embodiments, the processing component fusing the global feature sequence and the local feature sequence to obtain a first feature sequence composed of first segment features corresponding to the plurality of first video segments may specifically include:

[0246] The global feature sequence and the local feature sequence are fused in terms of feature dimension to obtain a first feature sequence composed of first segment features corresponding to the plurality of first video segments.

[0247] In some embodiments, the processing component obtaining, based on a preset feature acquisition algorithm, a second feature sequence consisting of second segment features corresponding to a plurality of first video segments may specifically include:

[0248] Divide the video to be identified into multiple initial video segments;

[0249] Performing segment extension processing on the multiple initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments;

[0250] Determine a plurality of first video segments obtained by sorting the plurality of initial video segments and the extended video segments corresponding to the plurality of initial video segments according to the segment start time;

[0251] A second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments is determined.

[0252] As a possible implementation, the processing component may obtain the second segment features corresponding to the plurality of first video segments respectively in the following manner:

[0253] Extracting initial segment features corresponding to the multiple initial video segments based on a preset video feature extraction algorithm;

[0254] Determining, based on the initial segment features corresponding to the multiple initial video segments, the extended segment features of the extended video segments corresponding to the multiple initial video segments;

[0255] Second segment features corresponding to the multiple first video segments are determined based on the initial segment features corresponding to the multiple initial video segments and the extended segment features of the extended video segments corresponding to the multiple initial video segments.

[0256] In some embodiments, the processing component determining, based on the initial segment features corresponding to the multiple initial video segments, the extended segment features of the extended video segments corresponding to the multiple initial video segments may specifically include:

[0257] Determining, from the multiple initial video segments, associated video segments corresponding to the multiple initial video segments respectively;

[0258] For any initial video segment, the initial segment feature of the initial video segment and the initial segment feature of the associated video segment corresponding to the initial video segment are used to determine the extended segment feature of the extended video segment corresponding to the initial video segment.

[0259] As a possible implementation, the processing component may determine, for any initial video segment, the extended segment features of the extended video segment corresponding to the initial video segment using the initial segment features of the initial video segment and the initial segment features of the associated video segments corresponding to the initial video segment, and specifically may include:

[0260] For any initial video segment, determining the proportion of the extended video segment corresponding to the initial video segment to the first segment of the initial video segment and the proportion of the extended video segment to the second segment of the associated video segment corresponding to the initial video segment;

[0261] A feature weighted calculation is performed based on the first segment ratio and the initial video features of the initial video segment, as well as the second segment ratio and the initial video features of the associated video segment corresponding to the initial video segment, to obtain the extended segment features of the extended video segment corresponding to the initial video segment.

[0262] In some embodiments, the processing component performs segment extension processing on the initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments. Specifically, the processing component performs segment extension processing on the initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments may include:

[0263] Determining, from the multiple initial video segments, associated video segments corresponding to the multiple initial video segments respectively;

[0264] Determining the number of segment extensions corresponding to each of the multiple initial video segments based on a preset segment analysis total amount;

[0265] For any initial video segment, the initial video segment and the associated video segments corresponding to the initial video segment are used to obtain the extended video segments corresponding to the initial video segment according to the segment extension quantity, so as to obtain the extended video segments corresponding to the multiple initial video segments respectively.

[0266] Further, optionally, the processing component, for any initial video segment, uses the initial video segment and the associated video segments corresponding to the initial video segment to obtain, according to the number of segment extensions, the extended video segments corresponding to the initial video segment. To obtain the extended video segments corresponding to the multiple initial video segments, the following steps may be specifically performed:

[0267] For any initial video segment, a linear interpolation algorithm is used to perform segment interpolation between the initial video segment and the associated video segment corresponding to the initial video segment according to the segment extension number to obtain the extended video segment corresponding to the initial video segment, so as to obtain the extended video segments corresponding to multiple initial video segments.

[0268] As an embodiment, the processing component may also be used to:

[0269] determining candidate segment features corresponding to the plurality of candidate video segments respectively according to the first segment features respectively corresponding to the plurality of first video segments in the first feature sequence;

[0270] The processing component identifies a target video segment that meets a preset behavior recognition condition from multiple candidate video segments, and determines that the target behavior corresponding to the target video segment may specifically include:

[0271] According to the candidate segment features corresponding to the multiple candidate video segments, a target video segment that meets the preset behavior recognition condition is identified from the multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0272] In some embodiments, the processing component performs segment combination processing on the multiple first video segments to obtain the multiple candidate video segments may specifically include:

[0273] Determine the number of segments to be selected and the step length of segment selection; wherein the number of segments to be selected is a positive integer from 1 to M; M is the total number of segments of the plurality of first video segments; the step length of segment selection is a constant

[0274] Starting from the number of selected segments being 1, based on the segment selection step, selecting a first video segment whose number of segments matches the number of selected segments from the multiple first video segments to obtain a candidate video segment, until the selection ends when the number of selected segments is M;

[0275] A plurality of candidate video segments are obtained when the selection is completed.

[0276] In one possible design, the processing component selects and determines any one candidate video segment from multiple first video segments in the following manner:

[0277] Obtaining a first video segment corresponding to any candidate video segment and matching the selected number of segments, and obtaining at least one first video segment corresponding to the candidate video segment;

[0278] Determining a minimum segment start time and a maximum segment end time according to a segment start time and a segment end time respectively corresponding to at least one first video segment in the candidate video segments;

[0279] Obtain candidate video segments consisting of the minimum segment start time and the maximum segment end time in the video to be identified.

[0280] In some embodiments, the processing component may obtain the candidate segment features of the candidate video segment by:

[0281] A feature weighted calculation is performed on first segment features respectively corresponding to at least one first video segment in the candidate video segments to obtain candidate segment features of the candidate video segments.

[0282] As another embodiment, the processing component identifies a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determines that the target behavior corresponding to the target video segment may specifically include:

[0283] Based on the behavior recognition algorithm, identify the candidate behaviors and behavior scores corresponding to the candidate feature segments corresponding to the multiple candidate video segments;

[0284] According to the candidate behaviors and behavior scores corresponding to the multiple candidate video segments, a target video segment that meets the preset behavior recognition conditions is selected from the multiple candidate video segments, and the target behavior corresponding to the target video segment is determined.

[0285] As yet another embodiment, the processing component may also be configured to:

[0286] Outputting a plurality of candidate video segments to the first user, so that the first user can confirm the plurality of candidate video segments;

[0287] The processing component identifies a target video segment that meets a preset behavior recognition condition from multiple candidate video segments, and determines that the target behavior corresponding to the target video segment may specifically include:

[0288] In response to a confirmation operation performed by the first user on the plurality of candidate video segments, a target video segment that meets a preset behavior recognition condition is identified from the plurality of candidate video segments, and a target behavior corresponding to the target video segment is determined.

[0289] In some embodiments, the processing component may also be used to:

[0290] In response to an adjustment request triggered by a first user for multiple candidate video clips, adjustment information of each of the multiple candidate video clips provided by the first user is obtained; multiple adjusted candidate video clips are determined based on the adjustment information corresponding to the multiple candidate video clips; target video clips that meet preset behavior recognition conditions in the multiple adjusted candidate video clips are identified, and target behaviors corresponding to the target video clips are determined.

[0291] As yet another embodiment, the processing component may also be configured to:

[0292] The target video segment and the target behavior corresponding to the target video segment are output to the second user, so that the second user can confirm the target video segment.

[0293] In one possible design, the processing component may also be used to:

[0294] Obtain the condition adjustment request initiated by the second user, adjust the behavior recognition condition, and return to the step of identifying the target video segment that meets the preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determine the target behavior corresponding to the target video segment to continue execution.

[0295] In some embodiments, the processing component may also be used to:

[0296] In response to a video recognition request initiated by a second user, a video to be recognized provided by the second user is obtained.

[0297] The specific implementation of the steps performed by the processing component of the computing device in the embodiment of the present application has been Figure 1 The video-based behavior recognition method involved in the embodiments is described in detail in the embodiments, and its implementation principles and technical effects are not repeated here.

[0298] like Figure 10 FIG2 is a schematic diagram of a structure of an embodiment of a user device provided in an embodiment of the present application. The user device may include: a storage component 1001 and a processing component 1002; wherein the storage component 1001 is used to store one or more computer instructions; and the one or more computer instructions are called and executed by the processing component 1002;

[0299] The processing component 1002 may be specifically configured to:

[0300] Detecting user-initiated video recognition requests and user-provided videos to be recognized;

[0301] The video to be recognized is sent to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface, where the processing resources are used to perform the following steps:

[0302] Determine multiple first video segments in a video to be identified; obtain a first feature sequence composed of first segment features corresponding to each of the multiple first video segments; wherein the first feature sequence is obtained by fusing a global feature sequence and a local feature sequence corresponding to the multiple first video segments; perform segment combination processing on the multiple first video segments to obtain multiple candidate video segments; identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments, and determine a target behavior corresponding to the target video segment;

[0303] Obtain a target video segment provided by a computing device and a target behavior corresponding to the target video segment.

[0304] The specific implementation of the steps performed by the processing component of the computing device corresponding to the user device in the embodiment of the present application is described in Figure 1 The video-based behavior recognition method involved in the embodiments is described in detail in the embodiments, and its implementation principles and technical effects are not repeated here.

[0305] In addition, an embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program. When the computer program is executed, it can execute the video-based behavior recognition method in the aforementioned embodiment.

[0306] The device embodiments described above are merely illustrative, and some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art may understand and implement the present invention without inventive effort.

[0307] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product. This application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0308] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0309] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0310] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0311] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0312] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0313] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0314] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video-based behavior recognition method, characterized in that: include: Determining a plurality of first video segments in a video to be identified; Based on a preset feature acquisition algorithm, a second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments is acquired; and the second feature sequence is divided into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension. Determining a feature extraction number N, where N is a positive integer greater than or equal to 1; performing global feature extraction N times on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence; performing local feature extraction N times on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence; fusing the global feature sequence and the local feature sequence to obtain a first feature sequence composed of first segment features corresponding to each of the plurality of first video segments; performing a segment combination process on at least one first video segment that is sequentially adjacent to each other among the multiple first video segments to obtain multiple candidate video segments; determining candidate segment features corresponding to the plurality of candidate video segments respectively according to the first segment features respectively corresponding to the plurality of first video segments in the first feature sequence; According to the candidate segment features respectively corresponding to the multiple candidate video segments, a target video segment that meets the preset behavior recognition condition among the multiple candidate video segments is identified, and a target behavior corresponding to the target video segment is determined.

2. The method according to claim 1, characterized in that Any global feature extraction step performed on the first sub-feature sequence includes: Determine the global convolution operator; Performing a convolution calculation using the global convolution operator and the first sub-feature sequence participating in the global convolution calculation to obtain a global feature sequence obtained by the convolution calculation; wherein the first feature sub-sequence participating in the global convolution calculation is the global feature sequence obtained by the previous convolution calculation, and the global feature sequence obtained by the Nth convolution calculation is the last global feature sequence obtained; Any local feature extraction step performed on the second sub-feature sequence includes: Determine the local convolution operator; The local convolution operator is used to perform convolution calculation with the second sub-feature sequence participating in the local convolution calculation to determine a local feature sequence obtained by the convolution calculation; wherein the second sub-feature sequence participating in the local convolution calculation is the local feature sequence obtained by the previous convolution calculation, and the local feature sequence obtained by the Nth convolution calculation is the last local feature sequence obtained.

3. The method according to claim 1, characterized in that The fusing of the global feature sequence and the local feature sequence to obtain the first feature sequence composed of first segment features respectively corresponding to the plurality of first video segments includes: The global feature sequence and the local feature sequence are fused in feature dimension to obtain the first feature sequence composed of first segment features corresponding to the plurality of first video segments.

4. The method according to claim 1, wherein The step of obtaining a second feature sequence composed of second segment features corresponding to each of the plurality of first video segments based on a preset feature acquisition algorithm includes: Dividing the video to be identified into a plurality of initial video segments; Performing segment extension processing on the multiple initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments; Determine the multiple first video segments obtained by sorting the multiple initial video segments and the extended video segments respectively corresponding to the multiple initial video segments according to segment start time; The second feature sequences formed by the second segment features respectively corresponding to the multiple first video segments are determined respectively.

5. The method according to claim 4, characterized in that The second segment features corresponding to the plurality of first video segments are obtained in the following manner: Extracting initial segment features corresponding to the multiple initial video segments based on a preset video feature extraction algorithm; determining, based on the initial segment features respectively corresponding to the multiple initial video segments, extended segment features of the extended video segments respectively corresponding to the multiple initial video segments; The second segment features respectively corresponding to the multiple first video segments are determined according to the initial segment features respectively corresponding to the multiple initial video segments and the extended segment features respectively corresponding to the multiple initial video segments.

6. The method according to claim 5, characterized in that The determining, based on the initial segment features respectively corresponding to the multiple initial video segments, the extended segment features of the extended video segments respectively corresponding to the multiple initial video segments includes: Determining, from the multiple initial video segments, associated video segments corresponding to the multiple initial video segments respectively; For any initial video segment, the extended segment feature of the extended video segment corresponding to the initial video segment is determined by using the initial segment feature of the initial video segment and the initial segment feature of the associated video segment corresponding to the initial video segment.

7. The method according to claim 6, characterized in that The determining, for any initial video segment, of an extended segment feature of an extended video segment corresponding to the initial video segment by using the initial segment feature of the initial video segment and the initial segment feature of an associated video segment corresponding to the initial video segment comprises: For any initial video segment, determining the proportion of the extended video segment corresponding to the initial video segment to the first segment of the initial video segment and the proportion of the extended video segment to the second segment of the associated video segment corresponding to the initial video segment; A feature weighted calculation is performed based on the first segment ratio and the initial video features of the initial video segment, as well as the second segment ratio and the initial video features of the associated video segment corresponding to the initial video segment, to obtain the extended segment features of the extended video segment corresponding to the initial video segment.

8. The method according to claim 4, characterized in that The performing segment extension processing on the initial video segments respectively to obtain extended video segments corresponding to the multiple initial video segments respectively includes: Determining, from the multiple initial video segments, associated video segments corresponding to the multiple initial video segments respectively; Determining, based on a preset total amount of segment analysis, the number of segment extensions corresponding to each of the plurality of initial video segments; For any initial video segment, the initial video segment and the associated video segment corresponding to the initial video segment are used to obtain the extended video segment corresponding to the initial video segment according to the segment extension quantity, so as to obtain the extended video segments corresponding to the multiple initial video segments respectively.

9. The method according to claim 8, characterized in that The step of obtaining, for any initial video segment, the extended video segments corresponding to the initial video segment according to the segment extension quantity using the initial video segment and the associated video segments corresponding to the initial video segment, so as to obtain the extended video segments corresponding to the multiple initial video segments respectively, comprises: For any initial video segment, a linear interpolation algorithm is used to perform segment interpolation between the initial video segment and the associated video segment corresponding to the initial video segment according to the segment extension number to obtain the extended video segment corresponding to the initial video segment, thereby obtaining the extended video segments corresponding to the multiple initial video segments respectively.

10. The method according to claim 1, characterized in that Also includes: determining candidate segment features corresponding to the multiple candidate video segments respectively according to the first segment features respectively corresponding to the multiple first video segments in the first feature sequence; The step of identifying a target video segment that meets a preset behavior recognition condition among the plurality of candidate video segments and determining a target behavior corresponding to the target video segment includes: According to the candidate segment features respectively corresponding to the multiple candidate video segments, a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments is identified, and a target behavior corresponding to the target video segment is determined.

11. The method according to claim 10, characterized in that The performing segment combination processing on the multiple first video segments to obtain multiple candidate video segments includes: Determine the number of segments to be selected and the step length of segment selection; wherein the number of segments to be selected is a positive integer from 1 to M; M is the total number of segments of the plurality of first video segments; the step length of segment selection is a constant Starting from the number of selected segments being 1, based on the segment selection step, selecting a first video segment whose number of segments matches the number of selected segments from the multiple first video segments to obtain a candidate video segment, until the selection ends when the number of selected segments is M; A plurality of candidate video segments are obtained when the selection is completed.

12. The method according to claim 11, characterized in that Any candidate video segment is selected and determined from multiple first video segments in the following manner: Obtaining a first video segment corresponding to any candidate video segment and matching the selected segment quantity, and obtaining at least one first video segment corresponding to the candidate video segment; Determining a minimum segment start time and a maximum segment end time according to a segment start time and a segment end time respectively corresponding to at least one first video segment in the candidate video segments; The candidate video segment consisting of the minimum segment start time and the maximum segment end time in the video to be identified is obtained.

13. The method according to claim 12, characterized in that The candidate segment features of the candidate video segment are obtained by: A feature weighted calculation is performed on first segment features respectively corresponding to at least one first video segment in the candidate video segments to obtain candidate segment features of the candidate video segments.

14. The method according to claim 1, wherein The step of identifying a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features respectively corresponding to the multiple candidate video segments, and determining a target behavior corresponding to the target video segment includes: Based on the behavior recognition algorithm, identifying candidate behaviors and behavior scores corresponding to the candidate feature segments corresponding to the multiple candidate video segments; According to the candidate behaviors and behavior scores respectively corresponding to the multiple candidate video segments, a target video segment that meets a preset behavior recognition condition is selected from the multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

15. The method according to claim 1, wherein Also includes: outputting the plurality of candidate video segments to a first user, so that the first user can confirm the plurality of candidate video segments; The step of identifying a target video segment that meets a preset behavior recognition condition among the plurality of candidate video segments and determining a target behavior corresponding to the target video segment includes: In response to a confirmation operation performed by the first user on the multiple candidate video segments, a target video segment that meets a preset behavior recognition condition is identified among the multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

16. The method according to claim 15, characterized in that Also includes: In response to an adjustment request triggered by the first user for the multiple candidate video segments, obtaining adjustment information of each of the multiple candidate video segments provided by the first user; determining a plurality of adjusted candidate video segments according to the adjustment information respectively corresponding to the plurality of candidate video segments; A target video segment that meets a preset behavior recognition condition is identified from the adjusted multiple candidate video segments, and a target behavior corresponding to the target video segment is determined.

17. The method according to claim 1, wherein Also includes: The target video segment and the target behavior corresponding to the target video segment are output to the second user, so that the second user can confirm the target video segment.

18. The method according to claim 17, characterized in that Also includes: Obtain the condition adjustment request initiated by the second user, adjust the behavior recognition condition, and return to the step of identifying the target video segment that meets the preset behavior recognition condition according to the candidate segment features corresponding to the multiple candidate video segments, and determine the target behavior corresponding to the target video segment to continue execution.

19. The method according to claim 18, characterized in that Also includes: In response to the video recognition request initiated by the second user, the video to be recognized provided by the second user is obtained.

20. A video-based behavior recognition method, characterized in that: include: In response to a request to call a video recognition interface, determining a processing resource corresponding to the video recognition interface; The following steps are performed using the processing resources corresponding to the video recognition interface: Determining a plurality of first video segments in a video to be identified; Based on a preset feature acquisition algorithm, a second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments is acquired; and the second feature sequence is divided into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension. Determining a feature extraction number N, where N is a positive integer greater than or equal to 1; performing global feature extraction N times on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence; performing local feature extraction N times on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence; fusing the global feature sequence and the local feature sequence to obtain a first feature sequence composed of first segment features corresponding to each of the plurality of first video segments; performing a segment combination process on at least one first video segment that is sequentially adjacent to each other among the multiple first video segments to obtain multiple candidate video segments; determining candidate segment features corresponding to the plurality of candidate video segments respectively according to the first segment features respectively corresponding to the plurality of first video segments in the first feature sequence; According to the candidate segment features respectively corresponding to the multiple candidate video segments, a target video segment that meets the preset behavior recognition condition among the multiple candidate video segments is identified, and a target behavior corresponding to the target video segment is determined.

21. A video-based behavior recognition method, characterized in that: include: Detecting a video recognition request initiated by a user and a video to be recognized provided by the user; Send the video to be recognized to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface, wherein the processing resources are used to perform the following steps: Determine multiple first video segments in a video to be identified; based on a preset feature acquisition algorithm, obtain a second feature sequence consisting of second segment features corresponding to each of the multiple first video segments; divide the second feature sequence into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension; determine a feature extraction number N, where N is a positive integer greater than or equal to 1; perform N global feature extractions on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence; perform N local feature extractions on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence; fuse the global feature sequence and the local feature sequence to obtain a first feature sequence consisting of first segment features corresponding to each of the multiple first video segments; perform segment combination processing on at least one first video segment that is sequentially adjacent in the multiple first video segments to obtain multiple candidate video segments; determine candidate segment features corresponding to each of the multiple candidate video segments based on the first segment features corresponding to each of the multiple first video segments in the first feature sequence; identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to each of the multiple candidate video segments, and determine a target behavior corresponding to the target video segment; The target video segment and the target behavior corresponding to the target video segment provided by the computing device are obtained.

22. A computing device, characterized in that The invention comprises: a storage component and a processing component; wherein the storage component is used to store one or more computer instructions; the one or more computer instructions are called by the processing component for execution; The processing component is specifically used for: Determine multiple first video segments in a video to be identified; based on a preset feature acquisition algorithm, obtain a second feature sequence consisting of second segment features corresponding to each of the multiple first video segments; divide the second feature sequence into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension; determine a feature extraction number N, where N is a positive integer greater than or equal to 1; perform N times of global feature extraction on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence; perform N times of local feature extraction on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence; fuse the global feature sequence and the local feature sequence to obtain a first feature sequence consisting of first segment features corresponding to each of the multiple first video segments; perform segment combination processing on at least one first video segment adjacent to each other in the multiple first video segments to obtain multiple candidate video segments; determine candidate segment features corresponding to each of the multiple candidate video segments based on the first segment features corresponding to each of the multiple first video segments in the first feature sequence; identify a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to each of the multiple candidate video segments, and determine a target behavior corresponding to the target video segment.

23. A user equipment, characterized in that The invention comprises: a storage component and a processing component; wherein the storage component is used to store one or more computer instructions; the one or more computer instructions are called by the processing component for execution; The processing component is specifically used for: Detecting a video recognition request initiated by a user and a video to be recognized provided by the user; Send the video to be recognized to a computing device that provides a video recognition interface, so that the computing device can use processing resources corresponding to the video recognition interface, wherein the processing resources are used to perform the following steps: Determine a plurality of first video segments in a video to be identified; based on a preset feature acquisition algorithm, obtain a second feature sequence consisting of second segment features corresponding to each of the plurality of first video segments; divide the second feature sequence into a first sub-feature sequence and a second sub-feature sequence according to a feature dimension; determine a feature extraction number N, where N is a positive integer greater than or equal to 1; perform N times of global feature extraction on the first sub-feature sequence to obtain a global feature sequence corresponding to the second feature sequence; perform N times of local feature extraction on the second sub-feature sequence to obtain a local feature sequence corresponding to the second feature sequence; and fuse the global feature sequence and the local feature sequence to obtain a first feature sequence consisting of first segment features corresponding to each of the plurality of first video segments; performing segment combination processing on at least one first video segment that is sequentially adjacent among the multiple first video segments to obtain multiple candidate video segments; determining candidate segment features corresponding to the multiple candidate video segments respectively based on the first segment features corresponding to the multiple first video segments in the first feature sequence; identifying a target video segment that meets a preset behavior recognition condition among the multiple candidate video segments based on the candidate segment features corresponding to the multiple candidate video segments, and determining a target behavior corresponding to the target video segment; The target video segment and the target behavior corresponding to the target video segment provided by the computing device are obtained.

Images