Panoramic activity identification method based on adaptive bidirectional propagation prototype learning

Abstract

The invention discloses a panoramic activity recognition method based on adaptive two-way propagation prototype learning, and the method achieves the combined recognition of multi-granularity activities in a panoramic scene through an adaptive focusing detector and a two-way propagation prototype learning device. In a self-adaptive focusing detector, individuals are effectively detected through a panoramic adaptive focuser in a coarse and fine combination mode, and changes of individual sizes and shielding in a crowded panoramic scene are dealt with. In a two-way propagation prototype learning device, closed-loop interaction and information consistency among different granularities are promoted through the two-way propagation prototype learning device, and information loss caused by inaccurate positioning is reduced. The method comprises a detection branch and an identification branch, the detection branch outputs an original detection frame and inputs the original detection frame into a panoramic adaptive focuser to obtain a final size adaptive detection result, and the identification branch encodes a video and obtains different levels of feature representations for identification through a bidirectional propagation prototype device. And the overall framework is subjected to joint optimization through detection loss and multi-granularity identification loss.

Description

Technical Field

[0001] The invention relates to a panoramic activity recognition method based on adaptive two-way propagation prototype learning, and belongs to the technical field of computer vision. Background Art

[0002] Human activity recognition has attracted great interest and has been widely used in different fields, such as video surveillance and sports analysis. In the past decade, researchers have mainly focused on behavior recognition at a single granularity level, such as individual activities, interactions between people, and group activities. The former two usually focus on videos containing only one or a few people, while the latter focuses on recognizing the overall activities performed by multiple people. However, some real-world scenarios often involve not only an unpredictable number of individuals, but also groups of people who are connected to each other through some form of interaction, such as participating in a common activity, which forms an additional concept of group-level activities. For example, in a crowded panorama scene with individuals, it is crucial to jointly understand both individual-level activities and group-level activities.

[0003] Panoramic Activity Recognition (PAR) aims to jointly identify multi-granular behaviors, including individual activities, group activities, and global activities, in crowded panoramic scenes. Different from ordinary video scenes in human activity recognition tasks, panoramic scenes have the characteristics of occluded people of different sizes and multi-granular activities that interact with each other. Therefore, the key challenges of the panoramic activity recognition task mainly lie in two aspects: 1) how to accurately detect people of different sizes in crowded scenes; 2) how to capture the interactions between multi-granular activities to better recognize them.

[0004] Traditional methods only focus on recognizing multi-granular activities with prior information on individual locations. The usual solution is to extract individual features based on bounding boxes and then learn multi-granular features by modeling the interactions between multiple granularities. However, these aforementioned methods that rely on manually annotated bounding boxes are not only laborious but also inefficient for practical deployment. Therefore, recent methods attempt to use general detectors for individual detection before performing inference for multi-granular activity recognition. Nevertheless, general detectors designed for general scenes are difficult to adapt to multi-person panoramic scenes involving multi-person activities of varying sizes and spatial occlusions. In addition, multi-granular activities in panoramic scenes interact with each other, so the information loss caused by inaccurate individual detection may interfere with the performance of multi-granular activity recognition. Summary of the invention

[0005] Purpose of the invention: In view of the problems and shortcomings in the prior art, the present invention provides a panoramic activity recognition method based on adaptive two-way propagation prototype learning. Through the proposed adaptive focus two-way propagation prototype learning framework AdaFPP, by learning adaptive focus detectors and multi-granularity prototypes in an end-to-end manner as pre-tasks, individual activities, group activities and global activities in panoramic activity scenes are jointly recognized, solving the problems of inaccuracy of traditional methods in detecting people with size changes and spatial occlusions, and information loss when capturing interactions between multi-granularity activities.

[0006] Technical solution: A panoramic activity recognition method based on adaptive two-way propagation prototype learning. The method realizes the joint recognition of multi-granularity activities in panoramic scenes through an adaptive focus detector and a two-way propagation prototype learner. The adaptive focus detector and the two-way propagation prototype learner constitute an adaptive focus two-way propagation prototype learning framework AdaFPP. The implementation of the method includes the following steps:

[0007] Step 1: Input the video into the detection branch, the detection network outputs the original detection box, which is then input into the panoramic adaptive focuser (PAF) to obtain the final size-adaptive detection.

[0008] Step 2: Input the video into the recognition branch, encode it using the pre-trained encoder to obtain the feature map, and use the adaptive detection obtained in step 1 to obtain the features of each individual.

[0009] Step 3: According to the ViT method, flatten the individual features obtained in step 2 into in is the channel, P is the patch width of the individual feature, is the number of patches, and all patches are projected into D dimensions by linear projection to obtain the patch embedding f patch ∈R N×d , R represents the real number field, HW represents the size of the image, P 2 represents the size of the patch, and d represents the dimension of the vector.

[0010] Step 4: Input the patch embedding obtained in step 3 into the Bidirectional Propagation Prototype (BPP) to obtain the final feature representations at different levels for recognition.

[0011] Step 5: The entire model is trained jointly by the detection loss and the multi-granularity recognition loss.

[0012] The step 1 comprises the following specific steps:

[0013] For the input panoramic video v p ∈R C×T×H×W, where C is the number of channels, T is the total number of frames, H and W are the frame resolutions, and the final size adaptive detection is obtained through the following specific steps:

[0014] Step 201: Adaptively adjust the size of the original detection bounding box through the adaptive object adjustment (AOR) strategy; use the expansion ratio β to extract the original detection box set B ori The center of each bounding box in β is expanded by the width and height of the original detection box set to roughly surround its true boundary. β is expressed as follows:

[0015]

[0016] where w h represents the width of the individual detection frame, and β1 and β2 both represent the expansion ratio. The expanded detection frame set is obtained, denoted as B ext .

[0017] Step 202: Select dense sub-regions through the dense region merging (DRM) strategy; First, the expanded detection frame set B obtained in step 201 ext Select the smallest box a as the starting point for generation, and let b represent the box set B. ext And a box containing box a, we can get the minimum merged box c that contains the union a∪b, if the box area of ​​a∩b is not zero, that is Update a with c and get from B ext Remove b from the equation. This process is repeated until In this case, a is collected as a subregion into subregion set B. sub Repeat the above process until B ext becomes an empty set, and then obtains a collected dense sub-region set B sub .

[0018] Step 203: Crop and input the selected dense sub-region into the detection network to obtain fine-grained detection, and integrate the original detection and fine-grained detection through the detection fusion strategy to obtain the final size-adaptive detection; based on the sub-region B obtained in step 202 sub , crop the corresponding sub-regions from the original frame and input them into the detection network to obtain fine-grained detection B fin Then, the final size-adaptive detection B is calculated by non-maximum suppression (NMS). ada , as shown below:

[0019] B ada =NMS(B ori +B fin )

[0020] Where NMS(·) represents the operation of NMS, B adais the adaptive detection box, B ori is the original detection box, B fin It is a fine-grained detection box.

[0021] The unified bidirectional encoding block included in the bidirectional propagation prototype learner executes as follows:

[0022] Step 301: Given a feature sequence As the input of a unified bidirectional encoding block, the index l∈(patch,ind,group} represents different granularities, namely patch level, individual level and group level, d represents the dimension of the vector, M is the number of tokens, represents the i-th feature, all unified bidirectional encoding blocks UBE aim to learn multi-granularity prototypes to encode multi-granular features, where J is the number of tokens, Representing the j-th multi-granularity prototype, the UBE block consists of two parts: UME (bottom-up encoding) and CME (top-down encoding).

[0023] Step 302: Learnable Embeddings Added to sequence Earlier, we also added a learnable position embedding matrix P∈R (M+1)×d To get the input token z l , based on z l The output is obtained by using Multi-Head Self-Attention (MSA)

[0024] where z l , It is expressed as follows:

[0025]

[0026] Where MLP(·) represents a multi-layer perceptron consisting of two linear projections.

[0027] Step 303: In addition, the feature sequence The learnable prototype in step 301 Together they are fed into the UME module to group visual semantics across different granularities. Learnable Prototypes and token The similarity matrix A between can be defined as:

[0028]

[0029] where γ j are independent and identically distributed random samples drawn from a Gumbel(0,1) distribution, and w kare the linear projection weights of the prototype and visual tokens respectively. Then, we transform the feature sequence x with different weights into l Aggregate to Update each prototype Then all prototypes are averaged As shown below:

[0030]

[0031] where w o and w v is the weight used to project and merge features. The conventional grid structure does not constrain this UME block, and the information can be reorganized into arbitrary image fragments. For bottom-up encoding, the features of each level are obtained as:

[0032]

[0033] in Is from The CLS token.

[0034] Step 304: Inverse Cross-Granularity Motion Embedding (CME) is introduced, which aims to utilize higher-level information to guide the learning of low-level features. Specifically, it uses higher-level output tokens and low-level visual tokens x l As input, to achieve complementary information interaction between different granularities, the final feature o is obtained through multi-head cross attention. l , as shown below:

[0035]

[0036] The two-way propagation process of UBE is as follows:

[0037] Step 401: Given a patch embedding f from a video encoder patch ∈R N×d ,Using UBE blocks, a bidirectional hierarchical network is constructed to model activities at different granularities, which includes two processes: forward and backward.

[0038] Step 402: Forward propagation process: individual → group → global, embed the patch into f in UBE patch ∈R N×d Input into its UME module to obtain the individual features f aggregated across granularities in turn i ′ nd ∈R Q×d 、Group characteristics g ′ ro ∈R L×d and global features Right now:

[0039] f i ′ nd =UME p2i (f patch )

[0040] f g ′ ro =UME i2g (f i ′ nd )

[0041] f glo =UME g2g (f g ′ ro )

[0042] The subscripts of UME represent input and output of different granularities.

[0043] Step 303: Backward propagation process: group ← global. The global feature f obtained at the global level glo Together with the group-level visual tokens, they are input into CME to obtain the final group representation f gro ∈R L×d , expressed as:

[0044] f gro =CME g2g (f glo )

[0045] CME g2g Aims to exploit global information to guide group-level representation learning.

[0046] Step 304: Backward propagation process: individual ← group; similarly, the feature representation f obtained at the group level gro Together with the individual-level visual tokens, they are input into CME to obtain the final individual representation f ind ∈R Q×d , as shown below:

[0047] f ind =CME g2i (CME g2g (f glo ))=CME g2i (f gro )

[0048] CME g2i Aims to utilize group information to guide individual-level representation learning.

[0049] The joint optimization process is as follows:

[0050] The overall training loss of AdaFPP is combined with the conventional detection loss L det and multi-granularity recognition loss L rec , the detection loss is defined as:

[0051]

[0052] in represents the Intersection over Union (IoU) loss of the bounding box regression loss, represents the cross entropy loss for binary classification, represents the cross entropy loss for classification. reg is a constant scalar that balances the contribution of the loss term, and its default value is 5.

[0053] The recognition loss is defined as:

[0054]

[0055] in and They represent the binary cross entropy loss functions for individual, group, global activity recognition, and group detection respectively. The overall training loss formula of AdaFPP is as follows:

[0056]

[0057] Among them, λ is the weight coefficient.

[0058] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the panoramic activity recognition method based on adaptive bidirectional propagation prototype learning as described above are implemented.

[0059] A computer-readable storage medium stores a computer program for executing the above-mentioned panoramic activity recognition method based on adaptive bidirectional propagation prototype learning.

[0060] Beneficial effects: The proposed method for panoramic activity recognition based on adaptive bidirectional propagation prototype learning has achieved state-of-the-art results on the challenging panoramic activity recognition benchmark JRDB-PAR dataset. This method effectively detects individuals in a coarse and fine manner in an adaptive focus detector to cope with changes in individual size and occlusion in crowded panoramic scenes. In the bidirectional propagation prototype learner, the bidirectional propagation prototype promotes closed-loop interaction and information consistency between different granularities, reduces information loss caused by inaccurate positioning, and ultimately achieves state-of-the-art performance in recognizing panoramic activities. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0063] In order to solve the problem that traditional methods are inaccurate in detecting people with size changes and spatial occlusions, and the problem of information loss when capturing interactions between multi-granularity activities, the present invention forms an adaptive focusing two-way propagation prototype learning framework AdaFPP through an adaptive focusing detector and a two-way propagation prototype learner, and learns the adaptive focusing detector and multi-granularity prototypes in an end-to-end manner as a prerequisite to jointly identify personal activities, group activities and global activities in panoramic activity scenes. The panoramic adaptive focuser is used to effectively detect individuals in a coarse and fine manner to cope with changes in individual size and occlusion in crowded panoramic scenes. The two-way propagation prototype is used to promote closed-loop interaction and information consistency between different granularities, thereby reducing information loss caused by inaccurate positioning.

[0064] A panoramic activity recognition method based on adaptive two-way propagation prototype learning. This method realizes the joint recognition of multi-granular activities in panoramic scenes through an adaptive focus detector and a two-way propagation prototype learner, alleviating the problems of inaccuracy in detecting people with size changes and spatial occlusions, and information loss when capturing interactions between multi-granular activities. The adaptive focus detector and the two-way propagation prototype learner constitute the adaptive focus two-way propagation prototype learning framework AdaFPP. The method includes a detection branch and a recognition branch. The detection branch outputs the original detection frame and inputs the panoramic adaptive focuser to obtain the final size adaptive detection result. The recognition branch encodes the video and obtains different levels of feature representation for recognition through the two-way propagation prototype. The overall framework is jointly optimized by the detection loss and the multi-granularity recognition loss. Specifically, it includes the following steps:

[0065] Step 1: For the input panoramic video v p ∈R C×T×H×W , where C is the number of channels, T is the total number of frames, H and W are the resolutions of the frames, which are input to the detection network to output the original detection box.

[0066] Step 2: Adaptively adjust the size of the original detection bounding box through the adaptive object adjustment (AOR) strategy; use the expansion ratio β to extract the original detection box set B oriThe center of each bounding box in β is expanded by its width and height to roughly surround its true boundary. β is expressed as follows:

[0067]

[0068] where w h Represents the width of the individual detection frame. The expanded detection frame set is obtained, denoted as B ext .

[0069] Step 3: Select dense sub-regions through the dense region merging (DRM) strategy; First, the expanded detection box set B obtained in step 2 ext Select the smallest box a as the starting point for generation, and let b represent the box set B. ext And a box containing box a, we can get the minimum merged box c that contains the union a∪b, if the box area of ​​a∩b is not zero, that is Update a with c and get from B ext Remove b from the equation. This process is repeated until In this case, a is collected as a subregion into subregion set B. sub Repeat the above process until B ext becomes an empty set, and then obtains a collected dense sub-region set B sub .

[0070] Step 4: Crop and input the selected dense sub-region into the detection network to obtain fine-grained detection, and integrate the original detection and fine-grained detection through the detection fusion strategy to obtain the final size-adaptive detection; based on the sub-region B obtained in step 3 sub , crop the corresponding sub-regions from the original frame and input them into the detection network to obtain fine-grained detection B fin Then, the final size-adaptive detection B is calculated by non-maximum suppression (NMS). ada , as shown below:

[0071] B ada =NMS(B ori +B fin )

[0072] Where NMS(·) represents the operation of NMS.

[0073] Step 5: Given a feature sequence As the input of a unified bidirectional encoding block, where the index l∈(patch,ind,group} represents different granularities and M is the number of tokens, all UBE blocks aim to learn multi-granularity prototypes to encode multi-granular features, where J is the number of tokens. The UBE block consists of two parts: UME (bottom-up encoding) and CME (top-down encoding).

[0074] Step 6: Learned Embeddings Added to sequence Earlier, we also added a learnable position embedding P∈R (M+1)×d To get the input token z l , based on z l The output is obtained by using Multi-Head Self-Attention (MSA)

[0075] where z l , It is expressed as follows:

[0076]

[0077] Where MLP(·) represents a multi-layer perceptron consisting of two linear projections.

[0078] Step 7: In addition, the feature sequence Learnable Prototypes input together to group visual semantics across different granularities. Learnable prototypes and token The similarity matrix A between can be defined as:

[0079]

[0080] where γ j are independent and identically distributed random samples drawn from a Gumbel(0,1) distribution, and w k are the linear projection weights of the prototype and visual tokens respectively. Then, we transform the feature sequence x with different weights into l Aggregate to Update each prototype Then all prototypes are averaged As shown below:

[0081]

[0082] where w o and w v is the weight used to project and merge features. The regular grid structure does not constrain this block, and the information can be reorganized into arbitrary image fragments. For bottom-up encoding, the features of each level are obtained as:

[0083]

[0084] in Is from The CLS token.

[0085] Step 8: Introduce the reverse cross-granular motion embedding (CME), which aims to use higher-level information to guide the learning of low-level features. Specifically, it takes the output tokens of higher levels as and low-level visual tokens x l As input, to achieve complementary information interaction between different granularities, the final feature o is obtained through multi-head cross attention. l , as shown below:

[0086]

[0087] Step 9: Forward propagation process: individual → group → global, embed the patch into f in UBE patch ∈R N×d Input into its UME module to obtain the individual features f aggregated across granularities in turn i ′ nd ∈R Q×d 、Group characteristics g ′ ro ∈R L×d and global features Right now:

[0088] f i ′ nd =UME p2i (f patch )

[0089] f g ′ ro =UME i2g (f i ′ nd )

[0090] f glo =UME g2g (f g ′ ro )

[0091] The subscripts of UME represent input and output of different granularities.

[0092] Step 10: Backward propagation process: group ← global. The global feature f obtained at the global level glo Together with the group-level visual tokens, they are input into CME to obtain the final group representation f gro ∈R L×d , expressed as:

[0093] f gro =CME g2g (f glo )

[0094] CME g2g Aims to exploit global information to guide group-level representation learning.

[0095] Step 11: Backward propagation process: individual ← group; similarly, the feature representation f obtained at the group level gro Together with the individual-level visual tokens, they are input into CME to obtain the final individual representation f ind ∈R Q×d , as shown below:

[0096] f ind =CME g2i (CME g2g (f glo ))=CME g2i (f gro )

[0097] CME g2i Aims to utilize group information to guide individual-level representation learning.

[0098] Step 12: The overall training loss of AdaFPP is combined with the conventional detection loss L det and multi-granularity recognition loss L rec , the detection loss is defined as:

[0099]

[0100] in represents the Intersection-over-Union (IoU) loss of the bounding box regression loss, represents the cross entropy loss for binary classification, represents the cross entropy loss for classification. reg is a constant scalar that balances the contribution of the loss term, and its default value is 5.

[0101] The recognition loss is defined as:

[0102]

[0103] in and They represent the binary cross entropy loss functions for individual, group, global activity recognition, and group detection respectively. The overall training loss formula of AdaFPP is as follows:

[0104]

[0105] Among them, λ is the weight coefficient.

[0106] Table 1 Comparative performance of panoramic activity recognition

[0107]

[0108]

[0109] Table 1 shows the comparison of the results of this method with other methods on JRDB-PAR based on JRDB and JRDB-Act datasets, with F a As the evaluation index, F a =Average(F i +F p +F g ),Average(·) represents the average value.

[0110] Obviously, those skilled in the art should understand that the various steps of the panoramic activity recognition method based on adaptive two-way propagation prototype learning of the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented with executable program codes of the computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A panoramic activity recognition method based on adaptive bidirectional propagation prototype learning, characterized in that: The method achieves joint recognition of multi-granularity activities in panoramic scenes through an adaptive focus detector and a bidirectional propagation prototype learner. The adaptive focusing detector and the bidirectional propagation prototype learner constitute the adaptive focusing bidirectional propagation prototype learning framework AdaFPP. The implementation of the method includes the following steps: Step 1: Input the video, the detection network outputs the original detection box, which is then input into the panoramic adaptive focuser to obtain the final size-adaptive detection; Step 2: Encode the input video using the pre-trained encoder to obtain a feature map, and use the adaptive detection obtained in step 1 to obtain the features of each individual; Step 3: Flatten the individual features obtained in step 2 according to the ViT method, project all patches through linear projection, and obtain patch embeddings; Step 4: Input the patch embedding obtained in step 3 into the bidirectional propagation prototype learner to obtain the final feature representations at different levels for recognition; Step 5: The entire adaptive focused bidirectional propagation prototype learning framework is jointly trained through the detection loss and the multi-granularity recognition loss.

2. The method for panoramic activity recognition based on adaptive bidirectional propagation prototype learning according to claim 1 is characterized in that: The step 1 comprises the following specific steps: For the input panoramic video, the final size adaptive detection is obtained through the following specific steps: Step 201: adaptively adjusting the size of the original detection bounding box by an adaptive object adjustment strategy; using a set expansion ratio to expand the width and height of each bounding box in the original detection frame set from the center thereof to obtain an expanded detection frame set; Step 202: select dense sub-regions through a dense region merging strategy to obtain a collected dense sub-region set; Step 203: cropping and inputting the selected dense sub-regions into the detection network to obtain fine-grained detection, integrating the original detection and the fine-grained detection through the detection fusion strategy to obtain the final size-adaptive detection; based on the dense sub-region set obtained in step 202, cropping the corresponding sub-regions from the original frame, and inputting them into the detection network to obtain fine-grained detection; Subsequently, the final size-adaptive detection is computed via non-maximum suppression.

3. The method for panoramic activity recognition based on adaptive bidirectional propagation prototype learning according to claim 1, characterized in that: The execution process of the unified bidirectional encoding block included in the bidirectional propagation prototype learner is as follows: Step 301: Given a feature sequence as the input of a unified bidirectional encoding block, all UBE blocks aim to encode multi-granularity features by learning multi-granularity prototypes. The UBE block consists of two parts: bottom-up encoding UME and top-down encoding CME; Step 302: the learnable embedding is added to the front of the given feature sequence, and in addition, the learnable position embedding vector is added to obtain the input token, and the output is obtained based on the input token by using the multi-head self-attention mechanism, and the output is the token processed by the multi-head self-attention mechanism; Step 303: Input the feature sequence together with the learnable prototype into the bottom-up encoding module to group the visual semantics across different granularities; then update each learned prototype by aggregating the feature sequences with different weights together, and then average all the prototypes; for bottom-up encoding, obtain the features of each level; Step 304: Inverse cross-granularity motion embedding is introduced, aiming to utilize higher-level information to guide the learning of low-level features; specifically, higher-level output tokens and low-level visual tokens are used as input to achieve complementary information interaction between different granularities, and the final features are obtained through multi-head cross attention.

4. The method for panoramic activity recognition based on adaptive bidirectional propagation prototype learning according to claim 1, characterized in that: The two-way propagation process of UBE is as follows: Step 401: Given the patch embeddings from the video encoder, a bidirectional hierarchical network is constructed using UBE to model activities at different granularities, including two processes: forward and backward; Step 402: Forward propagation process: individual → group → global; In UBE, the patch embedding is input into its UME module to sequentially obtain individual features, group features, and global features aggregated across granularities; Step 403: Backward propagation process: group ← global; the global features obtained at the global level are input into CME together with the visual tokens at the group level to obtain the final group representation; Step 404: Backward propagation process: individual ← group; Similarly, the feature representation obtained at the group level is input into the CME together with the visual tokens at the individual level to obtain the final individual representation.

5. The method for panoramic activity recognition based on adaptive bidirectional propagation prototype learning according to claim 1, characterized in that: The joint optimization process of step 5 is as follows: The overall training loss of AdaFPP is combined with the conventional detection loss L det and multi-granularity recognition loss L rec , the detection loss is defined as: in represents the intersection-over-union loss of the bounding box regression loss, represents the cross entropy loss for binary classification, represents the cross entropy loss for classification; λ reg is a constant scalar contributing to the balance loss term; The recognition loss is defined as: in and They represent the binary cross entropy loss functions for individual, group, global activity recognition, and group detection respectively; the overall training loss formula of AdaFPP is as follows: Among them, λ is the weight coefficient.

6. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a panoramic activity recognition method based on adaptive bidirectional propagation prototype learning as described in any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the panoramic activity recognition method based on adaptive bidirectional propagation prototype learning as described in any one of claims 1 to 5.

Images