Multi-view positioning detection method and device

Abstract

This invention discloses a multi-view localization and detection method and apparatus. The method includes: acquiring video data input from multiple viewpoints; extracting corresponding first feature maps for each viewpoint using a first neural network; inputting the first feature maps of the multiple viewpoints into a second cascaded neural network to obtain second feature maps at each level output by the second cascaded neural network; performing correlation modeling processing on the second feature maps at each level between adjacent viewpoints to obtain similarity matrix data between adjacent viewpoints; aggregating the similarity matrix data between adjacent viewpoints to obtain fused feature data; concatenating the fused feature data of the multiple viewpoints to obtain a global feature map; and determining the temporal nomination and action classification of at least one event based on the global feature map to achieve localization of the video data from multiple viewpoints. The overall modeling of the video data from multiple viewpoints enables information interaction and mutual enhancement among the viewpoints, achieving event localization based on global features.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-viewing angle positioning detection method and device. Background Art

[0002] Most of the time domain event detection models based on deep models are used to locate videos. This model mainly includes two categories. One is a two-step model that completes action location in two consecutive steps, that is, first completing the time sequence nomination (determining the start time and end time of the action), and then classifying the nominated action (determining the action category of the located segment). The other is a single-step model that completes time sequence nomination and action classification at the same time. The two-step model generally obtains preliminary action nomination candidates through time domain windowing, and then uses a neural network to classify and sort these action nomination candidates, and finally obtains the event detection result. The single-step model is inspired by the image object detection algorithm, using the anchor points in the one-dimensional feature map to represent the time sequence nomination, and then uses the network to classify the anchor point features and regress in the time domain to obtain the event detection result.

[0003] However, existing time-domain event detection models generally only emphasize the use of single-lens video input and only use a single perspective for action positioning. For multi-lens situations, if the above model is used for each lens, it will consume huge computing resources and cannot model the correlation and complementarity between multiple lenses. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a multi-view positioning detection method and device that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a multi-view positioning detection method is provided, which includes:

[0006] Obtain video data input from multiple perspectives, and use a first neural network to extract corresponding first feature maps of the multiple perspectives;

[0007] The first feature maps of multiple perspectives are respectively input into the second cascade neural network to obtain the second feature maps of each level output by the second cascade neural network, and the second feature maps of each level between two adjacent perspectives are subjected to association modeling processing to obtain the similarity matrix data of the two adjacent perspectives;

[0008] Aggregate the similarity matrix data of each two adjacent perspectives to obtain fused feature data;

[0009] The fused feature data from multiple perspectives are spliced ​​together to obtain a global feature map;

[0010] The temporal nomination and action classification of at least one event are determined according to the global feature map to realize the positioning of multi-view video data.

[0011] According to another aspect of the present invention, a multi-view positioning detection device is provided, comprising:

[0012] An acquisition module, adapted to acquire video data input from multiple perspectives, and extract corresponding first feature maps of multiple perspectives using a first neural network;

[0013] An association module is adapted to input the first feature maps of multiple perspectives into the second cascade neural network respectively, obtain the second feature maps of each level output by the second cascade neural network, and perform association modeling processing on the second feature maps of each level between two adjacent perspectives to obtain similarity matrix data of the two adjacent perspectives;

[0014] An aggregation module is adapted to aggregate the similarity matrix data of two adjacent perspectives to obtain fused feature data;

[0015] A splicing module is suitable for splicing fused feature data from multiple perspectives to obtain a global feature map;

[0016] The positioning module is adapted to determine the time sequence nomination and action classification of at least one event according to the global feature map to realize the positioning of multi-view video data.

[0017] According to another aspect of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0018] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned multi-view positioning detection method.

[0019] According to yet another aspect of the present invention, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations corresponding to the above-mentioned multi-view positioning detection method.

[0020] According to the multi-view positioning detection method and device of the present invention, the video data input from multiple perspectives are modeled as a whole, and based on the complementarity of different perspectives for the same event, the information of the video data from each perspective is interacted and enhanced with each other, thereby constructing a global feature map to complete the positioning of the event.

[0021] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0023] Figure 1 A flow chart of a multi-view positioning detection method according to an embodiment of the present invention is shown;

[0024] Figure 2 A schematic diagram of the multi-view positioning detection network structure is shown;

[0025] Figure 3 A functional block diagram of a multi-view positioning detection device according to an embodiment of the present invention is shown;

[0026] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0028] Figure 1 FIG. 4 is a flow chart showing a multi-view positioning detection method according to an embodiment of the present invention. Figure 1 As shown, the multi-view positioning detection method specifically includes the following steps:

[0029] Step S101, obtain video data input from multiple perspectives, and use a first neural network to extract corresponding first feature maps of multiple perspectives.

[0030] In this embodiment, the video data input from multiple perspectives can be collected by using multiple cameras to obtain video data from multiple perspectives. Figure 2As shown in the figure, video data can be collected from k perspectives. The video data obtained from each perspective has different directions and collection angles. The video data between different perspectives are all collected for the same scene. The scenes include video scenes for sports events and video scenes for live events. Video data from different perspectives can be collected by multiple cameras for positioning, which is convenient for subsequent directing and editing.

[0031] After obtaining video data input from multiple perspectives, a first neural network, such as a 3D convolutional neural network, is used to extract data from the video data from multiple perspectives, such as Figure 2 As shown, the first feature map is obtained. The first feature map is the aggregation of the extracted one-dimensional features. The extracted one-dimensional features can be expressed as f k，i , where k represents k viewing angles, i ranges from 0 to T'-1, T' is T' time periods of the video data, and i corresponds to each time period. The first feature map is obtained by aggregating the one-dimensional features of each viewing angle and integrating the features of different time points in units of viewing angle.

[0032] In step S102, the first feature maps of multiple perspectives are respectively input into the second cascade neural network to obtain the second feature maps of each level output by the second cascade neural network, and the second feature maps of each level between two adjacent perspectives are subjected to association modeling processing to obtain the similarity matrix data of the two adjacent perspectives.

[0033] The obtained multi-view first feature maps are respectively input into the respective second cascade neural networks to obtain the respective second feature maps. The second cascade neural network is composed of multiple cascaded one-dimensional convolutional neural networks, such as Figure 2 As shown, the first feature maps of each perspective are respectively input into the respective second cascade neural networks, and the second feature maps of each level are outputted at each level of the second cascade neural network. For example, the first feature map of perspective a is input into the second cascade neural network to obtain the second feature map Where a represents the a perspective, and n represents the nth level of the second cascade neural network. The first feature map of the corresponding b perspective is input into the second cascade neural network to obtain the second feature map After obtaining the second feature map of each perspective, based on the complementarity between the video data of different perspectives of the same scene from multiple perspectives, the second feature maps of each level between two adjacent perspectives are associated with each other and modeled to enhance the feature maps between multiple perspectives and construct the association relationship between the feature maps between multiple perspectives.

[0034] Specifically, based on Figure 2The multi-view association processing unit between any two adjacent perspectives models the second feature maps at all levels of the two adjacent perspectives according to the similarity of the time domain position to obtain the similarity matrix data at all levels of the two adjacent perspectives. Taking perspective a and perspective b as an example, the similarity of the i time domain position of perspective a and the j time domain position of perspective b is modeled using dot multiplication, wherein the time domain modeling needs to fuse the feature maps at the same time domain position in perspective b and perspective a, and also needs to consider all feature maps of the entire time period. The multi-view association processing unit uses the feature embedding function to perform the dot multiplication of the second feature maps at all levels of the two adjacent perspectives for modeling. For example, using the following formula:

[0035]

[0036] Among them, θ and φ are feature embedding functions, and the feature embedding function is used to process the second feature maps of the a and b perspectives, and the second feature map of the b perspective is transposed (for example, the transposition function T is used to Transpose to achieve), get the similarity matrix data It is an element in the similarity matrix S. According to the specific implementation, the required feature embedding function is used to associate the features of the second feature maps from different perspectives. When modeling, Figure 2 As shown, any two adjacent perspectives are modeled.

[0037] Step S103, aggregating the similarity matrix data of each two adjacent perspectives to obtain fused feature data.

[0038] After obtaining the similarity matrix data of each adjacent view, for any two adjacent views, the similarity matrix data of the two adjacent views are normalized, and the normalization process can be performed by, for example, softmax. After the normalization process, an attention map can be obtained, and the processed similarity matrix data can be aggregated with the feature embedding function of the second feature map of the full time domain position of the first view in the two adjacent views through the attention mechanism to obtain fused feature data. The following formula can be used:

[0039]

[0040] Among them, γ is the feature embedding function, and perspective a and perspective b are examples. In the specific implementation, if there are k perspectives, modeling association is performed between every two perspectives to obtain k (k-1) fused feature data.

[0041] Furthermore, the feature data may be connected and fused with the input first feature map or second feature map based on the residual structure, so as to further optimize the processing of multi-view video data.

[0042] Step S104, splicing the fused feature data of multiple perspectives to obtain a global feature map.

[0043] After processing the video data of k perspectives as above, k(k-1) fused feature data are obtained, and the obtained fused feature data are spliced ​​to obtain a global feature map. The global feature map is constructed based on the video data of all perspectives and the mutual correlation relationship. The information between each perspective interacts, enhances and complements each other, which can better reflect the correlation between multiple perspectives.

[0044] Step S105, determining the time sequence nomination and action classification of at least one event according to the global feature map to achieve positioning of multi-view video data.

[0045] After obtaining the global feature map, the video data can be located based on the temporal sequence of the global feature map. Figure 2 As shown, frame selection is performed to determine the timing nomination, action classification, etc. Further, in order to make the positioning detection result more accurate, this embodiment also reorders the events in time to remove events below the quality threshold. Specifically, for each anchor point on the global feature map (a certain segment in the time domain, belonging to a certain period of time of the video data), corresponding to a certain timing nomination, such as anchor point t, the corresponding timing center position and timing width can be expressed as:

[0046] a c =(t+0.5) / T′, a w =r d / T′

[0047] Among them, a c is the time series center position, a w is the timing width. d is the scaling value of the time domain scale, which can be set according to the implementation, such as 1.0. T' is the time period of the video data.

[0048] The global feature map is predicted using a specified convolutional neural network. The specified convolutional neural network can use a one-dimensional convolutional neural network. Three one-dimensional convolutional neural networks are used to predict the event classification score, time domain positioning offset, and aliasing score. Event classification score represents the predicted probability of C action events and one background event. The temporal localization offset (Δc, Δw) represents the relative time sequence nomination (a c , a w ) can be corrected based on the time domain positioning offset. For any event, the timing center position and timing width of the event are corrected according to the time domain positioning offset, as shown below:

[0049]

[0050] in, and The timing center position and timing width nominated for the corrected timing. α1 and α2 are used to control the influence of time offset and can be set according to the implementation situation, such as setting it to 0.1, which is not limited here.

[0051] The event classification score can be obtained using the time domain event detection model, which will not be explained here. op Represents the intersection of the temporal nomination and the real timestamp. The real timestamp is used to guide model training. When using a specified convolutional neural network for training, the temporal nomination of the global feature map at each time domain scale can be collected for optimization. The optimization loss function can construct a multi-task loss, such as Figure 2 As shown in , it includes classification loss, positioning offset loss and aliasing loss to optimize the specified convolutional neural network. When performing positioning detection, the confidence score of the event is calculated based on the obtained event classification score and aliasing score, and the event classification score s of each event is calculated based on the event classification score s. e and the aliasing fraction s op Calculate, s f =max(s e )·s op ,s f It is the confidence score, which represents the ranking score of each time series nomination. Based on the confidence score, the non-maximum suppression algorithm (NMS) can be used to adjust the event classification score, increase the score of high-quality results, and reduce the score of low-quality results. Events with confidence scores lower than the quality threshold are screened out, and the events corresponding to the time period are removed according to the corrected time series center position and time series width, so that they are more accurately positioned and more reasonable.

[0052] Optionally, this embodiment can be used for various video scenes such as intelligent directing, video editing, etc., to perform on-site production of video input collected in real time from multiple perspectives, and to edit and broadcast corresponding videos according to the director's requirements, such as sports event videos, etc. By using multiple cameras, video inputs of sports events from multiple different perspectives can be obtained and modeled as a whole. The same event in the event can be complemented and enhanced from different perspectives, thereby facilitating the realization of intelligent directing, etc.

[0053] According to the multi-view positioning detection method provided by the present invention, the video data input from multiple perspectives are modeled as a whole, and based on the complementarity of different perspectives for the same event, the information of the video data from each perspective is interacted and enhanced with each other, thereby constructing a global feature map to complete the positioning of the event.

[0054] Figure 3A functional block diagram of a multi-view positioning detection device according to an embodiment of the present invention is shown.

[0055] like Figure 3 As shown, the multi-view positioning detection device includes the following modules:

[0056] An acquisition module 310 is adapted to acquire video data input from multiple perspectives, and to extract corresponding first feature maps of multiple perspectives using a first neural network;

[0057] The association module 320 is adapted to input the first feature maps of multiple perspectives into the second cascade neural network respectively, obtain the second feature maps of each level output by the second cascade neural network, and perform association modeling processing on the second feature maps of each level between two adjacent perspectives to obtain similarity matrix data of the two adjacent perspectives;

[0058] Aggregation module 330, adapted to aggregate similarity matrix data of two adjacent perspectives to obtain fused feature data;

[0059] A splicing module 340 is adapted to splice the fused feature data of multiple perspectives to obtain a global feature map;

[0060] The positioning module 350 is adapted to determine the time sequence nomination and action classification of at least one event according to the global feature map, so as to realize the positioning of multi-view video data.

[0061] Optionally, the second cascade neural network consists of multiple one-dimensional convolutional neural networks;

[0062] The association module 320 is further adapted to:

[0063] Inputting the first feature maps of multiple perspectives into the corresponding second cascade neural networks respectively to obtain the second feature maps of each level corresponding to each perspective;

[0064] Based on the multi-view correlation processing unit, the second feature maps at all levels of the two adjacent viewpoints are modeled according to the similarity of the time domain position to obtain the similarity matrix data of each level of the two adjacent viewpoints; wherein the multi-view correlation processing unit uses the feature embedding function to multiply the second feature maps at all levels of the two adjacent viewpoints for modeling.

[0065] Optionally, the aggregation module 330 is further adapted to:

[0066] For any two adjacent perspectives, the similarity matrix data of the two adjacent perspectives are normalized, and the processed similarity matrix data is aggregated with the feature embedding function of the second feature map of the full time domain position of the first perspective in the two adjacent perspectives to obtain fused feature data.

[0067] Optionally, the aggregation module 330 is further adapted to:

[0068] The feature data is connected and fused with the first feature map and / or the second feature map based on the residual structure.

[0069] Optionally, the viewing angle is k viewing angles;

[0070] The splicing module 340 is further adapted to:

[0071] The k(k-1) fused feature data of k perspectives are spliced ​​to obtain a global feature map.

[0072] Optionally, the positioning module 350 is further adapted to:

[0073] Based on the global feature map, a temporal nomination and an action classification of at least one event are determined, and a temporal reordering of at least one event is performed to remove events below a quality threshold.

[0074] Optionally, the positioning module 350 is further adapted to:

[0075] Use the specified convolutional neural network to predict the global feature map and obtain the event classification score, time domain positioning offset and aliasing score;

[0076] For any event, the time center position and time width of the event are corrected according to the time domain positioning offset; the confidence score of the event is calculated according to the event classification score and the aliasing score; the events with confidence scores lower than the quality threshold are screened out, and the corresponding events are removed according to the corrected time center position and time width.

[0077] The description of each module above refers to the corresponding description in the method embodiment and will not be repeated here.

[0078] The present application also provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the multi-view positioning detection method in any of the above method embodiments.

[0079] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0080] like Figure 4 As shown, the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0081] in:

[0082] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0083] The communication interface 404 is used to communicate with other devices such as clients or other servers.

[0084] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the above-mentioned multi-view positioning detection method embodiment.

[0085] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0086] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the electronic device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0087] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0088] The program 410 can be specifically used to enable the processor 402 to execute the multi-view positioning detection method in any of the above method embodiments. The specific implementation of each step in the program 410 can refer to the corresponding descriptions in the corresponding steps and units in the above multi-view positioning detection embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.

[0089] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0090] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0091] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0092] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0093] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0094] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the multi-view positioning detection device according to an embodiment of the present invention. The present invention may also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0095] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A multi-view positioning detection method, characterized in that: Methods include: Obtain video data input from multiple perspectives, and use a first neural network to extract corresponding first feature maps of the multiple perspectives; The first feature maps of multiple perspectives are respectively input into the second cascade neural network to obtain the second feature maps of each level output by the second cascade neural network, and the second feature maps of each level of two adjacent perspectives are modeled by point multiplication according to the similarity of the time domain position using the feature embedding function based on the multi-perspective association processing unit to obtain the similarity matrix data of the two adjacent perspectives; Normalizing the similarity matrix data of each adjacent two perspectives, and aggregating the processed similarity matrix data with the feature embedding function of the second feature map of the full time domain position of the first perspective of the two adjacent perspectives to obtain fused feature data; The fused feature data from multiple perspectives are spliced ​​together to obtain a global feature map; The temporal nomination and action classification of at least one event are determined according to the global feature map to achieve positioning of multi-view video data.

2. The method according to claim 1, characterized in that The second cascade neural network is composed of multiple one-dimensional convolutional neural networks; The step of inputting the first feature maps of multiple perspectives into the second cascade neural network respectively to obtain the second feature maps of each level output by the second cascade neural network further comprises: The first feature maps of multiple perspectives are respectively input into the corresponding second cascade neural networks to obtain the second feature maps of each level corresponding to each perspective.

3. The method according to claim 1, characterized in that: The aggregating the similarity matrix data of each two adjacent perspectives to obtain fused feature data further includes: The fused feature data is connected with the first feature map and / or the second feature map based on a residual structure.

4. The method according to any one of claims 1 to 3, characterized in that The number of viewing angles is k; The step of splicing the fused feature data from multiple perspectives to obtain a global feature map further includes: The k(k-1) fused feature data of k perspectives are spliced ​​to obtain a global feature map.

5. The method according to any one of claims 1 to 3, characterized in that: The step of determining the time sequence nomination and action classification of at least one event according to the global feature map to achieve positioning of multi-view video data further includes: Based on the global feature map, a time sequence nomination and an action classification of at least one event are determined, and the time sequence of the at least one event is reordered to remove events below a quality threshold.

6. The method according to claim 5, characterized in that The reordering of the at least one event to remove events below a quality threshold further comprises: Using a specified convolutional neural network to predict the global feature map, obtain an event classification score, a temporal positioning offset, and an aliasing score; For any event, the timing center position and timing width of the event are corrected according to the time domain positioning offset; the confidence score of the event is calculated according to the event classification score and the aliasing score; events with confidence scores lower than the quality threshold are screened out, and the corresponding events are removed according to the corrected timing center position and timing width.

7. A multi-view positioning detection device, characterized in that: The device includes: An acquisition module, adapted to acquire video data input from multiple perspectives, and extract corresponding first feature maps of multiple perspectives using a first neural network; The association module is adapted to input the first feature maps of multiple perspectives into the second cascade neural network respectively, obtain the second feature maps of each level output by the second cascade neural network, and perform modeling processing on the second feature maps of each level of two adjacent perspectives by multiplying the similarity points of the time domain positions based on the multi-perspective association processing unit using the feature embedding function to obtain the similarity matrix data of the two adjacent perspectives; an aggregation module, adapted to normalize the similarity matrix data of each adjacent two perspectives, and aggregate the processed similarity matrix data with the feature embedding function of the second feature map of the full time domain position of the first perspective of the two adjacent perspectives to obtain fused feature data; A splicing module is suitable for splicing fused feature data from multiple perspectives to obtain a global feature map; The positioning module is adapted to determine the time sequence nomination and action classification of at least one event according to the global feature map to achieve positioning of multi-view video data.

8. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the multi-view positioning detection method as described in any one of claims 1-6.

9. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the multi-view positioning detection method as described in any one of claims 1-6.

Images