Event-based multimedia segment classification from cross-modal features of multimedia content

Abstract

An electronic device for event-based multimedia segment classification from cross-modal features of multimedia content is disclosed. The electronic device receives multimedia content including a set of multimedia segments associated with a plurality of modalities. The electronic device determines, for each modality, a set of features associated with the multimedia content, based on an encoder model and applies a bottleneck attention model on the set of features. The electronic device determines cross-modal fused features for the plurality of modalities, based on the application of the bottleneck attention model and segregates cross-modal fused features into a set of nodes, based on one or more events associated with the multimedia content. The electronic device applies a classifier model on the segregated cross-modal fused features and classifies each multimedia segment into the one or more events, based on application of the classifier model.

Description

Docket No. SYP354615WO02 EVENT-BASED MULTIMEDIA SEGMENT CLASSIFICATION FROM CROSS-MODAL FEATURES OF MULTIMEDIA CONTENT CROSS-REFERENCE TO RELATED APPLICATIONS / INCORPORATION BY REFERENCE

[0001] This Application also makes reference to Indian Provisional Patent Application Ser. No. 202411015338, which was filed on March 01, 2024. The above stated Patent Application is hereby incorporated herein by reference in its entirety. FIELD

[0002] Various embodiments of the disclosure relate to image processing. More specifically, various embodiments of the disclosure relate to an electronic device and a method for event-based multimedia segment classification from cross-modal features of multimedia content. BACKGROUND

[0003] In media and entertainment, detection of anomalies (for example, violence and nudity) in image segments may involve development of algorithms and techniques to automatically identify and classify explicit or inappropriate content within specific regions or sections of an image. The anomaly detection (such as, detection of violence and nudity) in image segments may typically involve the use of machine learning and deep learning models. These models may be trained on large datasets that include labeled examples of explicit or inappropriate content. The models may learn to recognize patterns, features, and visual cues that may be indicative of violence or nudity. Such models may analyze various aspects of an image, such as color, texture, shape, and context. The models may identify explicit body parts, weapons, blood, or other visual elements associated with violent or explicit content. The detection of anomalies in image segments is a challengingDocket No. SYP354615WO02 problem due to the wide range of variations in visual content, cultural differences, and the constant evolution of explicit content.

[0004] Limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of described systems with some aspects of the present disclosure, as set forth in the remainder of the present application and with reference to the drawings. SUMMARY

[0005] An electronic device and method for event-based multimedia segment classification from cross-modal features of multimedia content is provided substantially as shown in, and / or described in connection with, at least one of the figures, as set forth more completely in the claims.

[0006] These and other features and advantages of the present disclosure may be appreciated from a review of the following detailed description of the present disclosure, along with the accompanying figures in which like reference numerals refer to like parts throughout. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG.1 is a block diagram that illustrates an exemplary network environment for event-based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure.

[0008] FIG.2 is a block diagram that illustrates an electronic device of FIG.1 for event- based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure.

[0009] FIG.3 is a diagram that illustrates an exemplary processing pipeline of event- based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure.Docket No. SYP354615WO02

[0010] FIG.4A and FIG.4B collectively illustrate an exemplary architecture of a model for event-based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure

[0011] FIG.5 is an exemplary graph illustrating an exemplary scenario for classification of event-based set of multimedia segments based on cross-modal fusion adaptor, in accordance with an embodiment of the disclosure.

[0012] FIG.6 is a flowchart that illustrates operations of an exemplary method for event- based multimedia segment classification based on cross-modal fusion adaptor, in accordance with an embodiment of the disclosure. DETAILED DESCRIPTION

[0013] The following described implementations may be found in a disclosed electronic device and a method for event-based multimedia segment classification based on cross- modal fusion adaptor. Exemplary aspects of the disclosure may provide an electronic device that may receive multimedia content (for example, videos, images, moving pictures, and the like) including a set of multimedia segments associated with plurality of modalities (for example, audio, video, and the like). The electronic device may determine a set of features associated with the received multimedia content for each modality of the plurality of modalities (for example, audio, video, and the like), based on an encoder model (for example, audio encoder, visual encoder, and the like). Further, the electronic device may apply a bottleneck attention model on the determined set of features and determine cross- modal fused features (such as, cross-modal aware features) for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model. The determined cross-modal fused features may be segregated into a set of nodes (for example, a node “A”, node “B”, and the like) based on one or more events (for example, a concert, a surveillance footage, and the like)Docket No. SYP354615WO02 associated with the received multimedia content and apply a classifier model on the segregated cross-modal fused features. The electronic device may classify each multimedia segment into one or more events, based on the application of the classifier model.

[0014] The detection of anomalies (for example, violence, nudity, and the like) in a video is a widely recognized computer vision challenge that holds potential for various practical applications, including surveillance systems, crime prevention, and content moderation. However, the determination of annotations for anomalies at the frame level in the videos can be both expensive and time-consuming. Recently, techniques such as, Weakly Supervised Video Anomaly Detection (WS-VAD) model, are used to detect anomaly events such as, violence and nudity, in videos using only video-level labels. For example, in a scenario of surveillance videos and associated labelled dataset, a WS-VAD label of the video may be either “normal” or “anomalous”. The WS-VAD model may learn to identify abnormal events, such as, a person with a weapon or a person involved in violent behavior, based on the video-level labels. This approach may reduce the annotation effort and makes it feasible to detect anomalies in large scale video datasets. However, the video anomaly detection task may have significant challenges, such as, issues related to imbalanced modality information and consistent differentiation between normal and abnormal features.

[0015] The disclosed electronic device may receive multimedia content (for example, videos, images, moving pictures, and the like) including a set of multimedia segments associated with plurality of modalities (for example, audio, video, and the like). The electronic device may determine a set of features associated with the received multimedia content for each modality of the plurality of modalities (for example, audio, video, and the like), based on an encoder model (for example, audio encoder, visual encoder, and theDocket No. SYP354615WO02 like). Further, the electronic device may apply a bottleneck attention model on the determined set of features and determine cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model. The determined cross-modal fused features may be segregated into a set of nodes (for example, node “A”, node “B” and the like) based on one or more events (for example, a concert, surveillance video, and the like) associated with the received multimedia content and apply a classifier model on the segregated cross- modal fused features. The electronic device may classify each multimedia segment into one or more events, based on the application of the classifier model. The bottleneck attention model may facilitate effective multi-modal interaction based on dynamic regulation of a contribution of each modality. Also, the bottleneck attention model may incorporate a prefix-tuned bottleneck attention adapter and a modulation model. The bottleneck attention model including the bottleneck attention adapter may help to adapt the set of features of each modality to a shared bottleneck representation, that may ensure seamless interaction between modalities and may also preserve modality-specific characteristics.

[0016] Thus, the electronic device may detect anomaly events such as violence and nudity accurately. The electronic device may address imbalanced modality information issues and effectively facilitate multi-modal interaction by dynamic selection of relevant features (for example, audio features) with corresponding visual features. Additionally, the electronic device may determine hierarchical relationships within normal and abnormal representations, which may improve the accuracy of differentiation between the normal and abnormal features. In an example of a security system installed in a public space, such as an airport, the electronic device may detect anomaly events like violence and nudity by processing the multimedia content received from the security system. ForDocket No. SYP354615WO02 instance, if someone gets involved in a physical altercation or attempts to expose themselves, the electronic device may promptly identify and alert the security personnel. Moreover, the electronic device may address imbalanced modality information issues. For example, an audio input may be crucial for detection of a potential threat, such as, threatful remarks from a person. The electronic device may select the relevant audio features and combine them with corresponding visual features to, for example, analyze body language or suspicious movements. Such a comprehensive approach may ensure that no vital information is overlooked, which may enhance the overall effectiveness of the security system. Additionally, the electronic device may determine the hierarchical relationships within the normal and abnormal representations. For example, the electronic devices may learn and recognize patterns of normal behavior, such as, people walking or talking, and distinguish them from abnormal behavior, such as, someone running or acting erratically. Based on the determination of the hierarchical structure of such representations, an accuracy of the electronic device may increase. For example, the electronic device may be able to more accurately separate the normal and abnormal features, reduce false alarms, and improve overall security measures of the security system.

[0017] FIG.1 is a block diagram that illustrates an exemplary network environment for event-based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure. With reference to FIG.1, there is shown a block diagram of a network environment 100. The network environment 100 may include an electronic device 102, a server 104, a communication network 106. The electronic device 102 may include or may be associated with a bottleneck attention model 112, and a classifier model 114. Further, the electronic device 102 may receive set of multimedia segments 110 as input. The server 104 may include and / or may be associated with a database 108. The electronic device 102, the server 104, and theDocket No. SYP354615WO02 database 108, may be communicatively coupled through one or more networks (such as, the communication network 106).

[0018] The electronic device 102 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive multimedia content including the set of multimedia segments 110 associated with a plurality of modalities (for example, image frames, video data, or moving pictures and the like). The electronic device 102 may determine a set of features (for example, visual features, audio features, and the like) associated with the received multimedia content for each modality. The electronic device 102 may apply the bottleneck attention model 112 on the determined set of features. The bottleneck attention model 112 may include a bottleneck layer to reduce a dimensionality of an input sequence (for example, set of multimedia segments 110), based on a compression of the set of multimedia segments 110 and preservation of important features (for example, text, images, and the like). The bottleneck layer may act as a bottleneck, that may force the bottleneck attention model 112 to focus on the most relevant information and discard less important details. This compression may help to reduce computational requirements and a number of parameters in the bottleneck attention model 112. The bottleneck layer is typically a fully connected layer or a convolutional layer (CNN) that may reduce the dimensionality of the set of multimedia segments 110. The electronic device 102 may determine cross-modal fused features for the various modalities with the received multimedia content, based on the application of the bottleneck attention model 112. The electronic device 102 may segregate the determined cross-modal fused features into a set of nodes (for example, a node “A”, a node “B”, and the like) based on one or more events associated with the received multimedia content. The electronic device 102 may apply the classifier model 114 on the segregated cross-modal fused features and classify each multimedia segment of the set of multimedia segments 110 into the one or more events.Docket No. SYP354615WO02 The classification of the set of multimedia segments 110 may be based on the application of the classifier model 114. Examples of the electronic device 102 may include, but may not be limited to, a desktop, a tablet, a television (TV), a laptop, a computing device, a smartphone, a cellular phone, a mobile phone, a machine learning computing device (enabled with or hosting, for example, a computing resource, a memory resource, and a networking resource), a consumer electronic (CE) device having a display.

[0019] The server 104 that may include suitable logic, circuitry, interfaces, and / or code configured to receive requests from the electronic device 102 to receive the multimedia content associated with the plurality of modalities. The server 104 may be configured to extract a set of features associated with the multimedia content. Further, the server 104 may be configured to extract cross-modal fused features for the modalities associated with the multimedia content. The server 104 may be configured to segregate the cross-modal fused features into the set of nodes based on the one or more events. The server 104 may be configured to extract the set of nodes. The server 104 may be configured to extract the one or more events based on the classifier model 114.

[0020] The server 104 may be implemented as a cloud server and may execute operations through web applications, cloud applications, HTTP requests, repository operations, file transfer, and the like. Other example implementations of the server 104 may include, but are not limited to, a database server, a file server, a web server, a media server, an application server, a mainframe server, a machine learning server (enabled with or hosting, for example, a computing resource, a memory resource, and a networking resource), or a cloud computing server.

[0021] In at least one embodiment, the server 104 may be implemented as a plurality of distributed cloud-based resources by use of several technologies that are well known to those ordinarily skilled in the art. A person with ordinary skill in the art will understand thatDocket No. SYP354615WO02 the scope of the disclosure may not be limited to the implementation of the server 104 and the electronic device 102, as two separate entities. In certain embodiments, the functionalities of the server 104 can be incorporated in its entirety or at least partially in the electronic device 102 without a departure from the scope of the disclosure. In certain embodiments, the server 104 may host the database 108. Alternatively, the server 104 may be separate from the database 108 and may be communicatively coupled to the database 108.

[0022] The database 108 may include suitable logic, circuitry, interfaces, and / or code configured to store information such as set of multimedia segments 110. Further, the database 108 may store instructions associated with operation of the electronic device 102. For example, the database 108 may store a mapping table including predefined instructions associated with different modalities of the set of multimedia segments 110 to control the operations of the electronic device 102. The database 108 may be derived from data of a relational or non-relational database or a set of comma-separated values (csv) files in conventional or big-data storage. The database 108 may be stored or cached on a device or server, such as the server 104. The device storing a database may be configured to query the database 108 for certain information (such as, the set of multimedia segments 110 or the mapping table) based on reception of a request for the particular information from the electronic device 102. In response, the device storing the database 108 may be configured to retrieve, from the database, results (for example, records related to the queried information) based on the received query.

[0023] In some embodiments, the database 108 may be hosted on the servers stored at the same or different locations. The operations of the database may be executed using hardware including a processor, a microprocessor (e.g., to perform or control performance of one or more operations), a field-programmable gate array (FPGA), or an application-Docket No. SYP354615WO02 specific integrated circuit (ASIC). In some other instances, the database 108 may be implemented using software.

[0024] The communication network 106 may include a communication medium through which the electronic device 102 and the server 104 may communicate with each other. The communication network 106 may be a wired or wireless communication network 106. Examples of the communication network 106 may include, but are not limited to, Internet, a cloud network, Cellular or Wireless Mobile Network (such as Long-Term Evolution and 5thGeneration (5G) New Radio (NR)), satellite communication system (using, for example, low earth orbit satellites), a Wireless Fidelity (Wi-Fi) network, a Personal Area Network (PAN), a Local Area Network (LAN), or a Metropolitan Area Network (MAN). Various devices in the network environment 100 may be configured to connect to the communication network 106, in accordance with various wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of a Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Zig Bee, EDGE, IEEE 802.11, light fidelity(Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device to device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0025] In operation, the electronic device 102 may be configured to receive multimedia content. The multimedia content may include the set of multimedia segments 110 associated with a plurality of modalities. The received set of multimedia segments 110 may be for example, images, video frames, Virtual Reality (VR) segments, and the like. The reception of the multimedia content including the set of multimedia segments 110Docket No. SYP354615WO02 associated with the plurality of modalities is described further, for example, in FIG.3, FIG. 4A and FIG.4B.

[0026] In an embodiment, the electronic device 102 may determine the set of features associated with the received multimedia content. The set of features may be for each modality of the plurality of modalities of the set of multimedia segments 110, based on an encoder model. The set of features may include visual features, audio features, text features, and the like. The visual features may include features of images and video frames, such as, lines, edges, color transitions, and the like. The audio features may include voice overs, background music, sound effects, audio transitions, and the like. The text features may include titles, captions, subtitles, text overlays, and the like. The modalities may be associated with the image frames, video data, moving pictures and the like. The encoder model may include visual encoders, audio encoders, and the like. The determination of the set of features is described further, for example, in FIG.3, FIG.4A, and FIG.4B.

[0027] In an embodiment, the electronic device 102 may apply the bottleneck attention model 112 on the determined set of features. The bottleneck attention model 112 may correspond to a prefix-tuned bottleneck attention model. The electronic device 102 may determine query attributes based on first features from the set of features. The first features may be associated with a first modality of the plurality of modalities. In an example, the first modality may be associated with the visual features. The electronic device 102 may determine prefix-tuned attributes based on second features from the set of features. The second features may be associated with a second modality of the plurality of modalities. In an example, the second modality may be associated with the audio features. The determined prefix-tuned attributes correspond to at least one of the first set of prefix-tuned keys may include first prefixes and keys of a key-value pair, and a secondDocket No. SYP354615WO02 set of prefix-tuned values that may include second prefixes and values of the key-value pair. The bottleneck attention model 112 may further correspond to a cross-modal multi- head attention model and a bottleneck adapter model. The bottleneck attention model 112 includes at least one of: a downscaling encoder model, a Gaussian Error Linear Unit (GELU) model, or an upscaling decoder model. The application of the bottleneck attention model is described further, for example, in FIG.3, FIG.4A and FIG.4B.

[0028] In an embodiment, the electronic device 102 may determine cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the encoder model (for example visual encoder, audio encoder and the like). The cross-modal fused features may be determined based on the application of the bottleneck attention model 112. The electronic device 102 may apply the cross-modal multi-head attention model on the determined query attributes and the determined prefix- tuned attributes to determine an attention score. The bottleneck adapter model may be applied on the determined attention score. The determination of the cross-modal fused features is described further, for example, in FIG.3, FIG.4A and FIG.4B.

[0029] In an embodiment, the electronic device 102 may segregate the determined cross-modal fused features into a set of nodes. The segregation of the cross-modal fused features may be based on the one or more events associated with the multimedia content. The electronic device 102 may apply a modulation model on the second features of the set of features and determine third features based on the application of the modulation model. Further, the electronic device 102 may determine fourth features based on the application of the bottleneck attention model 112 and determine fused features based on a fusion of the determined third features and the determined fourth features. The determination of the cross-modal fused features may be further based on the determined fused features and the first features. The segregation of the determined cross-modal fusedDocket No. SYP354615WO02 features may be based on a linear transformation and enhancement model corresponding to a Lorentz model. The segregation of the cross-modal fused features is described further, for example, in FIG.3, FIG.4A and FIG.4B.

[0030] In an embodiment, the electronic device 102 may apply the classifier model 114 on the segregated cross-modal fused features. The classifier model 114 may correspond to a hyperbolic classifier model. The electronic device 102 may convert the determined cross-modal fused features to a hyperbolic space. The segregation of the determined cross-modal fused features may be further based on the converted cross-modal fused features. A Hyperbolic Lorentzian Graph Attention (HL-GAtt) model may be applied on the converted cross-modal fused features. The application of the classifier model 114 may be further based on the application of the HL-GAtt model. The electronic device 102 may determine an enhanced feature vector based on the application of the HL-GAtt model. The converted cross-modal fused features may correspond to temporal and spatial features in the hyperbolic space, based on an enhanced temporal component. The application of the classifier model on the segregated cross-modal fused features is described further, for example, in FIG.3, FIG.4A and FIG.4B.

[0031] In an embodiment, the electronic device 102 may classify each multimedia segment of the set of multimedia segments into the one or more events, based on the application of the classifier model 114. The classification of the set of multimedia segments into the one or more events is described further, for example, in FIG.3, FIG.4A and FIG. 4B.

[0032] FIG.2 is a block diagram that illustrates an electronic device of FIG.1 for event- based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure. FIG.2 is explained in conjunction with elements from FIG.1. With reference to FIG.2, there is shown a block diagram 200Docket No. SYP354615WO02 of the electronic device 102. The electronic device 102 may include a circuitry 202, a memory 204, an input / output (I / O) device 206, and a network interface 208. In at least one embodiment, the I / O device 206 may also include a display device 206A. In at least one embodiment, the memory 204 may include set of multimedia segments 110, a set of modalities 204A, a set of features 204B. The electronic device 102 may further include the bottleneck attention model 112 and the classifier model 114. The circuitry 202 may be communicatively coupled to the memory 204, the I / O device 206, the network interface 208, the bottleneck attention model 112, and the classifier model 114, through wired or wireless communication of the electronic device 102.

[0033] The circuitry 202 may include suitable logic, circuitry, interfaces, and / or code that may be configured to execute program instructions associated with different operations to be executed by the electronic device 102. The operations may include reception of multimedia content and determination of a set of features associated with the received multimedia content. The operations may further include application of the bottleneck attention model 112 on the determined set of features and determination of cross-modal fused features for the plurality of modalities associated with the received multimedia content. The operations may further include segregation of the determined cross-modal fused features into the set of nodes based on the one or more events associated with the received multimedia content. The classifier model 114 may be applied on the segregated cross-modal fused features. The operation may further include classification of set of multimedia segments 110 into the one or more events based on the application of the classification of the classifier model 114.

[0034] The circuitry 202 may include one or more specialized processing units, which may be implemented as an integrated processor or a cluster of processors that perform the functions of the one or more specialized processing units, collectively. The circuitryDocket No. SYP354615WO02 202 may be implemented based on a number of processor technologies known in the art. Examples of implementations of the circuitry 202 may be an x86-based processor, a Graphics Processing Unit (GPU), a Reduced Instruction Set Computing (RISC) processor, an Application-Specific Integrated Circuit (ASIC) processor, a Complex Instruction Set Computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other computing circuits.

[0035] The memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store the program instructions to be executed by the circuitry 202. The program instructions stored on the memory 204 may enable the circuitry 202 to execute operations of the circuitry 202 (and / or the electronic device 102). In at least one embodiment, the memory 204 may store the multimedia content (for example, the set of multimedia segments 110). The electronic device 102 may determine the set of features (for example, visual features, audio features, and the like) associated with the received multimedia content for each modality. The set of features 204B may also be stored in the memory 204. The memory 204 may store the set of modalities 204A. Examples of implementation of the memory 204 may include, but are not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read- Only Memory (EEPROM), Hard Disk Drive (HDD), a Solid-State Drive (SSD), a CPU cache, and / or a Secure Digital (SD) card.

[0036] The I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive input and provide an output based on the received input. For example, the I / O device 206 may receive the multimedia content as the input. In an example, the I / O device 206 may output the classification of each multimedia segment of the set of multimedia segments 110 into the one or more events. Examples of the I / O device 206 may include, but are not limited to, a touch screen, a keyboard, a mouse, aDocket No. SYP354615WO02 joystick, a microphone, the display device 206A, and a speaker. Examples of the I / O device 206 may further include braille I / O devices, such as, braille keyboards and braille readers.

[0037] The I / O device 206 may include the display device 206A. The display device 206A may include suitable logic, circuitry, and interfaces that may be configured to receive inputs from the circuitry 202 to render on a display screen, for example multimedia content. The display device 206A may the render the classification of each multimedia segment of the set of multimedia segments 110 into the one or more events. In at least one embodiment, the display device 206A may be at least one of a resistive touch screen, a capacitive touch screen, or a thermal touch screen. The display device 206A may be realized through several known technologies such as, but not limited to, at least one of a Liquid Crystal Display (LCD) display, a Light Emitting Diode (LED) display, a plasma display, or an Organic LED (OLED) display technology, or other display devices.

[0038] The network interface 208 may include suitable logic, circuitry, and interfaces that may be configured to facilitate communication between the circuitry 202, the I / O device 206, and the memory 204, via the communication network 106. The network interface 208 may be implemented by using various known technologies to support wired or wireless communication of the electronic device 102 with the communication network 106. The network interface 208 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuitry.

[0039] The network interface 208 may be configured to communicate via wireless communication with networks, such as the Internet, an Intranet, or a wireless network, such as a cellular telephone network, a wireless local area network (LAN), a short-rangeDocket No. SYP354615WO02 communication network 106, and a metropolitan area network (MAN). The wireless communication may use one or more of a plurality of communication standards, protocols and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), Long Term Evolution (LTE), 5thGeneration (5G) New Radio (NR), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi- Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g or IEEE 802.11n), voice over Internet Protocol (VoIP), light fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), a near field communication protocol, and a wireless pear-to-pear protocol.

[0040] FIG.3 is a diagram that illustrates an exemplary processing pipeline of event- based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure. FIG.3 is explained in conjunction with elements from FIG. 1 and FIG. 2. With reference to FIG. 3, there is shown an exemplary execution pipeline 300 for event-based multimedia segment classification from cross-modal features of the multimedia content. The execution pipeline 300 may include operations 302 to 314 executed by a computing device, such as, the electronic device 102 of FIG.1 or the circuitry 202 of FIG.2.

[0041] At 302, multimedia content may be received. The circuitry 202 may be configured to receive multimedia content including the set of multimedia segments 110 (e.g., media segments 302A) associated with the plurality of modalities. The received set of multimedia segments 110 may be, for example, video frames, Virtual Reality (VR) segments, and the like.

[0042] At 304, the set of features associated with the received multimedia content may be determined. The circuitry 202 may be configured to determine, for each modality of theDocket No. SYP354615WO02 plurality of modalities, the set of features associated with the received multimedia content, based on the encoder model (e.g., an encoder model 304A). The set of features may include visual features, audio features 416, text features, and the like. The visual features may include features of images and video frames, such as, lines, edges, color transitions, and the like. The audio features may include voice overs, background music, sound effects, audio transitions, and the like. The text features may include titles, captions, subtitles, text overlays, and the like. The modalities may be associated with the image frames, video data, moving pictures and the like. The encoder model 304A may include visual encoders, audio encoders, and the like.

[0043] At 306, bottleneck attention model 112 may be applied. The circuitry 202 may be configured to apply the bottleneck attention model 112 on the determined set of features. The bottleneck attention model 112 includes various components such as a prefix-tuner, a cross-modal multi head attention model, a normal layer, a fully connected (FC) layer, an FC down-sampling layer, a Gaussian Error Linear Unit (GELU) model, and an FC up- sampling layer. The various components of the bottleneck attention model are described further, for example, in FIG.4A. The bottleneck attention model 112 may correspond to the prefix-tuned bottleneck attention model. Further, the bottleneck attention model 112 may correspond to the cross-modal multi-head attention model and the bottleneck adapter model. The bottleneck adapter model may include at least one of downscaling encoder model, the GELU model, or an upscaling decoder model.

[0044] The fusion model, for example, cross-modal fusion model, may be used to address the challenges of imbalanced modality information. The cross-modal fusion model may dynamically adjust the influence of each modality that may prioritize an importance of the audio features relative to the visual modality. The selective process may ensure that only relevant audio features that are crucial for visual learning are being utilized. Based onDocket No. SYP354615WO02 selection of the most appropriate features relative to the visual modality, the electronic device 102 of the disclosure may enhance visual feature learning by use of relevant audio features. Further, a hyperbolic graph convolution network based HLGAtt method may be introduced to maintain consistent discrimination between normal and abnormal features. The HLGAtt method may operate in a hyperbolic space to capture hierarchical relationships between normal and abnormal representations through spatial and temporal feature learning, which may aid to distinguish normal and abnormal features. The bottleneck attention model 112 may be used to address the imbalance issue in audio- visual modality information and effectively distinguish abnormal features from normal features so that anomaly events such as violence and nudity may be detected accurately. To address the imbalanced modality information issue, a fusion module called Cross- modal Fusion Adapter (CFA) may be used, which helps the proposed framework to facilitate multi-modal interaction effectively by dynamic regulation of the contribution of each modality. The attention model for example, the HLGAtt method, may be used to capture the hierarchical relationships between normal and abnormal features, that may enhance the feature separation.

[0045] At 308, cross-modal fused features may be determined for the plurality of modalities associated with the received multimedia content. The circuitry 202 may be configured to determine the cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model 112. The bottleneck attention model 112 may correspond to bottleneck adapter model. The bottleneck adapter model may include a prefix-tuned based bottleneck attention and a modulation model 420. The prefix-tuned bottleneck attention may help in efficient multi-modal interaction between the plurality of modalities (for example, audio, visual and the like). A modulation model may be used to dynamicallyDocket No. SYP354615WO02 regulate the contribution of each modality during the fusion process, to account for the importance of the audio feature with respect to the visual modality. The prefix-tuned bottleneck attention model may incorporate prior knowledge into the feature transformation process based on a combination of the learned representations with initialized parameters (query attributes and prefix-tuned attributes) through the prefix-tuning operation. The query attributes may include visual features. The prefix-tuned attributes may correspond to a first set of prefix-tuned keys including first prefixes and keys of a key value pair or a second set of prefix-tuned values including second prefixes and values of the key-value pair. The prefix-tuned bottleneck attention model may concatenate the keys (K), and values (V) obtained from the audio features FA with prefixes Pk and Pv, that may result in prefix-tuned keys Kpand values Vp, respectively. The prefix-tuned keys Kpand prefix-tuned values Vpmay be initialized as zero matrices with dimensions of ℝ^^^^^^^^^^^^^^^^^^^^^^^^^^^^, where B, DA& DPrepresent the batch size, audio feature dimension, andrespectively. These prefix-tuned keys Kp and values Vp along with the query Q, i.e., visual features FV, may be then passed on to the cross-modal multi-head attention model to determine the cross-modal fused features. The determination of the cross-modal fused features is described further, for example, in FIG.4A and FIG.4B.

[0046] At 310, the determined cross-modal fused features may be segregated. The circuitry 202 may be configured to segregate the determined cross-modal fused features into the set of nodes, based on the one or more events associated with the received multimedia content. The segregation of the determined cross-modal fused features may be based on a linear transformation and enhancement model corresponding to a Lorentz model. The determined cross-modal features may be converted to hyperbolic space features. The segregation of the determined cross-modal fused features may be furtherDocket No. SYP354615WO02 based on the converted cross-modal fused features. The segregation of the determined cross-modal fused features is described further, for example, in FIG.4A and FIG.4B.

[0047] At 312, the classifier model 114 may be applied on the segregated cross-modal fused features. The classifier model 114 may correspond to a hyperbolic classifier model. In an embodiment, the determined cross-modal fused features may be converted to the hyperbolic space features. An enhanced feature vector may be determined based on the application of the HL-GAtt model. The HL-GAtt may be applied on the converted cross- modal fused features. The enhanced features may be passed through a Leaky Rectified Linear Unit (ReLU) activation and Softmax normalization operations to introduce non- linearity and ensure standardization across the enhanced feature maps. This may ensure that distinct patterns, representative of normal and abnormal data. The enhanced feature maps may be processed via matrix multiplication to compute attention, followed by the ReLU activation to generate the output feature maps. This can be formulated as: ^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^ (^^�^^^^^^^^^^^^^^^^^^^^^^^^ .^^�^ ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^ ^^ ^^^^^ ) (1)where ^^^^^^^^^^^^^^^^^^^^^^^^^^^^may^^^^^^^^^^^^^^^^^^^^may represent a ReLU activation function; and^^�^^^^^^^^^^^^^^^^^^^^^^^^^^and ^^�^^^^^^^^^^^^^^^^^^^^^^^^^^may represent hyperbolic space features associated with the node “A”and the node “B”, respectively

[0048] The classifier model 114 may be utilized to take the HLGAtt method as input and predict confidence scores for normal and abnormal events. The final score can be represented as: ^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^(^^^^^^^^) (2) where ^^^^^^^^^^^^^^^^may

[0049] The Multiple Instance Learning (MIL)-based learning objective may be adopted to calculate the mean value of the top k-max predictive scores within the set of multimediaDocket No. SYP354615WO02 segments. The high-scoring positive predictions may indicate the presence of abnormal events, while the k-max negative scores usually represent hard samples. This learning objective function can be formulated as: ^^^^^^^^^^^^^^^^= 1 ^^^^∑^^^^^^^^=^^^^ −^^^^^^^^ . ^^^^^^^^^^^^(�^^^�^^�^^�^^^�^^�^^�^^^�^^^) (3)where�^^^�^^�^^�^^^�^^�^^�^^^�^^^ may indicate an average of the k-max scores in the set of multimediasegments 110, and ^^^^^^^^represents the binary media segment-level label. The application of the classifier model is described further, for example, in FIG.4A and FIG.4B.

[0050] At 314, the set of multimedia segments may be classified into one or more events. The circuitry 202 may be configured to classify each multimedia segment of the set of multimedia segments 110 into the one or more events, based on the application of the classier model. For example, the set of multimedia segments 110 may be classified into classified media segments 316. The classification of the set of multimedia segments into the one or more events is described further, for example, in FIG.4A and FIG.4B.

[0051] The detection of anomalies (for example, violence, nudity, and the like) in a video is a widely recognized computer vision challenge that holds potential for various practical applications, including surveillance systems, crime prevention, and content moderation. However, the determination of annotations for anomalies at the frame level in the videos can be both expensive and time-consuming. Recently, techniques such as, Weakly Supervised Video Anomaly Detection (WS-VAD) model, are used to detect anomaly events such as, violence and nudity, in videos using only video-level labels. For example, in a scenario of surveillance videos and associated labelled dataset, a WS-VAD label of the video may be either “normal” or “anomalous”. The WS-VAD model may learn to identify abnormal events, such as, a person with a weapon or a person involved in violent behavior, based on the video-level labels. This approach may reduce the annotation effort and makes it feasible to detect anomalies in large scale video datasets. However, the videoDocket No. SYP354615WO02 anomaly detection task may have significant challenges, such as, issues related to imbalanced modality information and consistent differentiation between normal and abnormal features.

[0052] The disclosed electronic device 102 may receive multimedia content (for example, videos, images, moving pictures, and the like) including a set of multimedia segments associated with plurality of modalities (for example, audio, video, and the like). The electronic device 102 may determine a set of features associated with the received multimedia content for each modality of the plurality of modalities (for example, audio, video, and the like), based on an encoder model (for example, audio encoder, visual encoder, and the like). Further, the electronic device 102 may apply a bottleneck attention model on the determined set of features and determine cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model. The determined cross-modal fused features may be segregated into a set of nodes (for example, node “A”, node “B” and the like) based on one or more events (for example, a concert, surveillance video, and the like) associated with the received multimedia content and apply a classifier model on the segregated cross- modal fused features. The electronic device 102 may classify each multimedia segment into one or more events, based on the application of the classifier model. The bottleneck attention model may facilitate effective multi-modal interaction based on dynamic regulation of a contribution of each modality. Also, the bottleneck attention model may incorporate a prefix-tuned bottleneck attention adapter and a modulation model. The bottleneck attention model including the bottleneck attention adapter may help to adapt the set of features of each modality to a shared bottleneck representation, that may ensure seamless interaction between modalities and may also preserve modality-specific characteristics.Docket No. SYP354615WO02

[0053] Thus, the electronic device 102 may detect anomaly events such as violence and nudity accurately. The electronic device 102 may address imbalanced modality information issues and effectively facilitate multi-modal interaction by dynamic selection of relevant features (for example, audio features) with corresponding visual features. Additionally, the electronic device 102 may determine hierarchical relationships within normal and abnormal representations, which may improve the accuracy of differentiation between the normal and abnormal features. In an example of a security system installed in a public space, such as an airport, the electronic device 102 may detect anomaly events like violence and nudity by processing the multimedia content received from the security system. For instance, if someone gets involved in a physical altercation or attempts to expose themselves, the electronic device 102 may promptly identify and alert the security personnel. Moreover, the electronic device 102 may address imbalanced modality information issues. For example, an audio input may be crucial for detection of a potential threat, such as, threatful remarks from a person. The electronic device 102 may select the relevant audio features and combine them with corresponding visual features to, for example, analyze body language or suspicious movements. Such a comprehensive approach may ensure that no vital information is overlooked, which may enhance the overall effectiveness of the security system. Additionally, the electronic device 102 may determine the hierarchical relationships within the normal and abnormal representations. For example, the electronic device 102s may learn and recognize patterns of normal behavior, such as, people walking or talking, and distinguish them from abnormal behavior, such as, someone running or acting erratically. Based on the determination of the hierarchical structure of such representations, an accuracy of the electronic device 102 may increase. For example, the electronic device 102 may be able to more accuratelyDocket No. SYP354615WO02 separate the normal and abnormal features, reduce false alarms, and improve overall security measures of the security system.

[0054] FIG.4A and FIG.4B collectively illustrate an exemplary architecture of a model for event-based multimedia segment classification from cross-modal features of multimedia content, in accordance with an embodiment of the disclosure. FIG.4A and FIG. 4B are explained in conjunction with elements from FIG. 1, FIG. 2, and FIG. 3. With reference to FIG.4A and FIG.4B, there is shown the exemplary architecture of a model 400 for event-based multimedia segment classification from cross-modal features of multimedia content.

[0055] The model 400 may include a visual encoder 404, an FC layer 406, visual features 410, an audio encoder 414, audio features 416, a prefix-tuned bottleneck attention 418, a modulation model 420, a cross-modal fusion adapter 408, cross-modal aware features 422, a hyperbolic space conversion 424, a Lorentz linear transformation and enhancement model 426, a Leaky Relu 428, a Softmax 430, a hyperbolic classifier 432, and an output 434 (for example, anomaly or normal). The video segments 402 and audio segments 412 may be provided as input to the bottleneck attention model 112. The video segment 402 may include images data, video data, moving pictures, and the like. The audio segments 412 may include voiceover narration, sound effects, music tracks, and the like. The video frames may include a set of “N” videos and corresponding ground- truth video-level labels, as per expressions (1) and (2), as follows: ^^^^ = {^^^^ ^^^^^^^^}^^^^=1 (4)^^^^ = {^^^ ^^^^^^^^^}^^^^=1 ^ {1,0} (5)where ^^^^ may represent the set of “N” videos; and ^^^^ may represent ground-truth video-level labels, wherein ^^^^^^^^=1 may indicate an abnormal event in the set of multimedia segments 110, while ^^^^^^^^=0 may indicate an absence of theDocket No. SYP354615WO02 abnormal event. The abnormal event or anomalous event may refer to an event or behavior that deviates significantly from the expected or normal patterns in the multimedia content. Anomalous events may be typically rare occurrences that differ from the normal activities or behaviors captured in the multimedia content.

[0056] The abnormal events may include, for example, violence and nudity, within the videos determined based on a weakly supervised manner. Each video feature may be represented as ^^^^^^^^and initially divided into 16-frame based T non-overlapping multi-modalsegments (^^^^ = {^^^^^^^^ ^^^^^^^^ ,^^^^^^^^ }^^^^^^^^=1 ), which may be processed by a pre-trained CNN network toextract thefeatures ^^^^ ^^^^^^^^^^^^^^^^ ∈ ℝ ^^^^ 410 and audio features 416 ^^^^^^^^ ∈ℝ^^^^^^^^^^^^^^^^, where ^^^^^^^^and ^^^^^^^^may represent the feature dimensions of video and audio respectively. The embodiments of the present disclosure may not be limited to only visual and audio features. Here, ^^^^^^^^^^^^and ^^^^^^^^^^^^denote the video features 410 and the audio features 416, respectively,forwarded to the bottleneck attention model 112.

[0057] The visual encoder 404 may extract the visual features 410 from images or video frames received as input (for example, a video segment 402). The visual encoder 404 may analyze the video segment 402 and encode it into a compressed format, that may reduce the amount of data required to represent the image with a preservation of important visual details. The visual encoder 404 may employ techniques, such as, color space conversion, spatial and temporal compression, and quantization to achieve efficient video encoding. The encoded data may then be stored, transmitted, or further processed. The audio encoder 414 may extract the audio features 416 from audio input (for example, an audio segment 412). The audio encoder 414 may employ techniques, such as, sampling, quantization, and compression to achieve efficient audio encoding.Docket No. SYP354615WO02

[0058] The FC layer 406 may be used as a last layer in a neural network architecture, which may be fed with an output from a previous layer to produce a final output or prediction. Each neuron in the fully connected layer may receive inputs from all the neurons in the previous layer and apply a set of weights and biases to those inputs. The set of weights and biases may be learned during a training phase of the neural network. The training phase may allow the neural network to adapt and make accurate predictions. The FC layer 406 may perform a linear transformation on its input, followed by an application of an activation function. The activation function may introduce a non-linearity into the neural network, that may enable the neural network to learn complex patterns and relationships in the data.

[0059] The set of multimedia segments 110 may be passed through the cross-modal fusion adapter 408. The cross-modal fusion adapter (CFA) 408 may include the prefix- tuned bottleneck attention 418. The prefix-tuned bottleneck attention 418 may include a cross-modal multi-head attention model 436, a normalization (norm) layer 438, the FC layer 406, an FC down layer 440, Gaussian Error Linear Unit (GELU) 442, an FC up layer 444. The CFA model 408 may include a prefix-tuned bottleneck attention 418 and modulation model 420. The prefix-tuned bottleneck attention 418 helps in efficient multi- modal interaction between the audio and visual modalities. The modulation model 420 may dynamically regulate the contribution of each modality during the fusion process, based on the importance of the audio features 416 with respect to the visual features 410. The prefix-tuning bottleneck attention model 418 may incorporate a prior knowledge into a feature transformation process based on a combination of the learned representation with initialized parameters through the prefix-tuning operation.

[0060] The feature transformation process may include a concatenation of the first features (keys K) associated with the first modality (for example, the visual features ^^^^^^^^410)Docket No. SYP354615WO02 and the second features (values V) obtained from the second modality (for example, the audio features ^^^^^^^^416) with prefixes ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^, that may result in prefix-tuned keys ^^^^^^^^and values ^^^^^^^^, respectively. The parameters ^^^^^^^^and ^^^^^^^^may be initialized as zero matrices with dimensions of ℝ^^^^^^^^^^^^^^^^^^^^^^^^^^^^, where B, ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^may represent a batch size, audio feature dimensions,respectively. The prefix-tunes keys ^^^^^^^^and values ^^^^^^^^along with query attributes Q based on the first features, that is visual features ^^^^^^^^410, may be then passed on to the cross-modal multi-head attention model 436. This cross-modal multi-head attention model 436 may enable an interaction between the prefix-tuned features of the audio and visual modalities, such that each modality’s relevant information selectively and contextually preserved. In this process, attention scores (for example, ^^^^^^^^^^^^^^^^) may be determined based on the query attributes, the prefix-tuned keys and the prefix- tuned values. The mathematical formulation of the cross-modal multi-head attention model 436 function (for example, ^^^^^^^^^^^^^^^^) can be formulated as: ^^^^ ^^^^ ^^^^.^^^^^^^^^^^^^^^^ where ^^^^ may

[0061] The attention scores ^^^^^^^^^^^^^^^^may be subsequently passed to the bottleneck adapter model. The bottleneck adapter model may ensure a smooth interaction between modalities based on a preservation of modality-specific characteristics. It comprises down-scaled FC layers (for example, the FC down layer, ^^^^^^^^^^^^^^^^^^^^440) followed by the Gaussian Error Linear Unit (GELU) 442 activation (that is, ^^^^^^^^^^^^^^^^^^^^) and the up-scaled FC layers (that is, ^^^^^^^^^^^^444). The output of the bottleneck adaptor model can be formulated as: ^^�^^^^^^^^^^^^^^ = ^^^^^^^^^^^^(^^^^^^^^^^^^^^^^^^^^(^^^^^^^^^^^^^^^^^^^^(^^^^^^^^^^^^^^^^))) (7)Docket No. SYP354615WO02

[0062] Here, the GELU 442 activation function may introduce non-linearity, that may allow intricate feature transformations and ensure that the adapter model effectively adjusts input features to the shared bottleneck representation.

[0063] The CFA model 408 includes the modulation model 420. The CFA model 408 may include modulation factors that dynamically adjust an impact of individual modalities based on an importance of the audio features 416 relative to the visual modality. The modulation model 420 may be a mechanism that may be facilitated by a learnable modulation function that operates on the audio features ^^^^^^^^416 to select relevant audio features 416 that are important to visual modalities. The modulated features ^^^^^^^^^^^^^^^^may be defined as: ^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^(^^^^^^^^) = ^^^^ (^^^^^^^^^^^^^^^^.^^^^^^^^) (8)where σ may represent the sigmoid activation, while Wmodmay represent the weights associated with the modulation function. The sigmoid activation function may ensure that modulation factors range between 0 and 1, which may regulate the degree of modulation applied to the fused representation.

[0064] Further, a fusion and refinement process may be used, wherein the modulated features may be fused with an output of the prefix-tuned bottleneck attention 418. The fused representation may then be refined through the FC layer 406. This operation can be expressed mathematically as: ^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^ + (^^^^^^^^ +�^^�^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^�) (9)

[0065] The modulation model 420 (e.g., fMF()) may modify the output of the prefix-tuned attributes based on a significance of the audio features 416 with respect to the visual modality. The fusion and refinement process may be performed based on the bottleneck adapter model. The final fused representation (for example, cross-modal fused features (such as, the cross-modal aware features 422)) may capture the most relevant informationDocket No. SYP354615WO02 from both modalities (for example, the visual features 410 and the audio features 416), that may simultaneously reduce noise and preserve the modality-specific characteristics. The CFA model 408 may learn multi-modal interactions effectively. The CFA model 408 may determine the cross-modal aware features 422, which may be converted into a hyperbolic space by an operation such as, the hyperbolic space conversion 424.

[0066] A hyperbolic graph convolution network may be based on a new attention mechanism for example, HLGAtt. The HLGAtt may use a hyperbolic Lorentz graph attention mechanism that learns layer wise curvature parameters to capture a hierarchical structure of an input graph, and thereby enhance the hierarchical relationship between normal and abnormal representations compared to existing graph-based or transformer- based approaches. The hyperbolic space conversion 424 may be executed to convert the cross-modal aware features 422 into the hyperbolic space and then the Lorentz linear transformation and enhancement model 426 may be executed on the converted features on parallel nodes.

[0067] Initially, the cross-modal aware features 422 may be fused into the hyperbolic space using an exponential function. As a result, the converted fused features map maybe obtained for example, ^^^^ ^^^^^^^^^ ∈ ℝ ^^^2^^^^^^^^, wherein T may denote a number of segments and^^^^^^^^may represent the hyperbolic dimension. The hyperbolic graph attention mechanism may utilize a parallel branch process to learn different features and patterns in respective branches for prediction tasks. The converted hyperbolic feature maps on two parallel branches, for example, a node A and a node B, to learn specific patterns from the input feature maps. The separation of the branches (for example, the node A and the node B) may ensure that features with similar characteristics are directed to the respective nodes (for example, the node A and the node B). The branches may include the Lorentz Linear Transformation and enhancement model 426. This may allow each branch to learn uniqueDocket No. SYP354615WO02 properties of normal and abnormal features in the set of multimedia segments 110, that may enable more precise discrimination between them.

[0068] The converted hyperbolic feature maps may be passed through the Lorentzian linear transformation and enhancement model 426 in each node. Here, the Lorentzian linear transformation for feature transformation and its transformed temporal and spatial features may be further enhanced using an enhancement method, based on the enhanced temporal component 450 (for example, an enhanced feature vector). An adjacency matrix^^^^ ∈ ℝ^^^^^^^^^^^^ may be established to capture hyperbolic feature similarities in Lorentzian linearHere, each entry Aij can be calculated as: ^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^�^^^^^^^^,^^^^ ,^^^^^^^^,^^^^� = ^^^^^^^^^^^^^^^^^^^^^^^^^^^^(exp�−^^^^^^^^�^^^^^^^^,^^^^ ,^^^^^^^^,^^^^ (10)where fsimevaluate a resemblance of snippets i and j based on their Lorentzian intrinsic distance dL. The exponential and Softmax functions 430 may be employed to maintain non-negativity and restrict the values of A within a range of [0,1]. The hyperbolic Lorentz linear (that is, fHL()) may be followed by neighborhood hyperbolic aggregation operation for feature transformation. The transformed hyperbolic features of the ithsnippet at the layer l (that is, ^^^^^^^^^^^^) can be expressed as: ^^^^^^^^ = ^^^^^^^^ ∑^^^^^^^^=1 ^^^^^^^^,^^^^ ^^^^^^^^^^^^(^^^^^^^^^^^−^,^^^1^)^^^^^(11) where, ^^^^

[0069] To enhance the transformed features z further, the transformed features may be processed based on temporal and spatial information, using a temporal component 446 and a temporal enhancement 448. The initial components of the input vector z[0] may signify the temporal aspect within hyperbolic space. This component may be processed via a sigmoid activation function followed by exponential scaling operation 452 and shiftingDocket No. SYP354615WO02 operations. Through this procedure, temporal features, based on temporal components 446 (i.e., TnodeA and TnodeB) may be computed for both node A and node B as: ^^^^ = ^ ( ^^^^^[ ]) ^^^^^^^^^^^^^^^^^^^^^^^^ ^^^ ^^^^^^^^^^^^^^^^^^^ 0 × ^^^^ + 1.1 (12A)^^^^ ^^^^ = ^^^^(^^^^^^^^^^^^^^^^^^^^^^^^[0]) × ^^^^^^^^^^^^^^^^^^^^ ^^^^ + 1.1 (12B)where ^^^^ may be a trainable parameter. The remaining elements of input vector z can be considered as the spatial features for the node A and the node B, which may be given by: ^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^[1],^^^^^^^^^^^^^^^^^^^^^^^^[2], … ,^^^^^^^^^^^^^^^^^^^^^^^^[^^^^]] (13A)^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^[1],^^^^^^^^^^^^^^^^^^^^^^^^[2], … ,^^^^^^^^^^^^^^^^^^^^^^^^[^^^^]] (13B)

[0070] These features may encapsulate the intricate spatial features in hyperbolic space, which may be critical to capture the hierarchical structure and relationships within the graph. To ensure the alignment of spatial components with the hyperbolic model, a scaling operation 452, referred to as ^^^^ may be computed. A scaling factor in the scaling operation 452 may take into account the temporal and spatial complexities of each node. It may ensure that the spatial components may be appropriately scaled to fit within the hyperbolic space. The scaled features for the node A and the node B may be given by: ^^^^^^^^^^^^^^^^^^^^^^^^2−1^^^^^^^^^^^^^^^^^^^^=^^^^∑^^^^ ^^^^ 2(14A) (14B)

[0071] The temporalspatial components 456 may be concatenated, to result in enhanced feature vectors (that is, ^^�^^^^^^^^^^^^^^^^^^^^^^^^^^). Mathematically, the concatenation process can be expressed as: ^^�^^^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^[^^^^^^^^^^^^^^^^^^^^^^^^, ^^^^^^^^^^^^^^^^^^^^^^^^ �^^^^^^^^^^^^^^^^^^^^^^^^] (15A)^^�^^^^^^^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^[^^^^^^^^^^^^^^^^^^^^^^^^, ^^^^^^^^^^^^^^^^^^^^^^^^ �^^^^^^^^^^^^^^^^^^^^^^^^] (15B)

[0072] Thethrough Leaky ReLU 428 activation and Softmax 430 normalization operations to introduce non-linearityDocket No. SYP354615WO02 and ensure standardization across the enhanced feature maps. This ensures that distinct patterns, that may represent normal and abnormal data, may be learned at each node. By doing so, different sets of features may be learned from those processed by the other node (i.e., the node B). Finally, the enhanced feature maps from node A and node B are processed via matrix multiplication to compute attention, followed by a ReLU activation to generate the output feature maps. This outcome of the proposed HLGAtt model may be formulated as: ^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^ ^^^^^^ = ^^^^^^^^^^^^^^^^^^^^(^^�^^^^^^ .^^�^^^^^^^^^^^^^^^^^^^^^^^^^ ) (16)

[0073] Theinclude the hyperbolic classifier 432 that may take the output of the HLGAtt model as input and predict the confidence scores for normal and abnormal events. The final score can be represented as: ^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^−^^^^^^^^^^^^(^^^^^^^^) (17)

[0074] To train thelearning (MIL)- based learning objective may be employed. The mean value of top k-max predictive scores within a multimedia segment may be calculated. High-scoring positive predictions may indicate the presence of abnormal events, while the k-max negative scores may represent normal samples. This learning objective function can be formulated as: ^^^^^^^^^^^^^^^^= 1 ^^^^∑^^^^^^^^=1 −^^^^^^^^ . log (�^^�^^^�^^�^^^�^^�^^�^^^�^^^) (18)where,�^^^�^^�^^�^^^�^^�^^�^^^�^^^ mayof the k-max scores in the multimedia content,and Yi may represent the binary video-level label.

[0075] It should be noted that the architecture 400 of FIG. 4A and FIG. 4B is for exemplary purpose and should not be construed to limit the scope of the disclosure.

[0076] FIG.5 is an exemplary graph illustrating an exemplary scenario for classification of event-based set of multimedia segments based on cross-modal fusion adaptor, inDocket No. SYP354615WO02 accordance with an embodiment of the disclosure. FIG.5 is explained in conjunction with elements from FIG.1, FIG.2, FIG.3, and FIG.4A and FIG.4B. With reference to FIG.5, there is shown an exemplary scenario 500.

[0077] The scenario 500 may include multiple images with various events. The multiple images may include, for example, image depicting explosion 502, image depicting a person exercising 504, image depicting soldiers fighting war 506, image depicting social conflict 508, and image depicting people having conversation 510. The scenario 500 may further include a graph 512 that illustrates exemplary prediction scores associated with the classification of the images based on the normal events and the abnormal events. The normal events may include, for example, the image depicting a person exercising 504, the image depicting people having conversation 510, and the like. The abnormal events may include, for example, the image depicting explosion 502, the image depicting soldiers fighting war 506, and the image depicting social conflict 508. The abnormal events may be indicated as high scores. The normal events may be indicated as low scores. The high score may indicate that the images include abnormal events, and the low score may indicate the normal events. The graph 512 may be a time to score graph. The scenario 500 may capture the time of abnormal event with respect to the score assigned to the set of multimedia segments 110. The objective of the proposed disclosure is to develop a robust video processing system to detect moderated content, such as violence and nudity, in the set of multimedia segments 110. The electronic device 102 may include a multi- modal approach that may integrate audio and visual cues. By leveraging both audio and visual information, the goal is to enhance the accuracy and reliability of violence and nudity detection to achieve AI-based content moderation, as it may be essential to comply with stream / broadcast entertainment content norms.Docket No. SYP354615WO02

[0078] It should be noted that the scenario 500 of FIG.5 is for exemplary purpose and should not be construed to limit the scope of the disclosure.

[0079] FIG.6 is a flowchart that illustrates operations of an exemplary method for event- based multimedia segment classification based on cross-modal fusion adaptor, in accordance with an embodiment of the disclosure. FIG.6 is described in conjunction with elements from FIG.1, FIG, 2, FIG.3, FIG.4A, FIG.4B, and FIG.5. With reference to FIG. 6, there is shown a flowchart 600. The exemplary method of the flowchart 600 may be executed by any computing system, for example, by the electronic device 102 of FIG.1. The exemplary method of the flowchart 600 may start at 602 and proceed to 604.

[0080] At 604, the multimedia content including the set of multimedia segments 110 associated with the plurality of modalities may be received. The circuitry 202 may be configured to receive the multimedia content including the set of multimedia segments 110 associated with the plurality of modalities. The multimedia content may include for example video frames, Virtual Reality (VR) segments, and the like. The video frames may include the set of videos (N) and corresponding ground-truth video-level labels. The abnormal event or anomalous event refers to the event or behavior that deviates significantly from the expected or normal patterns in the video segments 402. Anomalous events may typically be rare occurrences that differ from the regular activities or behaviors captured in the multimedia content. The reception of the multimedia content is described further, for example, in FIG.3, FIG.4A, and FIG.4B.

[0081] At 606, the set of features associated with the received multimedia content may be determined for each modality of the plurality of modalities, based on the encoder model. The circuitry 202 may be configured to determine, for each modality of the plurality of modalities, the set of features associated with the received multimedia content, based on the encoder model. The set of features may be, for example, the first features and theDocket No. SYP354615WO02 second features. The first features may be associated with the first modality. The query attributes may be determined based on the first features. The second features may be associated with the second modality. The prefix-tuned attributes may be determined based on the second features of the second modality. The prefix-tuned attributes may correspond to the first set of prefix-tuned keys including the first prefixes and keys of the key-value pair or the second set of prefix-tuned values including the second prefixes and values of the key-value pair.

[0082] The prefix-tuned bottleneck attention mechanism may integrate existing knowledge into the process of transforming features. This may be done based on a merger of the representations that have been learned with initial parameters, which include both query and prefix-tuned attributes, through the operation prefix-tuning. The query attributes may encompass the visual features 410, while the prefix-tuned attributes may be linked to either a set of prefix-tuned keys comprising initial prefixes and the keys from the key-value pair or the set of prefix-tuned values that include the secondary prefixes and the values from the key-value pair. This mechanism may concatenate the keys (K), and values (V) derived from the audio features FA416 with the prefixes Pkand Pv, creating the prefix- tuned keys Kp and values Vp. Initially, these prefix-tuned keys Kp and values Vp may be set up as zero matrices with the dimensions (ℝ^^^^^^^^^^^^^^^^^^^^^^^^^^^^), where B, DA, and Dp may denote the batch size, the dimension of the audioand the dimension of the prefix, respectively. These initialized keys and values, along with the query Q, that is, the visual features FV410, may be then fed into the cross-modal multi-head attention model. This model ascertains the features that result from the fusion of different modalities. The determination of the set of features is described further, for example, in FIG.3, FIG.4A, and FIG.4B.Docket No. SYP354615WO02

[0083] At 608, the bottleneck attention model 112 may be applied on the determined set of features. The circuitry 202 may be configured to apply the bottleneck attention model 112 on the determined set of features. The features may correspond to different modalities of the set of multimedia segments 110. The bottleneck attention model 112 may include various models such as an encoding model, a prefix-tuned bottleneck attention model, a modulation model, a cross-modal fusion adapter model (that may output the cross-modal aware features 422), a model for the hyperbolic space conversion 424, a model for the Lorentz linear transformation and enhancement model 426, a model for the leaky Relu 428, a model for the Softmax 430, a model for the hyperbolic classifier 432, and the like. The encoding model may include a visual encoding model and an audio encoding model. The visual encoding model may extract the visual features 410 and the audio encoding model may extract the audio features 416. The prefix-tuned bottleneck attention model may include a cross-modal multi-head attention. The prefix-tuned keys and values along with the query may be passed on to the cross-modal multi-head attention model. The cross-modal multi-head attention model may enable the interaction between the prefix- tuned features of the audio and visual modalities, that may allow selective and contextual focus on each modality relevant information. The attention score may be determined based on the queries, prefixed tuned keys and values. The modulation model 420 may dynamically regulate the contribution of each modality during the fusion process, based on determination of importance of the audio features 416 with respect to the visual modality. The CFA model 408 may determine the cross-modal aware features 422, which may be converted into a hyperbolic space by an operation such as, the hyperbolic space conversion 424. The hyperbolic Lorentzian graph attention (HL-GAtt) model may be applied on the features converted into the hyperbolic space, to capture hierarchical visual and audio representations. The determined cross-modal aware features 422 may beDocket No. SYP354615WO02 segregated into the set of nodes. The enhanced feature vector may be determined based on the HL-GAtt model. The hyperbolic classifier 432 may be applied on the segregated cross-modal fused features. The application of the bottleneck attention model is described further, for example, in FIG.3, FIG.4A, and FIG.4B.

[0084] At 610, the cross-modal fused features (such as, the cross-modal aware features 422) may be determined for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model 112. The circuitry 202 may be configured to determine cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model 112. The determination of the cross-modal fused features (such as, the cross-modal aware features 422) may be further based on the application of the bottleneck adapter model. The attention score may be determined based on the application of the cross-modal multi-head attention model. The bottleneck adapter model may be applied on the determined attention score. Through the fusion and refinement process, the final fused representation (for example, cross-modal aware features 422) may be generated to capture the most relevant information from both modalities, simultaneously based on a reduced noise and preservation of the modality-specific characteristics. The determination of the cross-modal fused features is described further, for example, in FIG.3, FIG.4A, and FIG.4B.

[0085] At 612, the determined cross-modal fused features (such as, the cross-modal aware features 422) may be segregated into the set of nodes based on the one or more events associated with the received multimedia content. The circuitry 202 may be configured to segregate the determined cross-modal fused features into the set of nodes. The set of nodes may include Lorentz Linear Transformation and Enhancement model 426. Each node of the set of nodes may correspond to various modalities. For example,Docket No. SYP354615WO02 the various modalities may correspond to the at least one of image frames, video data, or moving pictures. One of the nodes may include the Leaky Relu 428 and the Softmax function 430. The Leaky ReLU 428 and the Softmax 430 may be activation functions that are used to introduce non-linearity into a neural network and make it capable of learning complex patterns and relationships. The Leaky ReLU 428 may allow a small, non-zero gradient for negative input values, which helps to prevent a vanishing gradient problem. Mathematically, the Leaky ReLU 428 function is defined as: F(x) = max (ax, x) (19) where ‘x’ may represent the input to a neuron, and ‘a’ may be a small positive constant that determines a slope of the function for negative input values.

[0086] The Softmax 430 may be an activation function that is commonly used in the output layer of the neural network for multi-class classification problems. It converts the raw output values of the network into a probability distribution over the classes. The Softmax function 430 takes the vector of real numbers as input and outputs the vector of values between 0 and 1, where the sum of all the values is equal to 1. Mathematically, the Softmax function 430 is defined as: f(x_i) = exp(x_i) / sum(exp(x_j)) (20) where ‘x_i’ may represent the input value for the i-th class, and ‘exp’ denotes the exponential function. The Softmax function 430 may amplify differences between the input values, that may make the network more confident in its predictions. The segregation of the cross-modal fused features is described further, for example, in FIG.3, FIG.4A, and FIG, 4B.

[0087] At 614, the classifier model 114 may be applied on the segregated cross-modal fused features. The circuitry 202 may be configured to apply the classifier model 114 on the segregated cross-modal fused features. The classifier model 114 may correspond toDocket No. SYP354615WO02 the hyperbolic classifier 432. The hyperbolic classifier 432 may take the output of the HLGAtt model as the input and predict the confidence scores for normal and abnormal events. The final score can be represented as: ^^^^^^^^^^^^^^^^^^^^ = ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^(^^^^^^^^) (21)

[0088] In order toobjective may be adopted, which may calculate a mean value of the top k−max predictive scores within the set of multimedia segments 110. The high-scoring positive predictions may indicate the presence of abnormal events, while the k−max negative scores may represent normal samples. This learning objective function can be formulated as: ^^^^ 1 ^^^^^^^^^^^^= ^^^^∑^^^^^^^^=^^^^ −^^^^^^^^ . ^^^^^^^^^^^^(�^^^�^^�^^�^^^�^^�^^�^^^�^^^) (22)

[0089] At 616, each multimedia segment of the set of multimedia segments 110 may be classified into the one or more events, based on the application of the classifier model 114. The circuitry 202 may be configured to classify each multimedia segment of the set of multimedia segments 110 into the one or more events, based on the application of the classifier model 114. The video may be classified as normal if no anomalous event is detected. In contrast, the video may be classified as an anomaly if any form of abnormal events, such as violence or nudity, is present in the video. The WS-VAD methods usually employ the MIL for model training. Here, a regular video may be seen as a negative bag with no anomalous segments, while the anomalous video is viewed as positive bag with one or more anomalous segments. The anomaly evaluation function may be trained by optimizing the MIL loss to ensure that positive bag includes a higher anomaly value than negative (normal) bag. Control may pass to end.

[0090] Although the flowchart 600 is illustrated as discrete operations, such as, 604, 606, 608, 610, 612, 614, and 616, the disclosure is not so limited. Accordingly, in certain embodiments, such discrete operations may be further divided into additional operations,Docket No. SYP354615WO02 combined into fewer operations, or eliminated, depending on the particular implementation without detracting from the essence of the disclosed embodiments.

[0091] Various embodiments of the disclosure may provide a non-transitory computer- readable medium and / or storage medium having stored thereon, computer-executable instructions executable by a machine and / or a computer to operate an electronic device (for example, the electronic device 102). Such instructions may cause the electronic device 102 to perform operations that may include reception of multimedia content including a set of multimedia segments (such as, the set of multimedia segments 110) associated with the plurality of modalities. The received multimedia content may be used to determine the set of features associated with the received multimedia content for each modality. The operations may further include application of a bottleneck attention model (e.g., the bottleneck attention model 112) on the determined set of features. Based on the application of the bottleneck attention model 112 cross-modal fused features (such as, the cross-modal aware features 422) may be determined for the plurality of modalities associated with the received multimedia content. The operations may further include segregation of the determined cross-modal fused features into a set of nodes, based on one or more events associated with the received multimedia content and apply a classifier model (e.g., the classifier model 114) on the segregated cross-modal fused features to classify each multimedia segment of the set of multimedia segments 110 into the one or more events, based on the application of the classifier model 114.

[0092] Various embodiments of the disclosure may provide an electronic device (for example, the electronic device 102). The electronic device 102 may include circuitry (e.g., the circuitry 202) and memory (e.g., the memory 204). The circuitry 202 of the electronic device 102 may be configured to receive multimedia content including the set of multimedia segments 110 associated with a plurality of modalities. The circuitry 202 mayDocket No. SYP354615WO02 be configured to determine a set of features associated with the received multimedia content, based on an encoder model, for each modality of the plurality of modalities. Further, the circuitry 202 of the electronic device 102 may be configured to apply a bottleneck attention model (e.g., the bottleneck attention model 112) on the determined set of features and determine cross-modal fused features (such as, the cross-modal aware features 422) for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model 112. The circuitry 202 of the electronic device 102 may be further configured to segregate the determined cross- modal fused features (such as, the cross-modal aware features 422) into a set of nodes, based on one or more events associated with the received multimedia content. The circuitry 202 may be configured to apply a classifier model (e.g., the classifier model 114) on the segregated cross-modal fused features to classify each multimedia segment of the set of multimedia segments 110 into the one or more events, based on the application of the classifier model 114.

[0093] In an embodiment, the plurality of modalities may correspond to at least one or image frames, video data, or moving pictures.

[0094] In an embodiment, the bottleneck attention model 112 may correspond to a prefix-tuned bottleneck attention 418.

[0095] In an embodiment, the circuitry 202 may be further configured to determine query attributes based on first features from the set of features. The first features may be associated with a first modality of the plurality of modalities. The circuitry 202 may be configured to determine prefix-tuned attributes based on second features from the set of features. The second features may be associated with a second modality of the plurality of modalities.Docket No. SYP354615WO02

[0096] In an embodiment, the determined prefix-tuned attributes may correspond to at least one of a first set of prefix-tuned keys including first prefixes and keys of a key-value pair or a second set of prefix-tuned values including second prefixes and values of the key-value pair.

[0097] In an embodiment, the bottleneck attention model 112 may correspond to a cross-modal multi-head attention model and a bottleneck adapter model.

[0098] In an embodiment, the bottleneck adapter model may include at least one of a downscaling encoder model, a Gaussian Error Linear Unit (GELU) model, or an upscaling decoder model.

[0099] In an embodiment, the circuitry 202 may be further configured to apply the cross- modal multi-head attention model on the determined query attributes and the determined prefix-tuned attributes and determine an attention score based on the application of the cross-modal multi-head attention model. Further, the circuitry 202 may be configured to apply the bottleneck adapter model on the determined attention score. The determination of the cross-modal fused features (such as, the cross-modal aware features 422) may be further based on the application of the bottleneck adapter model.

[0100] In an embodiment, the circuitry 202 may be further configured to apply a modulation model on the second features of the set of features and determine third features based on the application of the modulation model. The circuitry 202 may be further configured to determine fourth features based on the application of the bottleneck attention model 112 and determine fused features based on a fusion of the determined third features and the determined fourth features. The determination of the cross-modal fused features (such as, the cross-modal aware features 422) may be further based on the determined fused features and the first features.Docket No. SYP354615WO02

[0101] In an embodiment, the classifier model 114 may correspond to a hyperbolic classifier model.

[0102] In an embodiment, the segregation of the determined cross-modal fused features (such as, the cross-modal aware features 422) may be based on a linear transformation and enhancement model corresponding to a Lorentz model.

[0103] In an embodiment, the circuitry 202 may further be configured to convert the determined cross-modal fused features (such as, the cross-modal aware features 422) to hyperbolic space features. The segregation of the determined cross-modal fused features may be further based on the converted cross-modal fused features. The circuitry 202 may be configured to apply a Hyperbolic Lorentzian Graph Attention (HL-GAtt) model on the converted cross-modal fused features. The application of the classifier model 114 may be further based on the application of the HL-GAtt model.

[0104] In an embodiment, the circuitry 202 may further be configured to determine an enhanced feature vector based on the application of the HL-GAtt model. The converted cross-modal fused features correspond to a temporal component and a spatial component in a hyperbolic space.

[0105] The present disclosure may be realized in hardware, or a combination of hardware and software. The present disclosure may be realized in a centralized fashion, in at least one computer system, or in a distributed fashion, where different elements may be spread across several interconnected computer systems. A computer system or other apparatus adapted to carry out the methods described herein may be suited. A combination of hardware and software may be a general-purpose computer system with a computer program that, when loaded and executed, may control the computer system such that it carries out the methods described herein. The present disclosure may beDocket No. SYP354615WO02 realized in hardware that includes a portion of an integrated circuit that also performs other functions.

[0106] The present disclosure may also be embedded in a computer program product, which includes all the features that enable the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program, in the present context, means any expression, in any language, code or notation, of a set of instructions intended to cause a system with information processing capability to perform a particular function either directly, or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.

[0107] While the present disclosure is described with reference to certain embodiments, it will be understood by those skilled in the art that various changes may be made, and equivalents may be substituted without departure from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departure from its scope. Therefore, it is intended that the present disclosure is not limited to the particular embodiment disclosed, but that the present disclosure will include all embodiments that fall within the scope of the appended claims.

Claims

Docket No. SYP354615WO02 CLAIMS What is claimed is:

1. An electronic device, comprising: circuitry configured to: receive multimedia content including a set of multimedia segments associated with a plurality of modalities; determine, for each modality of the plurality of modalities, a set of features associated with the received multimedia content, based on an encoder model; apply a bottleneck attention model on the determined set of features; determine cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model; segregate the determined cross-modal fused features into a set of nodes, based on one or more events associated with the received multimedia content; apply a classifier model on the segregated cross-modal fused features; and classify each multimedia segment of the set of multimedia segments into the one or more events, based on the application of the classifier model.

2. The electronic device according to claim 1, wherein the plurality of modalities corresponds to at least one or image frames, video data, or moving pictures.

3. The electronic device according to claim 1, wherein the bottleneck attention model corresponds to a prefix-tuned bottleneck attention model.

4. The electronic device according to claim 1, wherein the circuitry is further configured to:Docket No. SYP354615WO02 determine query attributes based on first features from the set of features, wherein the first features are associated with a first modality of the plurality of modalities; and determine prefix-tuned attributes based on second features from the set of features, wherein the second features are associated with a second modality of the plurality of modalities.

5. The electronic device according to claim 4, wherein the determined prefix-tuned attributes correspond to at least one of: a first set of prefix-tuned keys including first prefixes and keys of a key-value pair, or a second set of prefix-tuned values including second prefixes and values of the key-value pair.

6. The electronic device according to claim 4, the bottleneck attention model corresponds to a cross-modal multi-head attention model and a bottleneck adapter model.

7. The electronic device according to claim 6, wherein the bottleneck adapter model includes at least one of: a downscaling encoder model, a Gaussian Error Linear Unit (GELU) model, or an upscaling decoder model.Docket No. SYP354615WO02 8. The electronic device according to claim 6, wherein the circuitry is further configured to: apply the cross-modal multi-head attention model on the determined query attributes and the determined prefix-tuned attributes; determine an attention score based on the application of the cross-modal multi-head attention model; and apply the bottleneck adapter model on the determined attention score, wherein the determination of the cross-modal fused features is further based on the application of the bottleneck adapter model.

9. The electronic device according to claim 4, wherein the circuitry is further configured to: apply a modulation model on the second features of the set of features; determine third features based on the application of the modulation model; determine fourth features based on the application of the bottleneck attention model; and determine fused features based on a fusion of the determined third features and the determined fourth features, wherein the determination of the cross-modal fused features is further based on the determined fused features and the first features.

10. The electronic device according to claim 1, wherein the classifier model corresponds to a hyperbolic classifier model.Docket No. SYP354615WO02 11. The electronic device according to claim 1, wherein the segregation of the determined cross-modal fused features is based on a linear transformation and enhancement model corresponding to a Lorentz model.

12. The electronic device according to claim 1, wherein the circuitry is further configured to: convert the determined cross-modal fused features to hyperbolic space features, wherein the segregation of the determined cross-modal fused features is further based on the converted cross-modal fused features; and apply a Hyperbolic Lorentzian Graph Attention (HL-GAtt) model on the converted cross-modal fused features, wherein the application of the classifier model is further based on the application of the HL-GAtt model.

13. The electronic device according to claim 12, wherein the circuitry is further configured to: determine an enhanced feature vector based on the application of the HL- GAtt model, wherein the converted cross-modal fused features correspond to temporal and spatial features in a hyperbolic space.

14. A method, comprising: in an electronic device:Docket No. SYP354615WO02 receiving multimedia content including a set of multimedia segments associated with a plurality of modalities; determining, for each modality of the plurality of modalities, a set of features associated with the received multimedia content, based on an encoder model; applying a bottleneck attention model on the determined set of features; determining cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model; segregating the determined cross-modal fused features into a set of nodes, based on one or more events associated with the received multimedia content; applying a classifier model on the segregated cross-modal fused features; and classifying each multimedia segment of the set of multimedia segments into the one or more events, based on the application of the classifier model.

15. The method according to claim 14, further comprising: determine query attributes based on first features from the set of features, wherein the first features are associated with a first modality of the plurality of modalities; andDocket No. SYP354615WO02 determine prefix-tuned attributes based on second features from the set of features, wherein the second features are associated with a second modality of the plurality of modalities.

16. The method according to claim 15, wherein the determined prefix-tuned attributes include at least one of: a first set of prefix-tuned keys including first prefixes and keys of a key-value pair, or a second set of prefix-tuned values including second prefixes and values of the key-value pair.

17. The method according to claim 15, the bottleneck attention model corresponds to a cross-modal multi-head attention model and a bottleneck adapter model.

18. The method according to claim 17, further comprising: applying the cross-modal multi-head attention model on the determined query attributes and the determined prefix-tuned attributes; determining an attention score based on the application of the cross-modal multi-head attention model; and applying the bottleneck adapter model on the determined attention score, wherein the determination of the cross-modal fused features is further based on the application of the bottleneck adapter model.Docket No. SYP354615WO02 19. The method according to claim 17, further comprising: applying a modulation model on the second features of the set of features; determining third features based on the application of the modulation model; determining fourth features based on the application of the bottleneck attention model; and determining fused features based on a fusion of the determined third features and the determined fourth features, wherein the determination of the cross-modal fused features is further based on the determined fused features and the first features.

20. A non-transitory computer-readable medium having stored thereon, computer- executable instructions that when executed by an electronic device, causes the electronic device to execute operations, the operations comprising receiving multimedia content including a set of multimedia segments associated with a plurality of modalities; determining, for each modality of the plurality of modalities, a set of features associated with the received multimedia content, based on an encoder model; applying a bottleneck attention model on the determined set of features; determining cross-modal fused features for the plurality of modalities associated with the received multimedia content, based on the application of the bottleneck attention model; segregating the determined cross-modal fused features into a set of nodes, based on one or more events associated with the received multimedia content; applying a classifier model on the segregated cross-modal fused features; andDocket No. SYP354615WO02 classifying each multimedia segment of the set of multimedia segments into the one or more events, based on the application of the classifier model.

Images