Interactive behavior feature extraction method and device, equipment, medium and product

Abstract

The invention discloses an interactive behavior feature extraction method and device, equipment, a medium and a product, and relates to the technical field of computers, the method comprises the following steps: obtaining original live broadcast data generated in a live broadcast room, the live broadcast room comprising at least one interactive object; determining multi-modal live broadcast behavior data of the original live broadcast data in the target time window based on the multi-modal behavior data of the interactive object; dividing the multi-modal live broadcast behavior data into a plurality of behavior subsequences; based on the behavior data corresponding to the behavior subsequences, determining behavior cooperation features between the behavior subsequences, behavior interaction features between the interaction objects and time evolution features; and determining a target feature of the live broadcast room based on a fusion result of the behavior cooperation feature, the behavior interaction feature and the time evolution feature. By implementing the technical scheme of the invention, real-time modeling for the multi-behavior sub-sequence of the live broadcast room is realized, and the global feature of the live broadcast room level can be accurately extracted.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to methods, apparatus, devices, media, and products for feature extraction of interactive behaviors. Background Technology

[0002] With the rapid development of real-time interactive technology, live streaming has become an important infrastructure in many fields such as social networking, entertainment, and e-commerce. As the basic unit of live streaming services, the live streaming room carries intensive and complex interactive behaviors between the host and the audience. Currently, the modeling of interactive behaviors is insufficient to express the complex interactive patterns in the live streaming room. How to perform live streaming room-level feature representation based on these interactive behaviors is an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the present disclosure provides a method, apparatus, device, medium and product for feature extraction of interactive behavior, in order to solve the problem that complex interactive behavior is difficult to accurately represent.

[0004] In a first aspect, this disclosure provides a method for feature extraction of interactive behavior, comprising: acquiring raw live streaming data generated in a live streaming room, the live streaming room including at least one interactive object; determining multimodal live streaming behavior data of the raw live streaming data within a target time window based on the multimodal behavior data of the interactive object; dividing the multimodal live streaming behavior data into multiple behavior subsequences; determining behavioral coordination features between behavior subsequences, behavioral interaction features between interactive objects, and temporal evolution features based on the behavior data corresponding to each behavior subsequence; and determining the target features of the live streaming room based on the fusion result of behavioral coordination features, behavioral interaction features, and temporal evolution features.

[0005] Secondly, this disclosure provides a feature extraction device for interactive behavior, comprising: an acquisition module for acquiring raw live data generated in a live room, the live room including at least one interactive object; a multimodal processing module for determining multimodal live behavior data of the raw live data within a target time window based on the multimodal behavior data of the interactive object; a segmentation module for dividing the multimodal live behavior data into multiple behavior subsequences; a feature determination module for determining behavioral coordination features between behavior subsequences, behavioral interaction features between interactive objects, and temporal evolution features based on the behavior data corresponding to each behavior subsequence; and a feature fusion module for determining the target features of the live room based on the fusion result of behavioral coordination features, interaction features, and temporal evolution features.

[0006] Thirdly, this disclosure provides an electronic device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the feature extraction method for interactive behavior described in the first aspect or any corresponding embodiment.

[0007] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform the feature extraction method for interactive behavior described in the first aspect or any corresponding embodiment.

[0008] Fifthly, this disclosure provides a computer program product, including computer instructions for causing a computer to perform a feature extraction method for interactive behaviors described in the first aspect or any corresponding embodiment.

[0009] The interactive behavior feature extraction method, apparatus, device, medium, and product provided in this disclosure acquire raw live streaming data generated in a live streaming room, combine it with multimodal behavior data of various interactive objects during the interaction process, determine the multimodal live streaming behavior data of the raw live streaming data within a target time window, divide the multimodal live streaming behavior data into multiple behavior subsequences, and determine the behavioral coordination features between behavior subsequences, the behavioral interaction features between interactive objects, and the temporal evolution features based on the behavior data corresponding to each behavior subsequence. Then, the behavioral coordination features, behavioral interaction features, and temporal evolution features are fused to obtain target features at the live streaming room level. This allows for feature extraction for multiple interactive objects and multiple behavior subsequences, enabling real-time modeling of multiple behavior subsequences in the live streaming room to accurately extract global features at the live streaming room level, thereby accurately representing the interactive behavior generated in the live streaming room through global features. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of this disclosure, the accompanying drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram illustrating an application scenario according to an embodiment of this disclosure;

[0012] Figure 2 This is a flowchart illustrating a method for extracting features of interactive behaviors according to an embodiment of the present disclosure;

[0013] Figure 3 This is a flowchart illustrating another interactive behavior feature extraction method according to an embodiment of the present disclosure;

[0014] Figure 4 This is a flowchart illustrating another interactive behavior feature extraction method according to an embodiment of the present disclosure;

[0015] Figure 5 This is a schematic diagram illustrating the specific process of feature extraction for interactive behavior according to an embodiment of the present disclosure;

[0016] Figure 6 This is a structural block diagram of an interactive behavior feature extraction device according to an embodiment of the present disclosure;

[0017] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0023] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0024] Currently, users can achieve real-time content delivery and multi-party interaction through live streaming platforms, building a highly immersive online social ecosystem. As the basic unit of live streaming services, the live streaming room carries dense and complex interactive behaviors between the host and the audience: the host continuously outputs audio and video content, while the audience participates in real-time interaction through comments, likes, and rewards, forming a dynamic behavioral network with multiple modalities, roles, and rhythms.

[0025] Although live streaming platforms provide rich interaction mechanisms, their openness and real-time nature also bring a series of challenges to live streaming behavior modeling, specifically: (1) Complex fusion of multimodal behavior data: During the live streaming process, multi-source heterogeneous data such as video, audio, text, and interactive actions are generated simultaneously, which have obvious temporal and contextual dependencies, and the static modeling methods in related technologies are difficult to cover their complete behavioral semantics; (2) Sparse and ambiguous behavior fragments: Key behaviors often exist in short temporal fragments and usually depend on specific contexts for triggering. They are sparsely distributed and have unclear boundaries in large-scale data, making them difficult to extract accurately; (3) Significant user behavior collaboration: Live streaming interaction often involves short-term collaboration among multiple users. Single user behavior cannot effectively represent the overall pattern, and it is necessary to model the group behavior structure within the time window; (4) Weak supervision signals and coarse labeling granularity: The actual live streaming data only has overall labels at the live streaming level and lacks fine labeling for specific time periods or user behaviors; (5) High requirements for real-time processing and multi-task adaptation: The feature extraction system needs to have efficient and low-latency processing capabilities, support multiple tasks downstream of the platform, and ensure that the model output is reusable and has a certain degree of interpretability.

[0026] To address the aforementioned challenges, sequence-based modeling methods or multiple instance learning (MIL) methods are primarily employed. However, while sequence-based modeling methods extract dynamic user behavior patterns by modeling temporal dependencies within behavioral sequences, they rely on a complete time-series structure, making them ill-suited for issues such as discontinuous user participation and sparse behavior. Furthermore, they are highly dependent on annotation quality and have limited adaptability to weakly supervised scenarios. Multiple instance learning (MIL) methods aggregate a set of local behavioral fragments to achieve abstract modeling of the overall features of the live stream, adapting to weakly supervised scenarios with only live stream-level labels. However, they neglect the temporal structure and collaborative relationships between instances, making it difficult to fully represent the complex interaction patterns within the live stream. Therefore, these technologies still face a balancing challenge in terms of modeling granularity, temporal representation, and adaptability, making it difficult to perform live stream-level feature representation.

[0027] Based on this, the disclosed technical solution treats each live streaming room as a "bag" and abstracts the behavioral data of interactive objects within a time period into behavioral subsequences with an "object-timeslot capsule" structure. Subsequently, by combining local behavior modeling and cross-user dependency modeling, high-precision modeling and feature extraction of live streaming room behavior are achieved, taking into account temporal dynamics and multi-user dependencies. It can accurately represent the interactive behavior generated by the live streaming room through global features, thus adapting to scenarios with sparse behavior, heterogeneity, and collaborative concealment.

[0028] As one optional application scenario of this disclosure embodiment, such as Figure 1 As shown, this application scenario includes an electronic device 101 and a live streaming creation platform 102. The live streaming creation platform 102 is deployed within the electronic device 101 and is used for creating live streaming rooms and collecting live streaming data. The electronic device 101 and the live streaming creation platform 102 are connected via an API interface. The electronic device 101 performs live streaming data extraction and analysis to achieve accurate feature extraction of the live streaming data.

[0029] Electronic device 101 can be a device with computing capabilities. For example, electronic device 101 may be equipped with a processor and memory, and may also be equipped with a dedicated accelerator (such as a graphics processing unit (GPU)). In addition, electronic device 101 can store and maintain data.

[0030] Examples of electronic device 101 may include supercomputers, personal computers, laptop computers, in-vehicle computing devices, mobile devices (such as smartphones, tablets, etc.), or combinations thereof. It should be understood that the electronic devices described herein are merely exemplary and not limiting; other different types of electronic devices may also be employed.

[0031] According to an embodiment of this disclosure, a method for feature extraction of interactive behavior is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a method for feature extraction of interactive behaviors, which can be used in the aforementioned electronic devices, such as mobile phones, tablets, and computers. Figure 2 This is a flowchart of a feature extraction method for interactive behavior according to an embodiment of this disclosure, such as... Figure 2 As shown, the process includes the following steps:

[0033] Step S201: Obtain the raw live data generated in the live room, which includes at least one interactive object.

[0034] A live streaming room is an online video playback space created by a live streaming platform. It is created by the host (or content creator) and used to deliver information, entertainment, education, or other types of content to viewers. The raw live streaming data includes various data generated during the live streaming process, such as interactive communication, live streaming duration, activity level, bullet comments, and gift rewards. The interactive objects are the objects that participate in the interactive behavior, including the live stream audience and the viewers.

[0035] Specifically, the live streaming platform has an API interface, which can be used to retrieve raw live streaming data generated in the live room; raw live streaming data can also be obtained by accessing the logs of the live streaming server corresponding to the platform; or raw live streaming data can be captured from the live stream using network packet capture tools. Alternatively, a custom script can be written using a programming language to capture raw live streaming data. No specific restrictions are placed on the method of collecting raw live streaming data here.

[0036] Step S202: Based on the multimodal behavior data of the interactive objects, determine the multimodal live streaming behavior data of the original live streaming data within the target time window.

[0037] Multimodal behavioral data is data based on the granularity of the interactive objects' behaviors. This multimodal behavioral data can include video content in the live broadcast room (such as the host's performance, product display, etc.), audio content in the live broadcast room (such as background music, voiceovers, voice speech, etc.), text behavioral data (such as bullet screen interaction, comments, live broadcast room title, etc.), and other behavior-related data (such as likes, shares, rewards, viewing time, object identifiers, object login information, etc.). The target time window is the time window specified for feature extraction, such as [0, T] after the broadcast starts; multimodal live broadcast behavioral data is all the raw behavioral data generated by each interactive object within the target time window of the live broadcast room.

[0038] Specifically, for the original live streaming data in the live streaming room, based on the finest object behavior granularity, the original live streaming data in the live streaming room is processed and displayed in combination with the multimodal behavior data generated by the interactive objects in the live streaming room to obtain the corresponding multimodal live streaming behavior data.

[0039] Step S203: Divide the multimodal live streaming behavior data into multiple behavior subsequences.

[0040] A behavior subsequence is a sequence of behaviors corresponding to each time slice within the target time window. Specifically, multimodal live streaming behavior data is serialized according to time to obtain the corresponding live streaming behavior sequence. Then, the target time window is divided into multiple consecutive time slices, and the live streaming behavior sequence is further divided according to these time slices to obtain the behavior subsequences corresponding to each time slice. This allows the modeling of multimodal live streaming behavior data to be transformed into a multi-instance learning task, treating each live streaming room as a "package" to abstract the behavior data generated by interactive objects within each time slice into a behavior sequence.

[0041] Step S204: Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral coordination characteristics between behavioral subsequences, the behavioral interaction characteristics between interactive objects, and the temporal evolution characteristics.

[0042] Collaborative features are used to characterize the collaborative relationships between behavioral data of different behavioral subsequences; behavioral interaction features are used to characterize the interaction relationships between behavioral data of different behavioral subsequences; temporal evolution features are used to characterize the dynamic changes in behavior between behavioral data of different behavioral subsequences.

[0043] Each behavioral subsequence includes corresponding behavioral data. The behavioral data between different behavioral subsequences are correlated, and this correlation can include semantic correlation, temporal correlation, object interaction correlation, etc. Based on the behavioral data corresponding to each behavioral subsequence, the system captures the behavioral coordination characteristics of multiple interactive objects and multiple time slices within the live broadcast room.

[0044] By analyzing the behavioral data corresponding to each behavioral subsequence, the behavioral interaction structure between various interactive objects in the live broadcast room is captured, and the behavioral interaction characteristics between interactive objects are determined. Simultaneously, by analyzing the behavioral data corresponding to each behavioral subsequence, the cross-temporal behavioral evolution state of different behavioral subsequences is captured, and the temporal evolution characteristics of interactive behavior between different behavioral subsequences are determined.

[0045] Step S205: Based on the fusion results of behavioral collaboration features, behavioral interaction features and temporal evolution features, determine the target features of the live broadcast room.

[0046] The target feature is a global behavioral representation at the live stream level. It is used to understand and extract features from the interactive behaviors generated in the live stream. Specifically, it adaptively weights and fuses behavioral collaboration features, behavioral interaction features, and temporal evolution features to obtain the feature fusion result of behavioral collaboration features, behavioral interaction features, and temporal evolution features, which is the target feature corresponding to the live stream.

[0047] The interactive behavior feature extraction method provided in this embodiment acquires the original live streaming data generated in the live streaming room, combines it with the multimodal behavior data of each interactive object during the interaction process, determines the multimodal live streaming behavior data of the original live streaming data within a target time window, divides the multimodal live streaming behavior data into multiple behavior subsequences, and determines the behavioral coordination features between behavior subsequences, the behavioral interaction features between interactive objects, and the temporal evolution features based on the behavior data corresponding to each behavior subsequence. Then, the behavioral coordination features, behavioral interaction features, and temporal evolution features are fused to obtain the target features at the live streaming room level. This allows for feature extraction for multiple interactive objects and multiple behavior subsequences, enabling real-time modeling of multiple behavior subsequences in the live streaming room to accurately extract global features at the live streaming room level. Thus, the interactive behavior generated in the live streaming room can be accurately represented through global features.

[0048] This embodiment provides a method for feature extraction of interactive behaviors, which can be used in the aforementioned electronic devices, such as mobile phones, tablets, and computers. Figure 3 This is a flowchart of a feature extraction method for interactive behavior according to an embodiment of this disclosure, such as... Figure 3 As shown, the process includes the following steps:

[0049] Step S301: Obtain the raw live streaming data generated in the live streaming room, which includes at least one interactive object. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above; they will not be repeated here.

[0050] Step S302: Based on the multimodal behavior data of the interactive objects, determine the multimodal live streaming behavior data of the original live streaming data within the target time window.

[0051] Specifically, step S302 includes:

[0052] Step S3021: Extract the target live data within the target time window from the original live data.

[0053] The target live streaming data refers to the original live streaming data within the target time window. Specifically, the target time window for feature extraction is specified by the live streaming platform. For example, all original live streaming data of live streaming room r within the target time window [0, T] after the start of the broadcast is determined as the target live streaming data.

[0054] Step S3022: Analyze the target live streaming data to determine the object identifier, behavior timestamp, behavior type, and behavior text features of each interactive object in the live streaming room. The object identifier includes the anchor object identifier or the viewer object identifier.

[0055] The object identifier is the identifier of the interactive object, which can be the identifier of the broadcasting object or the identifier of the viewing object; the behavior timestamp is the time when the interactive behavior occurs; the behavior type is the type of interactive behavior, such as interactive communication, commenting, sending gifts, etc.; the behavior text feature is the feature generated by converting the interactive behavior into a text description.

[0056] Specifically, the target live stream data is parsed to extract object identifiers. For the behavioral data associated with each object identifier, the start time of the generated behavioral data, i.e., the behavior timestamp, is extracted. Simultaneously, the behavior type is identified for each behavioral data point, and the data is converted into text to obtain text features related to the behavioral data, i.e., behavioral text features.

[0057] Step S3023: The object identifier, behavior timestamp, behavior type and behavior text features are fused to obtain multimodal behavior data corresponding to each interactive object.

[0058] For any given live stream room, the multimodal behavioral data within that room is represented using a four-tuple format. Specifically, the object identifier, behavior timestamp, behavior type, and behavior text features within the live stream room are concatenated and fused according to the four-tuple structure to obtain the multimodal behavioral data corresponding to each interactive object within the live stream room. The specific representation of this multimodal behavioral data is as follows:

[0059] α=(u,t,a,x)

[0060] Where u∈U represents the set of object identifiers (IDs); t∈T is the behavior timestamp; a∈A=A streamer ∪A viewer Let x∈R be the set of all behavior types of the broadcaster and the viewer; d , which are behavioral text features related to behavior.

[0061] Step S3024: Based on the fusion results of various multimodal behavior data, determine the multimodal live streaming behavior data generated by the live streaming room within the target time window.

[0062] For any given live streaming room, the multimodal behavioral data generated by each interactive object within the live streaming room within the target time window is aggregated to obtain the behavioral data set of the live streaming room, that is, the multimodal live streaming behavioral data generated by the live streaming room within the target time window.

[0063] Specifically, the multimodal live streaming behavior data is represented as follows:

[0064]

[0065] in, This represents the multimodal live streaming behavior data of live streaming room r; This represents the collection of objects in the live stream 'r', consisting of a unique streamer object. and multiple viewers composition; This represents all multimodal behavior data of the live stream room r within the target time window [0, T] after the live stream begins.

[0066] Step S303: Divide the multimodal live streaming behavior data into multiple behavior subsequences.

[0067] Specifically, step S303 includes:

[0068] Step S3031: Divide the target time window into multiple preset time slices, and the multiple preset time slices are consecutive.

[0069] A preset time slice is a time interval obtained by dividing a target time window, and each preset time slice is continuous in time. Specifically, the target time window [0, T] is divided according to a preset time interval to obtain K consecutive preset time slices, i.e.

[0070] Step S3032: Divide the multimodal live streaming behavior data according to multiple preset time slices to obtain multiple continuous behavior subsequences.

[0071] The multimodal live streaming behavior data within a specified target time window is abstracted into a behavior sequence with an "object-time slice" structure. The multimodal live streaming behavior data is then divided according to determined preset time slices, resulting in behavior subsequences corresponding to each preset time slice. These behavior subsequences are continuous in time. Specifically, the behavior subsequences with the "object-time slice" structure are represented as follows:

[0072] C u,k ={α i |α i =(u i ,t i ,a i ,x i ),t i ∈T k}

[0073] Among them, C u,k For the interactive object u in time slice T k The behavior subsequence within.

[0074] Specifically, any live stream room r can be represented as a "package" B by combining various behavioral subsequences. r , means as follows:

[0075] B r={C u,k |u∈U,k=1,2…,K}

[0076] Among them, B r This represents the sequence of actions in the "object-time slice" structure corresponding to the live stream room r; K is the number of preset time slices.

[0077] Step S304: Based on the behavioral data corresponding to each behavioral sub-sequence, determine the behavioral coordination characteristics between behavioral sub-sequences, the behavioral interaction characteristics between interacting objects, and the temporal evolution characteristics. For details, please refer to the relevant descriptions of the steps in the embodiments shown above, which will not be repeated here.

[0078] Step S305: Based on the fusion results of behavioral collaboration features, behavioral interaction features, and temporal evolution features, determine the target features of the live streaming room. For details, please refer to the relevant descriptions of the corresponding steps in the above embodiments; they will not be repeated here.

[0079] The interactive behavior feature extraction method provided in this embodiment retrieves target live streaming data within a target time window to determine the object identifier, behavior timestamp, behavior type, and behavior text features of each interactive object in the live streaming room. This allows for processing of live streaming data within a specified time window. Multimodal behavior data is constructed using object identifiers, behavior timestamps, behavior types, and behavior text features. This multimodal behavior data is then used to determine multimodal live streaming behavior data within the target time window, thereby covering the complete behavioral semantics of the target live streaming data based on temporal sequence and contextual dependencies. By dividing the target time window into multiple consecutive preset time slices, and further dividing the multimodal live streaming behavior data into multiple consecutive behavior subsequences according to these preset time slices, the key behavioral features of the live streaming room can be accurately extracted by combining these multiple consecutive behavior subsequences. This effectively captures the spatiotemporal correlation features of interactive behavior without relying on specific context triggers, effectively improving the modeling accuracy and feature interpretability of live streaming behavior features.

[0080] This embodiment provides a method for feature extraction of interactive behaviors, which can be used in the aforementioned electronic devices, such as mobile phones, tablets, and computers. Figure 4 This is a flowchart of a feature extraction method for interactive behavior according to an embodiment of this disclosure, such as... Figure 4 As shown, the process includes the following steps:

[0081] Step S401: Obtain the raw live streaming data generated in the live streaming room, which includes at least one interactive object. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above; they will not be repeated here.

[0082] Step S402: Based on the multimodal behavior data of the interactive objects, determine the multimodal live streaming behavior data of the original live streaming data within the target time window. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above, which will not be repeated here.

[0083] Step S403: Divide the multimodal live streaming behavior data into multiple behavior subsequences. For details, please refer to the relevant descriptions of the corresponding steps in the embodiments shown above; they will not be repeated here.

[0084] Step S404: Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral coordination characteristics between behavioral subsequences, the behavioral interaction characteristics between interactive objects, and the temporal evolution characteristics.

[0085] Specifically, step S404 includes:

[0086] Step S4041: Based on the behavioral data corresponding to each behavioral subsequence, construct a behavioral graph structure between each behavioral subsequence.

[0087] Behavior graph structures are used to characterize the collaborative relationships between different behavioral subsequences. By using this behavior graph structure, cross-temporal behavioral interactions between different behavioral subsequences can be analyzed, thereby improving the interpretability and behavioral representation capabilities of the subsequently extracted features.

[0088] Specifically, the behavioral data in each behavioral subsequence is quantified, the data features corresponding to each behavioral subsequence are extracted, and the data similarity between different behavioral subsequences is calculated using the extracted data features. Then, the behavioral subsequences are connected according to the data similarity to generate the corresponding behavioral graph structure.

[0089] In some optional implementations, step S4041 above includes:

[0090] Step a1: Treat each row sub-column as a graph node and determine the node correlation between each graph node.

[0091] Step a2: Generate a behavioral graph structure based on the node correlations between graph nodes.

[0092] Node correlation represents the correlation between different behavioral subsequences. Specifically, each behavioral subsequence is used as a graph node to construct the behavioral graph structure, and each graph node is mapped to a global graph node, resulting in a global graph node set, as shown below:

[0093]

[0094] Wherein, each graph node v i Represents the interactive object u i In time slice Behavioral data within.

[0095] By combining selected similarity metrics (such as feature-based similarity, neighbor-based similarity, path-based similarity, and graph embedding-based similarity), the similarity between each pair of graph nodes is calculated and filled into a similarity matrix. This similarity matrix characterizes the node correlations between graph nodes, and the connection relationships between graph nodes are determined using these correlations, generating corresponding behavioral graph structures between each graph node. This allows for the focus on complex and time-series-based interaction patterns through behavioral graph structures, enabling the capture of collaborative relationships between interactive behaviors.

[0096] In some optional implementations, the method further includes: determining the type of behavioral relationship between each behavioral subsequence based on the behavioral data corresponding to each behavioral subsequence.

[0097] Behavioral relationship types represent the interaction relationships between different behavioral subsequences, specifically including temporal relationships, object relationships, role relationships, and collaborative support relationships. Each behavioral relationship type can be represented by a binary mask matrix. Specifically, temporal relationships represent interactions within behavioral subsequences corresponding to adjacent time slices; user relationships represent the evolution of behavioral patterns of the same object across time slices; role relationships represent interactions between the broadcaster and the viewer; and support relationships represent complementary relationships used to supplement unmodeled collaborative relationships. These support relationships can be obtained through logical reasoning or model prediction. Specifically, the predefined behavioral relationship types are as follows:

[0098] R = {t1, u1, r1, a1}

[0099] Where t1 represents time relationship; u1 represents object relationship; r1 represents role relationship; and a1 represents auxiliary relationship.

[0100] The behavioral data in each behavioral subsequence is analyzed to extract the corresponding behavioral features. Based on these features, the behavioral relationships within the subsequences are categorized. These behavioral features can then be used to capture key structures within the subsequences, enabling precise identification of different interaction patterns occurring in the live stream.

[0101] Step S4042: Based on the behavioral relationships represented by the behavioral graph structure, determine the behavioral synergy features of each behavioral subsequence.

[0102] Behavioral associations represent the relationships between different graph nodes across multiple dimensions, including semantics, time, objects, and interactions.

[0103] Specifically, the behavior graph structure is constructed using each behavior subsequence as a graph node. A graph-aware self-attention mechanism is used to inject the structural priors (i.e., the structural information of the behavior graph structure) into the corresponding neural network layers, enabling deep interaction modeling between different behavior subsequences. Based on the results of this deep interaction modeling, behavioral collaborative features between each behavior subsequence are extracted.

[0104] Among them, the neural network layer is the behavior subsequence feature layer with an "object-time slice" structure, also known as the capsule layer. The capsule layer includes various capsules formed based on behavior subsequences, and the contents of the capsules are the behavior data contained in the behavior subsequences.

[0105] In some optional implementations, step S4042 above includes:

[0106] Step b1: parse the behavioral association content corresponding to the behavioral association relationship, and determine the semantic similarity and multi-dimensional behavioral structured information between each behavioral sub-sequence.

[0107] Step b2: Integrate semantic similarity and multi-dimensional behavioral structured information to generate behavioral collaborative features.

[0108] Semantic similarity represents the semantic similarity between different behavioral subsequences; multi-dimensional behavioral structured information represents structured information in multiple dimensions such as time relationships, object relationships, and role relationships. Specifically, after constructing the behavioral graph structure for each behavioral subsequence in the "object-time slice" structure, the behavioral graph structure prior is injected into the capsule layer through a graph-aware self-attention mechanism to parse the behavioral association content corresponding to the behavioral association relationship, thereby obtaining the semantic similarity and multi-dimensional behavioral structured information between different behavioral subsequences. Specifically, the specific calculation method of the graph-aware self-attention mechanism is as follows:

[0109]

[0110] Among them, A G The adjacency matrix represents the graph-aware approach, which integrates semantic similarity between different behavioral subsequences and various behavioral structured relationships to guide information transfer and weighting between behavioral subsequences in the self-attention mechanism; the input behavioral subsequences are linearly transformed to obtain the query matrix Q, the key matrix K, and the value matrix V; This represents the scaling factor.

[0111] By leveraging the semantic similarity between behavioral subsequences and fusing them with multi-dimensional structured behavioral information such as temporal relationships, object relationships, and role relationships, corresponding behavioral collaboration features are obtained. Therefore, behavioral collaboration features can be used to comprehensively capture the behavioral interaction patterns of multiple interactive objects and across multiple time slices within a live stream, enabling the modeling of the behavioral structure of interactive object groups within a target time window and effectively representing the overall interactive characteristics of the live stream.

[0112] Step S4043: Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral change data corresponding to each interactive object and the behavioral interaction features corresponding to the behavioral change data.

[0113] Behavioral change data represents the individual behavioral evolution of each interactive object in different behavioral subsequences, as well as the collaborative behavioral evolution among interactive objects; behavioral interaction features represent the behavioral characteristics generated by the interactive behaviors among multiple interactive objects.

[0114] Specifically, the behavioral data corresponding to each behavioral subsequence is analyzed to identify the interactive objects within the behavioral data corresponding to each behavioral subsequence, in order to extract the behavioral change data corresponding to each interactive object. Behavioral change features are extracted from the behavioral change data to determine the behavioral interaction features represented by the behavioral change data.

[0115] In some optional implementations, step S4043 above includes:

[0116] Step c1 involves performing temporal encoding on the behavioral data generated by the interactive object in the behavioral subsequence to determine the behavioral change data of the interactive object.

[0117] Step c2 involves aggregating the behavioral change data corresponding to each interactive object to obtain behavioral interaction features.

[0118] The behavioral subsequences generated by each interactive object at each time slice are temporally encoded, and a recurrent neural network structure is used to capture the dynamic changes in the individual behaviors of each interactive object, i.e., behavioral change data. This recurrent neural network structure can be a gated recurrent unit (GRU).

[0119] Subsequently, based on an attention mechanism, the behavioral change data corresponding to all interactive objects are weighted and fused to obtain the corresponding behavioral interaction features. These behavioral interaction features can automatically identify and highlight key interactive objects (such as the broadcaster) to form a global structural representation of the interactive objects.

[0120] Step S4044: Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral summary data corresponding to each behavioral subsequence and the temporal evolution characteristics corresponding to the behavioral summary data.

[0121] Behavioral summary data is a behavioral summary representation generated by summarizing the behavioral subsequences corresponding to the same time slice; temporal evolution features represent the features generated by behavioral changes across time slices, and these temporal evolution features can characterize the changes in behavior over time.

[0122] Specifically, for any behavioral subsequence, the behavioral data is semantically summarized to determine the behavioral summary data corresponding to that subsequence. Features are extracted from the changing states of the behavioral summary data across different behavioral subsequences to obtain the temporal evolution features corresponding to the behavioral summary data.

[0123] In some optional implementations, step S4044 above includes:

[0124] Step d1: For any behavior subsequence, fuse the behavior data within the same behavior subsequence to obtain the behavior summary data corresponding to the behavior subsequence.

[0125] Step d2 involves aggregating the behavioral summary data corresponding to each behavioral subsequence to obtain the temporal evolution features.

[0126] For any given time slice, a behavior subsequence is fused using an attention mechanism to extract behavior summary representations for that time slice. This behavior subsequence is then input into a recurrent neural network (RNN) structure to capture the evolution of interactive behavior over time and determine the corresponding temporal evolution features. This RNN structure can be a gated recurrent unit (GRU). Therefore, these temporal evolution features can be used to identify dynamic patterns and collaborative behavior trends across time slices.

[0127] Step S405: Based on the fusion results of behavioral collaboration features, behavioral interaction features and temporal evolution features, determine the target features of the live broadcast room.

[0128] Specifically, step S405 includes:

[0129] Step S4051: Based on the feature fusion strategy corresponding to the live broadcast room, extract the features to be fused from the behavioral collaboration features, behavioral interaction features and time evolution features, and determine the fusion weights corresponding to the features to be fused.

[0130] Feature fusion strategies represent the merging of features from different levels into a single feature, and different live streaming rooms can correspond to different feature fusion strategies. Specifically, features to be fused are extracted from multiple levels of live streaming room behavior representations, such as behavioral collaboration features, behavioral interaction features, and temporal evolution features, according to the feature fusion strategy corresponding to the gating mechanism. A gating mechanism is used to assign corresponding fusion weights to each feature to be fused, and the fusion weights of each feature to be fused in the final fused representation are dynamically adjusted to enhance adaptability to information of different granularities.

[0131] Step S4052: The features to be fused are weighted and fused according to the fusion weights to obtain the target features corresponding to the live broadcast room.

[0132] The extracted features to be fused are weighted and fused according to the fusion weights, and the feature representation obtained by weighted fusion is input into a multi-layer perceptron (MLP) with several layers to generate a live room-level behavioral representation, that is, the target feature corresponding to the live room, which supports a variety of downstream tasks.

[0133] In a specific example, the feature representation obtained by weighted fusion is input into a two-layer MLP. Each MLP layer includes an input layer, one or more hidden layers, and an output layer. The input layer is used for data input, the hidden layers consist of multiple neurons, each of which processes its input through an activation function and passes the result to the next layer; the output layer is used to produce the final output. Specifically, the input layer of the first MLP is used to input the fused feature representation, the input layer of the second MLP is the output layer data of the first MLP; the output layer of the first MLP is used to output the preliminary perceived features, and the output layer of the second MLP is used to output the target features corresponding to the live broadcast room.

[0134] The interactive behavior feature extraction method provided in this embodiment constructs a behavior graph structure between various behavior subsequences, extracts behavioral collaboration features of each behavior subsequence based on the behavior graph structure, and extracts behavioral interaction features of each interactive object using the behavioral data corresponding to the behavior subsequences. This achieves modeling of the behavioral evolution of interactive objects and the collaborative behavioral structure between interactive objects. Using the behavioral data corresponding to the behavior subsequences, the method extracts the temporal evolution features of each interactive object, achieving temporal-dimensional behavioral aggregation and evolutionary trend modeling. Therefore, by capturing the behavioral interaction structure and dynamic behavioral evolution patterns across time among multiple interactive objects, the method significantly improves the representation capability of complex behaviors in live streaming rooms. By aggregating behavioral collaboration features, behavioral interaction features, and temporal evolution features, a global live streaming room-level feature representation is generated, enabling a more accurate and comprehensive understanding and feature extraction of the target features of the live streaming room.

[0135] As a specific application embodiment of this disclosure, such as Figure 5 As shown, by acquiring the original live streaming data from the live streaming room, the original live streaming data is processed into a user multimodal behavior sequence to obtain the behavior sequence corresponding to the live streaming room. Then, this behavior sequence is abstracted into "object-time slice" capsules to obtain multiple capsules corresponding to consecutive time slices. Each capsule is a behavior subsequence corresponding to its respective time slice, and the content of each capsule is the behavior data corresponding to its respective behavior subsequence.

[0136] By employing graph-structured reasoning within capsule layers, the system captures the collaborative relationships between various behaviors, focusing on complex and time-series-based interaction patterns to enhance feature interpretability and behavioral representation capabilities. Simultaneously, capsule features are reconstructed from two dimensions: interactive objects and time slices, enabling parallel modeling to capture the behavioral interaction structure among multiple interactive objects and the dynamic behavioral evolution patterns across time, significantly improving the representation capabilities of complex behaviors in live streaming rooms. Finally, live streaming room behavioral representations from multiple semantic levels, including capsule layers and parallel modeling, are aggregated and adaptively weighted through a gating mechanism to generate a global live streaming room-level representation for more accurate and comprehensive behavioral understanding and feature extraction.

[0137] This embodiment also provides a feature extraction device for interactive behavior, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0138] This embodiment provides a feature extraction device for interactive behavior, such as... Figure 6 As shown, it includes:

[0139] The acquisition module 501 is used to acquire the raw live data generated in the live room, which includes at least one interactive object.

[0140] The multimodal processing module 502 is used to determine the multimodal live streaming behavior data of the original live streaming data within the target time window based on the multimodal behavior data of the interactive object.

[0141] The segmentation module 503 is used to divide multimodal live streaming behavior data into multiple behavior subsequences.

[0142] The feature determination module 504 is used to determine the behavioral coordination features between behavioral subsequences, the behavioral interaction features between interactive objects, and the temporal evolution features based on the behavioral data corresponding to each behavioral subsequence.

[0143] The feature fusion module 505 is used to determine the target features of the live broadcast room based on the fusion results of behavioral collaboration features, interaction features and temporal evolution features.

[0144] In some alternative implementations, the multimodal processing module 502 includes:

[0145] The target data extraction unit is used to extract target live data within the target time window from the original live data.

[0146] The data parsing unit is used to parse the target live data and determine the object identifier, behavior timestamp, behavior type, and behavior text features of each interactive object in the live room. The object identifier includes the anchor object identifier or the viewer object identifier.

[0147] The feature fusion unit is used to concatenate object identifiers, behavior timestamps, behavior types, and behavior text features to obtain multimodal behavior data corresponding to each interactive object.

[0148] The multimodal data determination unit is used to determine the multimodal live streaming behavior data generated by the live streaming room within the target time window based on the fusion results of various multimodal behavior data.

[0149] In some alternative implementations, the partitioning module 503 includes:

[0150] The time slice division unit is used to divide the target time window into multiple preset time slices, and the multiple preset time slices are consecutive.

[0151] The subsequence determination unit is used to divide the multimodal live broadcast behavior data according to multiple preset time slices to obtain multiple continuous behavior subsequences.

[0152] In some alternative implementations, the feature determination module 504 includes:

[0153] The graph structure building unit is used to construct a behavior graph structure between various behavior subsequences based on the behavior data corresponding to each behavior subsequence.

[0154] The collaborative feature determination unit is used to determine the collaborative features of each behavioral subsequence based on the behavioral associations represented by the behavioral graph structure.

[0155] The interaction feature determination unit is used to determine the behavior change data and the behavior interaction features corresponding to each interactive object based on the behavior data corresponding to each behavior subsequence.

[0156] The evolution feature determination unit is used to determine the behavioral summary data corresponding to each behavioral subsequence and the temporal evolution features corresponding to the behavioral summary data based on the behavioral data corresponding to each behavioral subsequence.

[0157] In some alternative implementations, the graph structure building unit includes:

[0158] The correlation determination sub-unit is used to determine the node correlation between each graph node by treating each row sub-column as a graph node.

[0159] The graph structure generation sub-unit is used to generate behavioral graph structures based on the node correlations between graph nodes.

[0160] In some alternative implementations, the graph structure building unit further includes:

[0161] The relation type determination subunit is used to determine the behavior relation type between each behavior subsequence based on the behavior data corresponding to each behavior subsequence.

[0162] In some optional implementations, the collaborative feature determination unit includes:

[0163] The parsing subunit is used to parse the behavioral association content corresponding to the behavioral association relationship, determine the semantic similarity between each behavioral subsequence, and obtain multi-dimensional behavioral structured information.

[0164] The collaborative feature generation subunit is used to fuse semantic similarity and multi-dimensional behavioral structured information to generate behavioral collaborative features.

[0165] In some optional implementations, the interaction feature determination unit includes:

[0166] The encoding subunit is used to perform temporal encoding on the behavioral data generated by the interactive object in the behavioral subsequence to determine the behavioral change data of the interactive object.

[0167] The aggregation subunit is used to aggregate the behavioral change data corresponding to each interactive object to obtain behavioral interaction features.

[0168] In some optional implementations, the evolutionary feature determination unit includes:

[0169] The summary determination subunit is used to fuse behavioral data within the same behavioral subsequence for any given behavioral subsequence, thereby obtaining behavioral summary data corresponding to the behavioral subsequence.

[0170] The summary fusion subunit is used to aggregate the behavioral summary data corresponding to each behavioral subsequence to obtain temporal evolution features.

[0171] In some alternative implementations, the feature fusion module 505 includes:

[0172] The fusion weight determination unit is used to extract the features to be fused from behavioral collaboration features, behavioral interaction features and temporal evolution features based on the feature fusion strategy corresponding to the live broadcast room, and determine the fusion weight corresponding to the features to be fused.

[0173] The target feature determination unit is used to perform weighted fusion of the features to be fused according to the fusion weights to obtain the target features corresponding to the live broadcast room.

[0174] The interactive behavior feature extraction device provided in this disclosure can execute the interactive behavior feature extraction method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method. By acquiring the original live streaming data generated in the live streaming room, and combining it with the multimodal behavior data of each interactive object during the interaction process, the multimodal live streaming behavior data of the original live streaming data within the target time window is determined. The multimodal live streaming behavior data is then divided into multiple behavior sub-sequences, and based on the behavior data corresponding to each behavior sub-sequence, the behavioral coordination features between behavior sub-sequences, the behavioral interaction features between interactive objects, and the temporal evolution features are determined. Subsequently, the behavioral coordination features, behavioral interaction features, and temporal evolution features are fused to obtain the target features at the live streaming room level. This allows for feature extraction for multiple interactive objects and multiple behavior sub-sequences, enabling real-time modeling of multiple behavior sub-sequences in the live streaming room to accurately extract global features at the live streaming room level. This allows for accurate characterization of the interactive behavior generated in the live streaming room through global features.

[0175] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0176] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0177] The following is a detailed reference. Figure 7 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present disclosure. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0178] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0179] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the feature extraction method for interactive behavior of embodiments of this disclosure.

[0180] Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0181] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded via a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the feature extraction method for the interactive behavior shown in the above embodiments is implemented.

[0182] A portion of this disclosure can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide methods and / or technical solutions according to this disclosure through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0183] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for feature extraction of interactive behavior, characterized in that, The method includes: Acquire raw live data generated in a live streaming room, wherein the live streaming room includes at least one interactive object; Based on the multimodal behavior data of the interactive object, determine the multimodal live streaming behavior data of the original live streaming data within the target time window; The multimodal live streaming behavior data is divided into multiple behavior subsequences; Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral coordination characteristics between the behavioral subsequences, the behavioral interaction characteristics between the interactive objects, and the temporal evolution characteristics. Based on the fusion results of the behavioral coordination features, the behavioral interaction features, and the temporal evolution features, the target features of the live streaming room are determined.

2. The method according to claim 1, characterized in that, The determination of multimodal live streaming behavior data of the original live streaming data within a target time window based on the multimodal behavior data of the interactive object includes: Extract the target live stream data within the target time window from the original live stream data; The target live streaming data is analyzed to determine the object identifier, behavior timestamp, behavior type, and behavior text features of each interactive object in the live streaming room. The object identifier includes the anchor object identifier or the viewer object identifier. The object identifier, the behavior timestamp, the behavior type, and the behavior text features are fused to obtain multimodal behavior data corresponding to each interactive object; Based on the fusion results of the various multimodal behavioral data, the multimodal live streaming behavioral data generated by the live streaming room within the target time window is determined.

3. The method according to claim 1 or 2, characterized in that, The step of dividing the multimodal live streaming behavior data into multiple behavior sub-sequences includes: The target time window is divided into multiple preset time slices, and the multiple preset time slices are consecutive. The multimodal live streaming behavior data is divided according to multiple preset time slices to obtain multiple consecutive behavior subsequences.

4. The method according to claim 1, characterized in that, Based on the behavioral data corresponding to each of the behavioral sub-sequences, the spatiotemporal correlation characteristics between the behavioral sub-sequences, the behavioral interaction characteristics between the interactive objects, and the temporal evolution characteristics are determined, including: Based on the behavioral data corresponding to each behavioral subsequence, construct a behavioral graph structure between each behavioral subsequence; Based on the behavioral relationships represented by the behavioral graph structure, the behavioral collaborative features of each behavioral subsequence are determined; Based on the behavioral data corresponding to each behavioral subsequence, determine the behavioral change data corresponding to each interactive object and the behavioral interaction features corresponding to the behavioral change data. Based on the behavioral data corresponding to each behavioral subsequence, behavioral summary data corresponding to each behavioral subsequence and temporal evolution features corresponding to the behavioral summary data are determined.

5. The method according to claim 4, characterized in that, The step of constructing a behavior graph structure between the various behavior subsequences based on the behavior data corresponding to each behavior subsequence includes: Each of the aforementioned behavior sub-columns is used as a graph node, and the node correlation between the aforementioned graph nodes is determined. The behavioral graph structure is generated based on the node correlations between the graph nodes.

6. The method according to claim 5, characterized in that, Also includes: Based on the behavioral data corresponding to each behavioral subsequence, the behavioral relationship type between each behavioral subsequence is determined.

7. The method according to any one of claims 4-6, characterized in that, The determination of behavioral co-operation features for each behavioral sub-sequence based on the behavioral associations represented by the behavioral graph structure includes: Analyze the behavioral association content corresponding to the behavioral association relationship, and determine the semantic similarity and multi-dimensional behavioral structured information between each behavioral sub-sequence; The semantic similarity and multi-dimensional behavioral structured information are fused to generate the behavioral collaborative features.

8. The method according to claim 4, characterized in that, The step of determining the behavior change data corresponding to each interactive object and the behavior interaction features corresponding to the behavior change data based on the behavior data corresponding to each behavior subsequence includes: The behavioral data generated by the interactive object in the behavioral sub-sequence is temporally encoded to determine the behavioral change data of the interactive object; The behavioral change data corresponding to each of the interactive objects are aggregated to obtain the behavioral interaction features.

9. The method according to claim 4, characterized in that, The step of determining the behavioral evolution data corresponding to each behavioral subsequence and the temporal evolution features corresponding to the behavioral summary data based on the behavioral data corresponding to each behavioral subsequence includes: For any given behavior subsequence, the behavior data within the same behavior subsequence are fused to obtain the behavior summary data corresponding to the behavior subsequence; The behavioral summary data corresponding to each behavioral subsequence are aggregated to obtain the temporal evolution features.

10. The method according to claim 1, characterized in that, The determination of the target features of the live streaming room based on the fusion result of the behavioral coordination features, the behavioral interaction features, and the temporal evolution features includes: Based on the feature fusion strategy corresponding to the live broadcast room, features to be fused are extracted from the behavioral coordination features, the behavioral interaction features and the temporal evolution features, and the fusion weights corresponding to the features to be fused are determined. The features to be fused are weighted and fused according to the fusion weights to obtain the target features corresponding to the live broadcast room.

11. A feature extraction device for interactive behavior, characterized in that, The device includes: The acquisition module is used to acquire raw live data generated in the live room, which includes at least one interactive object. A multimodal processing module is used to determine the multimodal live streaming behavior data of the original live streaming data within a target time window based on the multimodal behavior data of the interactive object. The segmentation module is used to divide the multimodal live streaming behavior data into multiple behavior subsequences; The feature determination module is used to determine the behavioral coordination features between the behavioral subsequences, the behavioral interaction features between the interactive objects, and the temporal evolution features based on the behavioral data corresponding to each behavioral subsequence. The feature fusion module is used to determine the target features of the live broadcast room based on the fusion result of the behavioral coordination features, the interaction features and the temporal evolution features.

12. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the feature extraction method for interactive behavior according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the feature extraction method for interactive behavior according to any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes computer instructions for causing a computer to perform the feature extraction method for interactive behavior according to any one of claims 1 to 10.

Images