Repeated on-air event detection method and apparatus, device, and medium

Abstract

The application discloses a repeated broadcasting event detection method and device, equipment and a medium. The method comprises the following steps: obtaining a target face image in a live video stream of a target user; performing semantic matching on a face feature vector of the target face image and a face feature set of a stock user in a user feature library to obtain semantic similarities corresponding to multiple face feature vectors in the face feature set of each stock user; and determining whether the highest semantic similarity corresponding to each stock user exceeds a preset threshold value to determine that a repeated broadcasting event exists between the stock user and the target user when the highest semantic similarity exceeds the preset threshold value. The application accurately identifies the repeated broadcasting event based on the semantic matching between the target face image of the target user and multiple face images of each stock user, effectively protects the network live broadcast order, and ensures that the live broadcast environment is socially fair.

Description

Technical Field

[0001] This application relates to the field of live streaming technology, and in particular to a method for detecting repeated broadcasting events and the corresponding apparatus, electronic equipment and computer-readable storage medium. Background Technology

[0002] In online live streaming, users push video streams to their rooms to showcase their talents, share information, and educate others. This allows users to participate in social labor and earn income, contributing to overall social benefits. However, some users, seeking higher personal gains, may register multiple accounts to broadcast repeatedly, leading to disorder in the live streaming environment. Therefore, investigating and addressing duplicate broadcasts, maintaining order in online live streaming, and ensuring a greater sense of social fairness among users are of positive significance.

[0003] In traditional techniques, one method for identifying repeated broadcasting events involves clustering user accounts based on their features. This is done by extracting features from user registration information or other personal identity data, such as facial images, and then clustering them to identify the same user group forming a community relationship, thus enabling the identification of suspected repeated broadcasting relationships. This method is more suitable for identifying user groups that collaboratively increase followers through multiple accounts, but it struggles to pinpoint individual broadcasters and their personal information. Therefore, its effectiveness in solving the problem of repeated broadcasting events is very limited.

[0004] More detailed implementations of duplicate broadcasting event identification methods involve acquiring facial information from different users and comparing the similarity between these facial images to identify suspected duplicate broadcasting events. The main drawback of this approach is that it doesn't consider the reliability of the facial information data source. Even if high similarity is obtained as a basis for identification, the data source itself is often of poor quality, resulting in inaccurate facial information and ultimately, inaccurate identification results. Summary of the Invention

[0005] The primary objective of this application is to solve at least one of the above-mentioned problems by providing a method for detecting repeated broadcasting events, and corresponding apparatus, electronic equipment, and computer-readable storage medium.

[0006] To achieve the various objectives of this application, the following technical solution is adopted:

[0007] A method for detecting repeated broadcasting events, proposed to meet one of the purposes of this application, includes the following steps:

[0008] Acquire the target face image from the target user's live video stream;

[0009] The facial feature vector of the target face image is semantically matched with the facial feature set of existing users in the user feature database to obtain the semantic similarity of multiple facial feature vectors in the facial feature set of each existing user.

[0010] Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determine that there is a duplicate broadcasting event between the existing user exceeding the preset threshold and the target user.

[0011] Optionally, determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determine that there is a duplicate broadcast event between the existing user exceeding the preset threshold and the target user, including:

[0012] Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold. When the highest semantic similarity of all existing users in the user feature library does not exceed the preset threshold, the target user is identified as a new broadcast user.

[0013] When there is a candidate existing user whose highest semantic similarity exceeds the preset threshold, it is determined whether the feature identifier of the candidate existing user is the same as that of the target user. If the feature identifier is the same, the target user is confirmed as a historical user.

[0014] When the feature identifiers of the candidate existing users and the target user are different, it is determined that there is a duplicate live broadcast event between the target user and the candidate existing users, and live broadcast permission constraint control is implemented on the target user.

[0015] Optionally, after identifying the target user as a new broadcast user, the following steps are included:

[0016] Obtain image frames containing facial content from the live video stream of the new broadcast user, and extract the facial images from each image frame to form an image set to be added to the database;

[0017] Clustering is performed on the facial feature vectors of each face image in the image set to be added to the database. The largest cluster is determined from the multiple clusters obtained by clustering, and the largest cluster is used as the facial feature set of the new broadcast user.

[0018] Add the facial feature set of the newly broadcasting user to the user feature database.

[0019] Optionally, adding the facial feature set of the newly broadcasting user to the user feature database includes:

[0020] Based on the facial feature sets of all new broadcast users, the semantic similarity is calculated by cross-referencing the facial feature vectors in the facial feature sets of every two new broadcast users.

[0021] Compare whether the highest semantic similarity between any two new broadcast users exceeds a preset threshold. If the preset threshold is exceeded, it is confirmed that there is a duplicate broadcast event between the two new broadcast users.

[0022] The facial feature sets of new broadcasters who do not have duplicate broadcast events with other new broadcasters are added to the user feature database.

[0023] Optionally, implement live streaming permission constraints on the target user, including:

[0024] Send alarm messages to the target user and the candidate existing users who constitute a repeated broadcast event with it;

[0025] For the target user, and / or the live streaming rooms of candidate existing users who constitute repeated live streaming events with the target user, implement live streaming permission constraint control, including but not limited to any one or more of the following: restrict the right to use specific functions of the live streaming room, restrict the user traffic of the live streaming room, and prohibit the live streaming room from starting.

[0026] An application portal is provided to open the function of merging live streaming rooms to the target users and the candidate existing users.

[0027] Optionally, before acquiring the target face image from the target user's live video stream, the process includes:

[0028] Get a list of live stream user identifiers, and retrieve the historical video stream of the corresponding existing user based on the feature identifier of each existing user in the list;

[0029] Playback queries are performed on the historical video streams of each existing user to retrieve multiple image frames containing facial content. Facial images from each image frame of each existing user are extracted to form the corresponding image set to be added to the database.

[0030] Clustering is performed on the facial feature vectors of each image frame in the image set to be added to the database for each existing user. The largest cluster corresponding to each existing user is determined from the multiple clusters obtained by clustering, and the largest cluster is used as the facial feature set of its corresponding existing user.

[0031] The facial feature set is added to the user feature database.

[0032] Optionally, the historical video streams of each existing user are replayed and queried to retrieve multiple image frames containing facial content. Facial images from each image frame of each existing user are extracted to form a corresponding image set to be added to the database, including:

[0033] For each existing user, the historical video stream is replayed and queried. Face detection is performed on the replayed image frames to identify image frames containing face content.

[0034] Extract the corresponding face image from each image frame containing face content for each existing user;

[0035] Each existing user's face image is corrected for pose and then stored in the corresponding existing user's image set to be added to the database.

[0036] A repeat broadcasting event detection device provided for one of the purposes of this application includes:

[0037] The face image acquisition module is configured to acquire the target face image from the live video stream of the target user;

[0038] The image similarity matching module is configured to perform semantic matching between the facial feature vector of the target face image and the facial feature set of existing users in the user feature database, so as to obtain the semantic similarity of multiple facial feature vectors in the facial feature set of each existing user.

[0039] The duplicate broadcasting identification module is configured to determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and to determine whether there is a duplicate broadcasting event between the existing user exceeding the preset threshold and the target user.

[0040] An electronic device provided for one of the purposes of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the repeated broadcast event detection method described in this application.

[0041] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the described repeated broadcast event detection method, which, when invoked by a computer, performs the steps included in the method.

[0042] A computer program product provided for another purpose of this application includes a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0043] Compared with existing technologies, this application has several technical advantages, including but not limited to:

[0044] First, this application obtains a target face image from the target user's live video stream, performs semantic similarity matching between the face feature vector of the target face image and multiple face feature vectors in the face feature set of existing users, obtains the semantic similarity corresponding to each face feature vector of the existing users, and determines the existing user with the highest semantic similarity exceeding a preset threshold between the face feature vector of the target face image and the target user as having a duplicate live broadcast event, thereby realizing the identification of duplicate live broadcast events.

[0045] Secondly, when performing face feature vector matching for target users, this application uses multiple face feature vectors corresponding to multiple face images in the entire face feature set of each existing user to perform one-to-many matching. Through the face feature set, various differences in the face images of the corresponding existing users are represented separately. On this basis, it can ensure that all possible face feature vectors of each existing user are given as comprehensively as possible, thereby avoiding missed matching of the face feature vectors of target users and ensuring that the effectiveness and accuracy of the matching are improved.

[0046] Furthermore, this application achieves the identification of repeated broadcasting events by accurately matching the facial feature vectors of facial images. This enables the identification of repeated broadcasting events to be implemented at the level of real person images, thereby effectively identifying repeated broadcasting events of real people in factual behavior, so as to effectively maintain the order of live broadcasting and ensure that social fairness is effectively practiced in the online live broadcasting scenario. Attached Figure Description

[0047] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0048] Figure 1 This is an exemplary network architecture used for the live streaming service in the live streaming scenario of this application;

[0049] Figure 2 This is a flowchart illustrating one embodiment of the repeated broadcasting event detection method of this application;

[0050] Figure 3 This is a schematic diagram illustrating the specific process for identifying repeated broadcast events in an embodiment of this application;

[0051] Figure 4 This is a schematic diagram illustrating the process of constructing a facial feature set for a new broadcast user in an embodiment of this application;

[0052] Figure 5 This is a schematic diagram illustrating the process of adding the facial feature set of a new user to the user feature database in an embodiment of this application;

[0053] Figure 6This is a schematic diagram illustrating the specific process of implementing live streaming permission constraint control in the embodiments of this application;

[0054] Figure 7 This is a schematic diagram illustrating the process of initializing and constructing the user feature database in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram illustrating the process of constructing a set of images to be added to the database for existing users in an embodiment of this application;

[0056] Figure 9 This is a schematic block diagram of the repeated broadcasting event detection device of this application;

[0057] Figure 10 This is a schematic diagram of the structure of an electronic device used in this application. Detailed Implementation

[0058] Please see Figure 1 This application discloses an exemplary application scenario using a network architecture including a terminal device 80, a media server 81, and an application server 82. The terminal device 80 can run a live streaming terminal program, allowing broadcasters or viewers to use live streaming functions. For example, a broadcaster, as the target user, can upload a live video stream to the media server 81 via their terminal device 80, or the media server 81 can push the target user's live video stream to the viewer's terminal device 80 for playback. The media server 81 is primarily responsible for pushing the live video streams of each broadcaster to their respective live streaming rooms. The application server 82 can be used to deploy network live streaming services to maintain interaction between broadcasters and viewers based on the live streaming room. The computer program product implemented according to the duplicate broadcast event detection method of this application can run on the media server 81, the application server 82, or other arbitrarily extended security server. By running the computer program product, the various steps of the method are executed to realize the technical solution of this application. Thus, based on the facial image extracted from the live video stream of the target user, it is possible to identify whether there is a duplicate broadcast event between the target user and other existing users. The existing users mainly refer to the broadcasters whose facial images have been collected in advance.

[0059] Based on the above exemplary scenarios and related principle descriptions, please refer to Figure 2 In one embodiment of the repeated broadcasting event detection method of this application, the following steps are included:

[0060] Step S1100: Obtain the target face image from the target user's live video stream;

[0061] In the exemplary network live streaming application scenario of this application, the live video stream can be a live video stream generated in real time by the anchor user who is the target user during the live streaming process, which can be called from the media server, or it can be a live video stream archived by the target user after historical live streaming activities, which can be called from the relevant database.

[0062] For the live video stream, multiple image frames can be obtained by decoding it. Face detection can be performed on each image frame to determine whether there is face content in each image frame. These image frames carrying face content can be used as target image frames to extract face images.

[0063] In one embodiment, when performing face detection on each image frame, the image frame is input into a deep learning-based face detection model. The face detection model predicts candidate bounding boxes for face images within the frame, and the candidate bounding boxes are used to determine if the image is a face image. Thus, the image content corresponding to the candidate bounding boxes can be extracted from the image frame as the face image. In one embodiment, the face detection model can be implemented using a mature YOLO series model.

[0064] In a further embodiment, based on the face image extracted according to the candidate box, image segmentation can be performed on it using the U-Net series of image segmentation models, so that the face image removes background information and contains only pure face content to form the target face image.

[0065] In another embodiment, the pose can be corrected at any stage after the face image is obtained. The face key point detection model is used to detect each key point to obtain a key point information grid. Then, the grid is transformed by comparing it with a standard face grid so that the final target face image is corrected to a standard pose.

[0066] Step S1200: Perform semantic matching between the facial feature vector of the target face image and the facial feature set of existing users in the user feature database to obtain the semantic similarity of multiple facial feature vectors in the facial feature set of each existing user.

[0067] This application utilizes a pre-trained, converged image encoder to extract image features from various face images, thereby obtaining corresponding face feature vectors. The image encoder can be a deep learning network constructed using a convolutional neural network. By employing sufficient training samples and performing classification training, it is trained to convergence, enabling it to acquire the feature representation ability to extract deep semantic information from face images to obtain face feature vectors. The selection of the image encoder can be flexibly determined by those skilled in the art, including but not limited to convolutional neural networks (CNNs) and residual networks (ResNets).

[0068] This application also considers some or all of the live streamers as existing users. Multiple facial images are collected for each existing user. The method of collecting facial images can be the same as the previous step or be manually pre-defined, thereby forming a facial image set corresponding to each existing user. Then, for each facial image in the facial image set of each existing user, the image encoder is used to pre-extract its facial feature vector. All facial feature vectors of the same existing user are used to construct a facial feature set relative to that existing user. Then, the facial feature sets of each existing user are stored in a user feature library, so that each existing user's facial feature set in the user feature library can have multiple facial feature vectors.

[0069] To perform matching, after obtaining the target face image, it can be input into the image encoder for feature extraction to obtain its corresponding face feature vector. Then, using an arbitrary data distance algorithm, the data distance between the face feature vector of the target face image and multiple face feature vectors of each existing user in their face feature set is calculated and quantified as semantic similarity. Semantic similarity is used to characterize the degree of similarity between each face feature vector of the corresponding existing user and the face feature vector of the target face image, where a higher semantic similarity indicates that the two are more similar to each other, and vice versa.

[0070] The data distance algorithm can be any feasible existing algorithm, including but not limited to vector inner product algorithm, cosine similarity algorithm, Euclidean distance algorithm, Minkowski distance algorithm, Jaccard coefficient algorithm, Pearson correlation coefficient algorithm, etc., which can be flexibly selected by those skilled in the art.

[0071] Through semantic matching, it's easy to understand that each existing user's facial feature vector has a semantic similarity to the facial feature vector of the target facial image. By ranking the semantic similarity of these existing users, the highest semantic similarity can be determined, which is equivalent to determining the best matching facial image between the existing user and the target user's target facial image. Therefore, each existing user in the user feature database has its highest semantic similarity to the target user's target facial image.

[0072] As can be seen from the above process, when each existing user performs face matching with the target user, it is based on a many-to-one matching of the facial feature vectors of multiple face images with the facial feature vector of the target user's target face image. In this way, each existing user actually uses multiple face images to match with the target face image, ensuring that the basis for matching the target face image is the face images of each existing user with multiple sampling effects, covering as many face image effects of existing users as possible, which can avoid missed matching between the target user and existing users.

[0073] Step S1300: Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determine that there is a duplicate broadcasting event between the existing user exceeding the preset threshold and the target user.

[0074] If each existing user obtains a highest semantic similarity score corresponding to the target user, a preset threshold can be used to measure whether this highest semantic similarity score indicates that the facial images of the corresponding existing user and the target user are sufficiently similar. If they are sufficiently similar, then they are essentially facial images of the same real person; otherwise, they are not facial images of the same real person. The preset threshold can be an empirical threshold or an experimental threshold, which can be set as needed by those skilled in the art.

[0075] Based on the above principles, the highest semantic similarity corresponding to each existing user is compared with the preset threshold. When the highest semantic similarity exceeds the preset threshold, it can be confirmed that there is a duplicate live broadcast event between the existing user and the target user, which can trigger downstream business logic, such as actual live broadcast permission constraint control, and completion of the target user's identity verification process.

[0076] When the highest semantic similarity corresponding to the existing user does not exceed the preset threshold, it means that the facial images of the existing user and the target user are not similar. In this case, the target user can be regarded as a new broadcaster and its normal live broadcast can be maintained.

[0077] As can be seen from the above embodiments, this application has many technical advantages, including but not limited to:

[0078] First, this application obtains a target face image from the target user's live video stream, performs semantic similarity matching between the face feature vector of the target face image and multiple face feature vectors in the face feature set of existing users, obtains the semantic similarity corresponding to each face feature vector of the existing users, and determines the existing user with the highest semantic similarity exceeding a preset threshold between the face feature vector of the target face image and the target user as having a duplicate live broadcast event, thereby realizing the identification of duplicate live broadcast events.

[0079] Secondly, when performing face feature vector matching for target users, this application uses multiple face feature vectors corresponding to multiple face images in the entire face feature set of each existing user to perform one-to-many matching. Through the face feature set, various differences in the face images of the corresponding existing users are represented separately. On this basis, it can ensure that all possible face feature vectors of each existing user are given as comprehensively as possible, thereby avoiding missed matching of the face feature vectors of target users and ensuring that the effectiveness and accuracy of the matching are improved.

[0080] Furthermore, this application achieves the identification of repeated broadcasting events by accurately matching the facial feature vectors of facial images. This enables the identification of repeated broadcasting events to be implemented at the level of real person images, thereby effectively identifying repeated broadcasting events of real people in factual behavior, so as to effectively maintain the order of live broadcasting and ensure that social fairness is effectively practiced in the online live broadcasting scenario.

[0081] Based on any embodiment of this application, please refer to Figure 3 Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determine whether there is a duplicate broadcast event between the existing user exceeding the preset threshold and the target user, including:

[0082] Step S1310: Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold. When the highest semantic similarity of all existing users in the user feature library does not exceed the preset threshold, the target user is confirmed as a new broadcast user.

[0083] When determining whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, if the highest semantic similarity corresponding to all existing users in the user feature library does not exceed the preset threshold, it means that the facial feature vector of the target user's target face image failed to match in all facial feature vectors of all existing users in the user feature library, and the real face image corresponding to the target face image did not appear in the pre-collected face images of each existing user. In this case, the target user can be identified as a new broadcast user.

[0084] Step S1320: When there is a candidate existing user whose highest semantic similarity exceeds the preset threshold, determine whether the feature identifier of the candidate existing user is the same as that of the target user. If the feature identifier is the same, confirm the target user as a historical user.

[0085] When there are one or more existing users in the user feature database whose highest semantic similarity exceeds the preset threshold, these existing users can be identified as candidate existing users. In some embodiments, only one existing user with the highest semantic similarity can be identified as the only candidate existing user. The specific determination can be made as needed.

[0086] When the aforementioned candidate existing users exist, it is further determined whether the feature identifier of each candidate existing user is the same as that of the target user. The feature identifier is also known as the user identity identifier, which is usually represented by the corresponding user's UID. When the feature identifier of the candidate existing user is exactly the same as that of the target user, in this case, the target user and the candidate existing user are actually the same broadcaster user and belong to the historical user. Accordingly, the target user can be allowed to continue his broadcasting activities normally.

[0087] Step S1330: When the feature identifiers of the candidate existing users and the target user are different, it is determined that there is a duplicate live broadcast event between the target user and the candidate existing users, and live broadcast permission constraint control is implemented on the target user.

[0088] Conversely, when the feature identifiers of the candidate existing users are found to be different from those of the target user after feature identification comparison, it means that the two are different broadcasters. In this case, the candidate existing users and the target user actually constitute a duplicate broadcasting relationship. Therefore, the duplicate broadcasting event between the target user and the candidate existing users can be determined, and the live broadcasting permission constraint control can be implemented on the target user.

[0089] The above embodiments distinguish between new broadcast users and historical users based on whether the highest semantic similarity of existing users exceeds a preset threshold. By comparing the feature identifiers between existing users and new broadcast users, it further distinguishes whether there is a duplicate broadcasting relationship, realizes the identification of duplicate broadcasting events, comprehensively takes into account various real-world situations, and achieves effective maintenance of the order of online live streaming.

[0090] Based on any embodiment of this application, please refer to Figure 4 After identifying the target users as new broadcast users, this includes:

[0091] Step S2100: Obtain some image frames carrying facial content from the live video stream of the new broadcast user, and extract the facial images from each image frame to form an image set to be added to the database;

[0092] After identifying the target user as a new broadcaster, a facial feature set corresponding to the new broadcaster can be generated in the user feature database. This new broadcaster can then be used as an existing user in the user feature database, and its facial feature set can be used to perform facial matching for subsequent broadcasters.

[0093] Therefore, in accordance with the method described in step S1100 of this application, based on the live video stream generated during the live broadcast by the new broadcast user, face detection is performed on each image frame to determine multiple image frames carrying face content, and the corresponding face images are extracted from the image frames. Image segmentation, pose correction and other operations are performed on the face images, and these processed face images are used to construct the image set to be added to the database corresponding to the new broadcast user.

[0094] The image set to be added to the database can be processed individually, or it can be processed in batches together with the image sets to be added to the database of other new users, and then its corresponding facial feature vectors are added to the database.

[0095] Step S2200: Cluster the face feature vectors of each face image in the image set to be added to the database, determine the largest cluster from the multiple clusters obtained by clustering, and use the largest cluster as the face feature set of the new broadcast user.

[0096] For each face image in the image set to be added to the database for each new user, the corresponding face feature vector can be extracted by applying the image encoder as described above in this application.

[0097] Furthermore, using the facial feature vectors as sample points, any feasible clustering algorithm is applied to cluster all facial feature vectors corresponding to all facial images in the image set to be added to the database for the same new broadcaster. Multiple clusters are obtained through clustering, each containing multiple sample points. Then, the largest cluster containing the most sample points is determined as the cluster that best represents the facial information of the corresponding new broadcaster—usually the actual host in the live broadcast and should be considered the real broadcaster. Other clusters may correspond to facial images of bystanders or collaborators in the live broadcast. It is evident that the clustering algorithm here not only determines the facial feature vectors corresponding to all valid facial images that most effectively represent the facial information of the new broadcaster but also effectively eliminates facial feature vectors from other non-hosting facial images.

[0098] The clustering algorithms used in this application include, but are not limited to, k-means clustering, hierarchical clustering, SOM clustering, or FCM clustering, and can be flexibly selected by those skilled in the art.

[0099] After determining the facial feature vectors corresponding to all valid facial images in the image set to be added to the database for each new broadcaster, the facial feature vectors corresponding to all valid facial images of each new broadcaster can be used to construct the facial feature set corresponding to that new broadcaster.

[0100] Step S2300: Add the facial feature set of the newly broadcast user to the user feature database.

[0101] After determining the facial feature set corresponding to each new broadcaster, it is mapped and stored in the user feature database along with the feature identifier of the new broadcaster. The new broadcaster then becomes an existing user in the user feature database, which can be used to perform facial matching for other users.

[0102] As can be understood from the above embodiments, this application implements a self-learning mechanism for the user feature database. After identifying a new broadcaster, the facial image of the new broadcaster is collected from the live video stream of the new broadcaster. The effective facial images are determined by a clustering algorithm, and interfering facial images are eliminated. The facial feature vectors of the effective facial images are used to construct the facial feature set of the new broadcaster, which is stored in the user feature database for subsequent users to perform facial matching. This can achieve adaptive expansion of the user feature database as more new users are added, without manual intervention, and can achieve iterative upgrades of the user feature database, thereby continuously improving the recognition effect of repeated broadcasts in the online live streaming platform.

[0103] Based on any embodiment of this application, please refer to Figure 5 Adding the facial feature set of the newly broadcasting user to the user feature database includes:

[0104] Step S2310: Based on the face feature sets of all new broadcast users, calculate the semantic similarity by cross-referencing the face feature vectors in the face feature sets of every two new broadcast users.

[0105] In this embodiment, when performing the operation of adding the facial feature set of new broadcast users to the database, since a large number of users usually join the platform and start live broadcasting on the same day or at the same time, forming multiple facial feature sets of new broadcast users accumulated by the platform on a daily basis, there may also be duplicate broadcasting events among these new broadcast users. In order to avoid adding the facial feature sets of different new broadcast users who are actually the same real people to the database at the same time, it is necessary to cross-match the facial feature sets of these new broadcast users.

[0106] To this end, we obtain the facial feature sets of all new broadcast users. Then, we group each new broadcast user into pairs. For each pair of users in a group, we obtain a facial feature vector and perform similarity matching. This ensures that each facial feature vector of each new broadcast user in the same group is matched with each facial feature vector of the other new broadcast user, and the corresponding semantic similarity is calculated. This achieves cross-matching between two new broadcast users in the same group based on facial feature vectors. This matching is performed for each group, ensuring that all new broadcast users are cross-matched pairwise. Finally, we can determine the semantic similarity between any two pairs of facial feature vectors of new broadcast users, and from this, we can determine the highest semantic similarity between the two new broadcast users.

[0107] In one embodiment, to facilitate similarity calculation during matching, the facial feature set can be represented as a high-dimensional vector for each new user's facial feature vector. All facial feature vectors of the new user are then concatenated row-wise to construct a feature matrix, which constitutes the facial feature set corresponding to that new user. Therefore, when semantic similarity needs to be calculated during matching, the feature matrices of two new users in the same group are subjected to the data distance calculation described above, such as matrix multiplication, to obtain a similarity matrix. Each element in this similarity matrix stores the semantic similarity between a facial feature vector of the first new user corresponding to its row coordinate and a facial feature vector of the second new user corresponding to its column coordinate. It is easy to understand that by traversing this similarity matrix, the highest semantic similarity can be determined.

[0108] Step S2320: Compare whether the highest semantic similarity between any two new broadcast users exceeds a preset threshold. If it exceeds the preset threshold, confirm that there is a duplicate broadcast event between the two new broadcast users.

[0109] For any two new broadcasters whose highest semantic similarity has been determined, it's easy to understand that a preset threshold, as described above, can be used to determine whether the similarity of their facial images is sufficient to indicate a duplicate broadcast situation. Therefore, the highest semantic similarity between the two new broadcasters is compared to the preset threshold. If the highest semantic similarity exceeds the preset threshold, it can be confirmed that a duplicate broadcast event exists between them. In this case, the two new broadcasters are suspected of violating regulations, their facial feature sets can be excluded from the database, and a corresponding notification message can be generated and sent to the backend user for further processing. If the highest semantic similarity does not exceed the preset threshold, it indicates that there is no duplicate broadcast event between the two new broadcasters, and therefore no additional processing is required.

[0110] By comparing the highest semantic similarity with the preset threshold for any pair of new broadcast users, user pairs with repeated broadcast events and new broadcast user sets that have not participated in repeated broadcast events can be distinguished. This ensures that the facial feature set in the database itself complies with the specifications of the online live broadcast platform, thus laying the foundation for maintaining live broadcast order.

[0111] Step S2330: Add the facial feature set of new broadcast users who do not have duplicate broadcast events with other new broadcast users to the user feature database.

[0112] New broadcasters identified through the above process who have not participated in duplicate broadcasting events do not have duplicate broadcasting events with any other new broadcasters besides themselves. They are new broadcasters who comply with the network live streaming order norms. Therefore, their corresponding facial feature sets are directly added to the user feature database, making the new broadcaster an existing user in the user feature database, thus effectively expanding the user feature database.

[0113] As can be seen from the above embodiments, in the process of adding the facial feature sets of multiple new broadcast users to the user feature database, in order to investigate possible duplicate broadcasting events among new broadcast users, this application further investigates whether duplicate broadcasting events exist between pairs of new broadcast users through cross-matching of facial feature vectors. This ensures that the facial feature sets added to the user feature database are all facial feature sets of new broadcast users that conform to the regulations of the online live broadcasting order. This not only plays a role in maintaining the order of online live broadcasting, but also provides reliable basic data for subsequent identification of whether other users have duplicate broadcasting events, and realizes the automatic expansion of the facial feature sets in the user feature database.

[0114] Based on any embodiment of this application, please refer to Figure 6 To implement live streaming permission constraints and controls on the target user, including:

[0115] Step S1331: Send an alarm message to the target user and the candidate existing users that constitute the repeated broadcast event with it;

[0116] Once it is determined that a duplicate broadcast event exists between the target user and a candidate existing user, appropriate technical controls should be implemented to ensure the normal maintenance of the online live streaming order. To this end, a corresponding alarm message can be constructed to address the duplicate broadcast event, and this alarm message can be simultaneously sent to both the target user and the candidate existing user, implementing two-way notification.

[0117] To ensure the efficiency of alarm message transmission, in one embodiment, the face images corresponding to the highest semantic similarity between the target user and the candidate existing users can be obtained. The two face images are then associated with the feature identifiers of the target user and the candidate existing users, and encapsulated together in the alarm message. This allows the target user and the candidate existing users to quickly accept the opinions of the live streaming platform when they receive the alarm message, thereby reducing the invalid appeal rate of the live streaming platform and saving customer service costs.

[0118] Step S1332: Implement live streaming permission constraint control on the live streaming rooms of the target user and / or the candidate existing users who constitute repeated live streaming events with the target user, including but not limited to any one or more of the following: restricting the right to use specific functions of the live streaming room, restricting the user traffic of the live streaming room, and prohibiting the live streaming room from starting a live stream.

[0119] Furthermore, depending on the needs of implementing business logic, live streaming permission constraints can be imposed on the live streaming room of the target user, the candidate existing users who constitute repeated live streaming events with the target user, or any one of the two.

[0120] In one embodiment, live streaming permission control can be achieved by restricting the access rights to certain specific functions of the live streaming room that require constraint control. For example, the live streaming room can be restricted from publishing various live streaming advertisements or live streaming activities.

[0121] In another embodiment, live streaming permission control can be achieved by limiting user traffic to the live streaming room that needs to be restricted, for example, by stopping traffic to the live streaming room or not exposing the live streaming room to the recommendation list.

[0122] In another embodiment, live streaming permission constraint control can be achieved by restricting the live streaming permissions of the live streaming rooms that need to be constrained and controlled. For example, controlling the corresponding live streaming room to go offline immediately, prohibiting the corresponding user from continuing to broadcast, and controlling the corresponding user to leave the live streaming room.

[0123] Of course, the implementation of the live streaming permission constraint control for the corresponding live streaming room is not limited to the above various embodiments, and can be set by those skilled in the art as needed.

[0124] Step S1333: Open the application portal for the function of merging live streaming rooms to the target user and the candidate existing users.

[0125] In the actual operation of online live streaming platforms, some streamers may have to register additional accounts due to forgotten passwords or device malfunctions, triggering repeated live streaming events. However, such actions are unintentional. In response to this situation, this application can also open an application portal for merging live streaming rooms to both the target user and the candidate existing users. This allows the target user to initiate the process of merging two user identities that actually belong to them by operating the application portal, which effectively maintains the order of live streaming and provides technical relief for some special users.

[0126] As can be seen from the above embodiments, in the process of implementing live streaming permission constraints and control on target users identified as having repeated live streaming events, on the one hand, notifications can be sent to relevant users and effective technical constraints can be implemented through technical means; on the other hand, technical relief measures are also provided for some special users who do not intend to violate the rules. While effectively maintaining the order of online live streaming, it is also possible to ensure a good user experience.

[0127] Based on any embodiment of this application, please refer to Figure 7 Before acquiring the target face image from the target user's live video stream, the process includes:

[0128] Step S3100: Obtain a list of live user identifiers, and retrieve the historical video stream of the corresponding existing user based on the feature identifier of each existing user in the list.

[0129] When initializing and constructing the user feature library of this application, existing users can be identified by reading the list of live streaming user identifiers in the online live streaming platform, and then the set of facial features required for the user feature library can be determined based on each existing user.

[0130] The aforementioned list of live stream user identifiers can be retrieved from the user database of the online live streaming platform, containing the feature identifiers of each streamer user within the platform. In one embodiment, when retrieving the list of live stream user identifiers from the user database, the streamer users can be filtered. Filtering methods include, but are not limited to: deleting streamer users without live streaming records, deleting streamer users from the blacklist of the online live streaming platform, and deleting streamer users without historical video streams, thereby ensuring that the streamer users in the list of live stream user identifiers are all suitable users for constructing the user feature database, and are appropriate for the existing users in the user feature database.

[0131] After identifying existing users from the list of live stream user identifiers, the historical video streams generated by each existing user during their previous live streams can be retrieved to create corresponding facial feature sets.

[0132] Step S3200: Replay and query the historical video streams of each existing user, find multiple image frames that carry facial content, and extract the facial images from each image frame of each existing user to form the corresponding image set to be added to the database.

[0133] For each existing user's historical video stream, a playback query can be performed on the historical video stream. During the playback query, the same processing is performed as described above in this application, where facial images are obtained from the live video stream to construct the image set to be added to the database.

[0134] Specifically, during the playback query process, image frames containing facial content in the historical video streams of existing users are first identified through face detection, and the facial images are extracted from them. Based on these facial images, image segmentation can be performed to obtain clean facial images, or the facial images can be pose-corrected to make them target facial images conforming to standard facial images. Finally, the final facial images obtained after the above processing are added to the image set to be added to the database for the corresponding existing user, so that each existing user has its own corresponding image set to be added to the database.

[0135] Step S3300: Cluster the face feature vectors of each image frame in the image set to be added to the database for each existing user, determine the largest cluster corresponding to each existing user from the multiple clusters obtained by clustering, and use the largest cluster as the face feature set of its corresponding existing user.

[0136] For each face image in the image set to be added to the database for each existing user, the corresponding face feature vector can be extracted by applying the image encoder as described above in this application.

[0137] Furthermore, using the facial feature vectors as sample points, any feasible clustering algorithm is applied to cluster all facial feature vectors corresponding to all facial images in the image set to be added to the database for the same existing user. Multiple clusters are obtained through clustering, each containing multiple sample points. Then, the largest cluster containing the most sample points is determined as the cluster that best represents the facial information of the corresponding existing user's real-person anchor. This real-person anchor is usually the actual host in the live broadcast and should be considered the real-person anchor user. Other clusters may be clusters corresponding to the facial images of bystanders or collaborators in the live broadcast. It is evident that the clustering algorithm here not only determines the facial feature vectors corresponding to all valid facial images that most effectively represent the facial information of the existing user, but also effectively eliminates the facial feature vectors of other non-hosting facial images that might interfere.

[0138] The clustering algorithms used in this application include, but are not limited to, k-means clustering, hierarchical clustering, SOM clustering, or FCM clustering, and can be flexibly selected by those skilled in the art.

[0139] After determining the facial feature vectors corresponding to all valid facial images in the image set to be added to the database for each existing user, the facial feature vectors corresponding to all valid facial images for each existing user can be used to construct the facial feature set corresponding to that existing user.

[0140] Step S3400: Add the face feature set to the user feature database.

[0141] In this embodiment, the user feature library is in its initial creation state. Therefore, after creating the user feature library, the facial feature sets corresponding to each existing user are mapped and stored in the user feature library along with the feature identifiers of the existing users. This can be used to perform face matching for other users.

[0142] As can be seen from the above embodiments, in the initialization stage, based on the existing users in the live broadcast user identifier list obtained from the network live broadcast platform, the historical video streams of each existing user are replayed and queried to determine their corresponding image sets to be added to the database. Then, based on the image sets to be added to the database, interfering face images are eliminated, and multiple effective face images that can represent real people of the existing users are selected to form the corresponding face feature library, which is stored in the user feature library. The face feature sets of all existing users are prepared in batches, and the initial creation of the user feature library is completed. The process is efficient, and the face feature sets in the user feature library, based on the accurate screening and filtering of users and face feature vectors, also have higher basic data value.

[0143] Based on any embodiment of this application, please refer to Figure 8 The historical video streams of each existing user are replayed and queried to find multiple image frames containing facial content. Facial images are extracted from each image frame of each existing user to form a corresponding image set to be added to the database, including:

[0144] Step S3210: Perform a playback query on the historical video stream of each existing user, perform face detection on the playback image frames, and identify the image frames containing face content.

[0145] When constructing the corresponding image set to be added to the database for each existing user based on the historical video stream of that existing user, the historical video stream is first replayed and queried. The replay query operation can be implemented by decoding the historical video stream into image frames and storing them in the image space in memory, and then traversing each image frame in the image space in an orderly manner by timestamp.

[0146] When performing face detection on each image frame in the image space, in one embodiment, a binary classification model that can identify whether a face exists can be used for rapid identification. The image frame is input into the binary classification model to obtain the judgment result of whether it carries face content. Subsequently, only image frames carrying face content are operated on.

[0147] Step S3220: Extract the corresponding face image from each image frame containing face content of each existing user;

[0148] For each existing user's image frame containing facial content, the image frame is input into a deep learning-based face detection model. The face detection model predicts candidate bounding boxes for the facial images within the frame. Based on these candidate bounding boxes, the image content corresponding to the candidate bounding boxes is extracted from the image frame as the facial image. In one embodiment, the face detection model can be implemented using a mature Yolo series model.

[0149] In a further embodiment, the face image extracted based on the candidate bounding box can be segmented using a U-Net series image segmentation model, thereby removing background information from the face image and containing only pure face content.

[0150] Step S3230: After correcting the pose of each face image of each existing user, store it in the corresponding existing user's image set to be added to the database.

[0151] After obtaining each existing user's face image, the face key point detection model is used to detect the face key points, and the corresponding face key point information is represented as a key point information grid. Then, the face key point information is transformed by comparing it with the standard face grid, thereby realizing the posture correction of the face image, so that the final face image is corrected to a standard posture.

[0152] Finally, all the face images obtained by each existing user through the above process are added to the corresponding image set to be added to the database for that existing user, so that each existing user has its own image set to be added to the database. This set can then be used to create the face feature set corresponding to each existing user, so as to complete the construction of the user feature database.

[0153] As can be seen from the above embodiments, when constructing the image set to be added to the database for existing users, this application obtains image frames containing facial content by querying the historical video streams of existing users. By accurately extracting facial images and performing pose correction on the facial images, it ensures that the obtained facial images have a high quality standard. Multiple such facial images are constructed into the corresponding image set to be added to the database for existing users to prepare facial feature vectors. This provides as comprehensive a range of differentiated facial information of real people of existing users under multiple sampling conditions, ensuring that when performing face matching, multiple facial feature vectors of each existing user can be compared for similarity, thereby more accurately determining the degree of similarity between the target user and the existing user.

[0154] Please see Figure 9 A duplicate live streaming event detection device provided to meet one of the purposes of this application includes a face image acquisition module 1100, an image similarity matching module 1200, and a duplicate live streaming recognition module 1300. The face image acquisition module 1100 is configured to acquire a target face image from a live video stream of a target user. The image similarity matching module 1200 is configured to semantically match the face feature vector of the target face image with the face feature sets of existing users in a user feature database to obtain the semantic similarity corresponding to multiple face feature vectors within the face feature set of each existing user. The duplicate live streaming recognition module 1300 is configured to determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and to determine that there is a duplicate live streaming event between the existing user exceeding the preset threshold and the target user.

[0155] Based on any embodiment of this application, the repeated broadcast identification module 1300 includes: a new broadcast user identification unit, configured to determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and when the highest semantic similarity of all existing users in the user feature library does not exceed the preset threshold, the target user is identified as a new broadcast user; a historical user identification unit, configured to determine whether the feature identifier of the candidate existing user is the same as that of the target user when there is a candidate existing user whose highest semantic similarity exceeds the preset threshold, and when the feature identifier is the same, the target user is identified as a historical user; and a repeated broadcast confirmation unit, configured to determine that there is a repeated broadcast event between the target user and the candidate existing user when the feature identifier of the candidate existing user is different from that of the target user, and to implement live broadcast permission constraint control on the target user.

[0156] Based on any embodiment of this application, the repeated broadcast event detection device of this application includes: a new broadcast image acquisition module, configured to acquire image frames carrying facial content from the live video stream of the new broadcast user, and extract facial images from each image frame to form an image set to be added to the database; a new broadcast feature optimization module, configured to cluster based on the facial feature vectors of each facial image in the image set to be added to the database, determine the largest cluster from the multiple clusters obtained by clustering, and use the largest cluster as the facial feature set of the new broadcast user; and a new broadcast feature database entry module, configured to add the facial feature set of the new broadcast user to the user feature database.

[0157] Based on any embodiment of this application, the new broadcast feature entry module includes: a cross-matching unit, configured to calculate semantic similarity by cross-calculating the facial feature vectors in the facial feature sets of every two new broadcast users based on their respective facial feature sets; a duplicate identification unit, configured to compare whether the highest semantic similarity between every two new broadcast users exceeds a preset threshold, and if it exceeds the preset threshold, confirm that there is a duplicate broadcast event between the two new broadcast users; and a new broadcast entry unit, configured to add the facial feature sets of new broadcast users who do not have duplicate broadcast events with other new broadcast users to the user feature database.

[0158] Based on any embodiment of this application, the repeated broadcast confirmation unit includes: an alarm subunit, configured to send alarm messages to the target user and the candidate existing users that constitute a repeated broadcast event with the target user; a control subunit, configured to implement live broadcast permission constraint control on the live broadcast room of the target user and / or the candidate existing users that constitute a repeated broadcast event with the target user, including but not limited to any one or more of the following: restricting the right to use specific functions of the live broadcast room, restricting the user traffic of the live broadcast room, and prohibiting the live broadcast from starting; and a guidance subunit, configured to open the application entry for the function of merging live broadcast rooms to the target user and the candidate existing users.

[0159] Based on any embodiment of this application, the repeated broadcast event detection device of this application includes: a user list acquisition module, configured to acquire a list of live broadcast user identifiers, and call the historical video stream of the corresponding existing user according to the feature identifier of each existing user in the list; a video playback query module, configured to perform playback query on the historical video stream of each existing user, query multiple image frames carrying facial content, and extract facial images from each image frame of each existing user to form a corresponding image set to be added to the database; an existing feature optimization module, configured to perform clustering based on the facial feature vectors of each image frame in the image set to be added to the database for each existing user, determine the largest cluster corresponding to each existing user from the multiple clusters obtained by clustering, and use the largest cluster as the facial feature set of its corresponding existing user; and an existing feature database module, configured to add the facial feature set to the user feature database.

[0160] Based on any embodiment of this application, the video playback query module includes: an image query detection unit, configured to perform playback query on the historical video stream of each existing user, perform face detection on the played-out image frames, and determine the image frames containing face content; a face image extraction unit, configured to extract the corresponding face image from each image frame containing face content of each existing user; and a posture correction processing unit, configured to perform posture correction on each face image of each existing user and store it in the corresponding existing user's image set to be added to the database.

[0161] To address the aforementioned technical problems, embodiments of this application also provide an electronic device. For example... Figure 10 The diagram shows the internal structure of an electronic device. This electronic device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, the processor can implement a method for detecting repeated broadcast events. The processor provides computational and control capabilities to support the operation of the entire electronic device. The memory stores computer-readable instructions, which, when executed by the processor, can cause the processor to execute the method for detecting repeated broadcast events of this application. The network interface of the electronic device is used for communication with a terminal. Those skilled in the art will understand that… Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0162] In this embodiment, the processor is used to execute... Figure 9 The system contains the specific functions of each module and its sub-modules. The memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the repeated broadcast event detection device of this application. The server can call the server's program code and data to execute the functions of all sub-modules.

[0163] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the repeated broadcast event detection method of any embodiment of this application.

[0164] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0165] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0166] In summary, this application accurately identifies repeated broadcasting events by semantically matching the target user's facial image with multiple facial images of each existing user, effectively protecting the order of online live streaming and ensuring social fairness in the live streaming environment.

Claims

1. A method for detecting repeated broadcast events, characterized in that, Includes the following steps: Acquire the target face image from the target user's live video stream; The facial feature vector of the target face image is semantically matched with the facial feature set of existing users in the user feature database to obtain the semantic similarity of multiple facial feature vectors in the facial feature set of each existing user. Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determine whether there is a duplicate broadcasting event between the existing user exceeding the preset threshold and the target user; The step of determining whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and determining that there is a duplicate broadcast event between the existing user exceeding the preset threshold and the target user, includes: Determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold. When the highest semantic similarity of all existing users in the user feature library does not exceed the preset threshold, the target user is identified as a new broadcast user. When there is a candidate existing user whose highest semantic similarity exceeds the preset threshold, it is determined whether the feature identifier of the candidate existing user is the same as that of the target user. If the feature identifier is the same, the target user is confirmed as a historical user. When the feature identifiers of the candidate existing users and the target user are different, it is determined that there is a duplicate live broadcast event between the target user and the candidate existing users, and live broadcast permission constraint control is implemented on the target user.

2. The method for detecting repeated broadcast events according to claim 1, characterized in that, After identifying the target users as new broadcast users, the following is included: Obtain image frames containing facial content from the live video stream of the new broadcast user, and extract the facial images from each image frame to form an image set to be added to the database; Clustering is performed on the facial feature vectors of each face image in the image set to be added to the database. The largest cluster is determined from the multiple clusters obtained by clustering, and the largest cluster is used as the facial feature set of the new broadcast user. Add the facial feature set of the newly broadcasting user to the user feature database.

3. The method for detecting repeated broadcast events according to claim 2, characterized in that, Adding the facial feature set of the newly broadcasting user to the user feature database includes: Based on the facial feature sets of all new broadcast users, the semantic similarity is calculated by cross-referencing the facial feature vectors in the facial feature sets of every two new broadcast users. Compare whether the highest semantic similarity between any two new broadcast users exceeds a preset threshold. If the preset threshold is exceeded, it is confirmed that there is a duplicate broadcast event between the two new broadcast users. The facial feature sets of new broadcasters who do not have duplicate broadcast events with other new broadcasters are added to the user feature database.

4. The method for detecting repeated broadcast events according to claim 1, characterized in that, Implement live streaming permission restrictions and controls on the target user, including: Send alarm messages to the target user and the candidate existing users who constitute a repeated broadcast event with it; Implement live streaming permission constraint control on the live streaming rooms of the target user and / or the candidate existing users who constitute repeated live streaming events with the target user, including any one or more of the following: restricting the right to use specific functions of the live streaming room, restricting the user traffic of the live streaming room, and prohibiting the live streaming room from starting. An application portal is provided to open the function of merging live streaming rooms to the target users and the candidate existing users.

5. The method for detecting repeated broadcast events according to any one of claims 1 to 4, characterized in that, Before acquiring the target face image from the target user's live video stream, the process includes: Get a list of live stream user identifiers, and retrieve the historical video stream of the corresponding existing user based on the feature identifier of each existing user in the list; Playback queries are performed on the historical video streams of each existing user to retrieve multiple image frames containing facial content. Facial images from each image frame of each existing user are extracted to form the corresponding image set to be added to the database. Clustering is performed on the facial feature vectors of each image frame in the image set to be added to the database for each existing user. The largest cluster corresponding to each existing user is determined from the multiple clusters obtained by clustering, and the largest cluster is used as the facial feature set of its corresponding existing user. The facial feature set is added to the user feature database.

6. The method for detecting repeated broadcast events according to claim 5, characterized in that, The historical video streams of each existing user are replayed and queried to retrieve multiple image frames containing facial content. Facial images from each image frame of each existing user are extracted to form a corresponding image set to be added to the database, including: For each existing user, the historical video stream is replayed and queried. Face detection is performed on the replayed image frames to identify image frames containing face content. Extract the corresponding face image from each image frame containing face content for each existing user; Each existing user's face image is corrected for pose and then stored in the corresponding existing user's image set to be added to the database.

7. A device for detecting repeated broadcast events, characterized in that, include: The face image acquisition module is configured to acquire the target face image from the live video stream of the target user; The image similarity matching module is configured to perform semantic matching between the facial feature vector of the target face image and the facial feature set of existing users in the user feature database, so as to obtain the semantic similarity of multiple facial feature vectors in the facial feature set of each existing user. The duplicate broadcasting identification module is configured to determine whether the highest semantic similarity corresponding to each existing user exceeds a preset threshold, and to determine whether there is a duplicate broadcasting event between the existing user exceeding the preset threshold and the target user. The repeated broadcasting identification module includes: The new user identification unit is configured to determine whether the highest semantic similarity of each existing user exceeds a preset threshold. When the highest semantic similarity of all existing users in the user feature library does not exceed the preset threshold, the target user is identified as a new user. The historical user identification unit is configured to determine whether the feature identifier of the candidate existing user is the same as that of the target user when there is a candidate existing user whose highest semantic similarity exceeds the preset threshold. If the feature identifier is the same, the target user is identified as a historical user. The duplicate broadcast confirmation unit is configured to determine that there is a duplicate broadcast event between the target user and the candidate user when the feature identifiers of the candidate users are different, and to implement live broadcast permission constraint control on the target user.

8. The repeated broadcasting event detection device according to claim 7, characterized in that, include: The new broadcast image acquisition module is configured to acquire image frames carrying facial content from the live video stream of the new broadcast user, and extract facial images from each image frame to form an image set to be added to the database; the new broadcast feature selection module is configured to perform clustering based on the facial feature vectors of each facial image in the image set to be added to the database, determine the largest cluster from the multiple clusters obtained by clustering, and use the largest cluster as the facial feature set of the new broadcast user; the new broadcast feature database module is configured to add the facial feature set of the new broadcast user to the user feature database.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 6, which, when invoked by a computer, executes the steps included in the corresponding method.

Images