Video chat matching method, device, equipment and storage medium based on short video

Abstract

The present invention provides a video chat matching method, device, equipment and storage medium based on short videos. The method comprises the following steps: firstly receiving videos uploaded by category A users and category B users, reviewing the videos and storing the reviewed videos in a video pool; receiving a video chat matching request initiated by category A users in the video pool, obtaining multi-dimensional video features of category A users, and selecting a matching list of category B users in the video pool according to the multi-dimensional video features; playing category B user videos to category A users according to the matching list, and initiating a video chat request to a target category B user based on an operation of category A users on the category B user videos, and pushing the videos saved in the video pool by category A users; finally, establishing a video call connection between the target category B and the category A user according to the operation of the target category B user on the pushed video, thereby solving the problem that traditional information display cannot effectively convey the personality features of users, resulting in difficulty in further communication after matching.

Description

Technical Field

[0001] The present invention relates to the field of social matching, and in particular to a short video-based video chat matching method, device, equipment and storage medium. Background Art

[0002] Social networks and online communication applications have become an indispensable part of people's social life. In existing technical solutions, users mainly rely on text descriptions, static pictures and basic personal profile information to understand and screen potential matches.

[0003] However, the information display dimensions provided by existing platforms are relatively single, and it is difficult for users to accurately judge whether the matching object truly meets their expectations through limited static information; the traditional information display method is too static and rigid, and cannot effectively convey the user's dynamic characteristics such as personality traits and expression methods; due to the lack of sufficiently in-depth initial understanding of the matching object, many users often hesitate to further voice or video communication after a successful match.

[0004] In view of this, this application is filed. Summary of the invention

[0005] The present invention discloses a video chat matching method, device, equipment and storage medium based on short videos, aiming to solve the problem that traditional information display cannot effectively convey the personality characteristics of users, resulting in difficulty in further communication after matching.

[0006] A first aspect of the present invention provides a video chat matching method based on short videos, comprising:

[0007] Receive videos uploaded by category A users and category B users, review the videos and save the reviewed videos in the video pool, use a multi-task cascade convolutional neural network model to detect the face area in the video frame, and use the FaceNet model to generate a face feature vector for the detected face area; use the DEX model to analyze the face features and generate an age prediction value, map the prediction value to a predefined age group, and generate a one-hot encoding vector for the age group; use the ResNet50 model to detect the human body area from the video frame, and predict the clothing style of the human body area through multi-label classification, generate a corresponding probability value for each category, and select the top N categories with the highest probability as the clothing style vector; separate the segment containing speech from the audio signal of the video through the human voice activity detection model The method comprises the following steps: performing pitch estimation on the paragraph containing speech using the CREPE model to generate pitch features; performing fast Fourier transform on the paragraph containing speech, and calculating MFCC coefficients through Mel filter and discrete cosine transform; performing sentiment classification analysis on the MFCC coefficients, processing feature sequences using the LSTM model, and generating sentiment prediction values; identifying the paragraph containing speech using the x-vector model to generate user voiceprint features; constructing portrait features using the face feature vector, the one-hot encoding vector of the age group, and the clothing style vector, constructing voice features using the pitch features, MFCC coefficients, sentiment prediction values, and the user's voiceprint features, and adding multi-dimensional labels to the video using the portrait features and the voice features;

[0008] receiving a video chat matching request initiated by a class A user in the video pool, obtaining a multi-dimensional video feature of the class A user, and selecting a matching list of class B users in the video pool according to the multi-dimensional video feature;

[0009] Play the video of the class B user to the class A user according to the matching list, initiate a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and push the video saved by the class A user in the video pool;

[0010] According to the operation of the target class B user on pushing the video, a video call connection is established between the target class B user and the class A user.

[0011] Preferably, the category B users in the matching list are online users.

[0012] Preferably, the receiving of videos uploaded by category A users and category B users, reviewing the videos and storing the reviewed videos in a video pool is specifically as follows:

[0013] Extract key frames from the received video, and extract features from the key frames using a convolutional neural network (CNN) to generate spatial features, wherein the spatial features include visual features and semantic features, and the key frames are video frames whose image differences between adjacent video frames are greater than a preset value;

[0014] The spatial features of consecutive key frames are input into the recurrent neural network (RNN) to generate the temporal dependencies of the video content.

[0015] Separating audio information from the video using a human voice activity detection model, converting the audio information into text, and extracting audio features from the audio information;

[0016] When it is determined that the behavior pattern is normal according to the temporal dependency relationship, and when it is determined that the audio information is normal according to the text and the audio feature, the video is saved in the video pool.

[0017] Preferably, the adding of multi-dimensional labels to the video by using the portrait features and the voice features is specifically:

[0018] The portrait features and the vocal features are reduced in dimension by principal component analysis, and the reduced portrait features and vocal features are input into a multi-label neural network to generate probabilities of multiple labels, and when the probability value of a label exceeds a preset threshold, it is marked as a candidate label suitable for the current video;

[0019] The cosine similarity is used to measure the similarity between the reduced-dimensional portrait features, voice features and candidate tags to generate initial weights, and the frequency factor is calculated based on the frequency of tag occurrence and the tag weight is adjusted;

[0020] Tags with weights higher than a preset value are filtered into a final tag set, and the final tag set is added to the metadata of the video, and the updated metadata is stored in a database based on a unique identifier of the video.

[0021] Preferably, receiving a video chat matching request initiated by a class A user in the video pool, obtaining a multi-dimensional video feature of the class A user, and selecting a matching list of class B users in the video pool according to the multi-dimensional video feature, specifically comprises:

[0022] The multi-dimensional video features of the category A user are obtained according to the video chat matching request, the similarity between the preference of the category A user and the category B user's video is calculated by cosine similarity, and a matching list is generated according to the similarity value, wherein the multi-dimensional video features include user portraits and multi-dimensional labels of videos.

[0023] Preferably, after playing the category B user video to the category A user according to the matching list, it also includes: obtaining the operation behavior of the category A user on the recommended video and updating the user portrait.

[0024] A second aspect of the present invention provides a video chat matching device based on short videos, comprising:

[0025] The video review unit is used to receive videos uploaded by Category A users and Category B users, review the videos and save the reviewed videos in a video pool, use a multi-task cascade convolutional neural network model to detect the face area in the video frame, and use the FaceNet model to generate a face feature vector for the detected face area; use the DEX model to analyze the face features and generate an age prediction value, map the prediction value to a predefined age group, and generate a one-hot encoding vector for the age group; use the ResNet50 model to detect the human body area from the video frame, and predict the clothing style of the human body area through multi-label classification, generate a corresponding probability value for each category, and select the top N categories with the highest probability as the clothing style vector; separate the human voice activity from the audio signal of the video through the human voice activity detection model For a paragraph of speech, the pitch of the paragraph containing speech is estimated using the CREPE model to generate a pitch feature; the paragraph containing speech is subjected to a fast Fourier transform, and the MFCC coefficients are calculated by using a Mel filter and a discrete cosine transform; the MFCC coefficients are subjected to sentiment classification analysis, and the feature sequence is processed using an LSTM model to generate a sentiment prediction value; the paragraph containing speech is recognized using an x-vector model to generate a user's voiceprint feature; a portrait feature is constructed using the face feature vector, the one-hot encoding vector of the age group, and the clothing style vector, a voice feature is constructed using the pitch feature, the MFCC coefficient, the sentiment prediction value, and the user's voiceprint feature, and a multi-dimensional label is added to the video using the portrait feature and the voice feature;

[0026] a matching list generating unit, configured to receive a video chat matching request initiated by a class A user in the video pool, obtain a multi-dimensional video feature of the class A user, and select a matching list of class B users in the video pool according to the multi-dimensional video feature;

[0027] a video pushing unit, configured to play the video of the class B user to the class A user according to the matching list, initiate a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and push the video of the class A user saved in the video pool;

[0028] The call connection establishing unit is used to establish a video call connection between the target class B user and the class A user according to the operation of the target class B user on pushing the video.

[0029] A third aspect of the present invention provides a short video-based video chat matching device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the computer program can be executed by the processor to implement a short video-based video chat matching method as described in any one of the above.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored therein, and the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement a short video-based video chat matching method as described in any one of the above items.

[0031] A short video-based video chat matching method, apparatus, device and storage medium provided by the present invention first receives videos uploaded by category A users and category B users, reviews the videos and saves the reviewed videos in a video pool; then, receives a video chat matching request initiated by a category A user in the video pool, obtains the multi-dimensional video features of the category A user, and selects a matching list of category B users in the video pool according to the multi-dimensional video features; then, plays the category B user video to the category A user according to the matching list, and initiates a video chat request to the target category B user based on the operation of the category A user on the category B user video, and pushes the video saved by the category A user in the video pool; finally, establishes a video call connection between the target category B and the category A user according to the operation of the target category B user on the pushed video, thereby solving the problem that traditional information display cannot effectively convey the personality characteristics of the user, resulting in difficulty in further communication after matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of a video chat matching method based on short videos provided by the first aspect of the present invention;

[0033] Figure 2 It is a module schematic diagram of a video chat matching device based on short videos provided in the second aspect of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] In order to better understand the technical solution of the present invention, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0036] It should be clear that the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0038] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0039] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0040] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0041] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] The present invention discloses a video chat matching method, device, equipment and storage medium based on short videos, aiming to solve the problem that traditional information display cannot effectively convey the personality characteristics of users, resulting in difficulty in further communication after matching.

[0043] See also Figure 1In a first aspect, the present invention provides a short video-based video chat matching method, which can be executed by a short video-based video chat matching device (hereinafter referred to as a matching device), and in particular, executed by one or more processors in the matching device to at least implement the following steps:

[0044] S101, receiving videos uploaded by category A users and category B users, reviewing the videos and saving the reviewed videos in a video pool;

[0045] In this embodiment, the matching device can be a terminal with data processing capabilities such as a server, desktop computer, workstation, etc. The matching device can be installed with a corresponding operating system and application software, and the functions required by this embodiment are realized through the combination of the operating system and application software.

[0046] It should be noted that the matching device first receives the short video uploaded by the user and extracts key frames. The extraction of key frames can be to calculate the image difference between adjacent video frames. The difference is calculated by the inter-frame pixel change rate, color histogram difference or motion vector amplitude. When the difference exceeds the preset threshold, the corresponding frame is marked as a key frame. In the continuous video clips, the key frames are selected according to the preset time interval to ensure that the main content of the video can be fully expressed without processing redundant information. Then, the matching device uses the convolutional neural network (CNN) to conduct an in-depth analysis of these key frames and extract spatial features, including visual features (color distribution features, texture features, edge features) and semantic features (target category, human posture, scene information). In order to maintain the temporal coherence of the content, the matching device inputs the extracted spatial features of the continuous key frames into the recurrent neural network (RNN) to generate the temporal dependency of the video content. RNN can capture the dynamic association between video frames to identify the behavior pattern in the video. It can effectively detect abnormal behavior and provide a basis for video review.

[0047] In terms of audio processing, the matching device uses a voice activity detection (VAD) model to separate the audio information in the video, effectively distinguishing noise and background sounds from the speech segments. By converting the audio segments, the matching device converts them into text, and further extracts audio features including pitch and emotion from the audio. Through comprehensive analysis of text and audio features, the matching device can judge the rationality of the audio information on the basis of ensuring that the behavior pattern is normal, thereby screening out video content that meets the standards and saving it to the video pool.

[0048] When further optimizing the content of the video pool, the matching device will also use the multi-task cascade convolutional neural network (MTCNN) to detect the face area in the video frame, and generate feature vectors in combination with the FaceNet model to achieve accurate extraction of facial features. At the same time, the matching device uses the DEX model to predict the age of facial features, maps the age value to a predefined age group, and forms a one-hot encoding vector of the age group to ensure the structured expression of user information. Through the ResNet50 model, the matching device can identify the human body area in the video and perform multi-label classification of its clothing style to obtain multi-dimensional label information that matches the user's style.

[0049] In terms of audio features, after the matching device extracts the speech segment through the voice activity detection (VAD) model, it further uses the CREPE model to estimate the pitch of the speech and generate pitch features; then, it performs fast Fourier transform (FFT) and Mel filtering on the speech to extract Mel frequency cepstral coefficients (MFCC), and performs sentiment analysis through the LSTM model to generate sentiment prediction values. The x-vector model is used to identify the speech segment to obtain the user's voiceprint features. These steps together construct comprehensive voice features, providing key support for personalized recommendations and user matching.

[0050] Furthermore, in this embodiment, the portrait features and voice features extracted from the video are first subjected to dimensionality reduction processing to reduce the feature dimension and improve computational efficiency. By adopting the principal component analysis (PCA) method, the matching device can identify and retain the main information in the feature data while effectively reducing noise and redundant data. The reduced-dimensional features, including the user's portrait features (such as facial features, age group, clothing style) and voice features (such as pitch, timbre, emotional features, etc.), are input into a multi-label neural network for processing. The multi-label neural network model can generate multiple labels for a video at the same time and output the probability values ​​of these labels. The matching device determines the relevance of the labels based on these probability values. If the probability value of the label exceeds a preset threshold, the label will be marked as a candidate label for the video.

[0051] After generating candidate tags, the matching device further measures the similarity between the reduced-dimensional portrait features, voice features and each candidate tag through cosine similarity. This can effectively evaluate the similarity between feature vectors and tags. By calculating the angle between each pair of features and tag vectors, the matching device can generate an initial weight for each tag, and the weight value indicates the relevance of the tag to the video content.

[0052] After calculating the preliminary tag weights, the matching device will also make adjustments based on the frequency of occurrence of each tag, especially giving higher weights to tags with lower frequencies. The calculation method of the tag frequency factor ensures that the matching device can fully consider the coverage and importance of different tags, avoid the over-representation of common tags, and make the distribution of tags more fair and accurate. Finally, the matching device will filter all tags with weights greater than the preset value into the final tag set. The filtered and optimized tag set represents the main features and multi-dimensional information of the video and will be added to the metadata of the video. The content features of the video are effectively identified.

[0053] S102, receiving a video chat matching request initiated by a class A user in the video pool, obtaining a multi-dimensional video feature of the class A user, and selecting a matching list of class B users in the video pool according to the multi-dimensional video feature;

[0054] In this embodiment, first, when user A initiates a video chat matching request, the matching device receives the request from the video pool and obtains the multi-dimensional video features of user A. These features include not only the basic information of user A (such as age, gender, etc.), but also multi-dimensional tags generated based on the uploaded video. These multi-dimensional tags may include the user's visual features, sound features, emotional features, etc., so as to fully describe the interests and preferences of user A and his performance in the video.

[0055] Furthermore, the matching device can calculate the similarity between the preferences of category A users and the videos of category B users through the cosine similarity algorithm. It can accurately evaluate the degree of association between the video content of category A users and category B users. In this process, the matching device not only considers the video tag information of category A users, but also considers the personal portrait of category A users to fully understand the interests and needs of category A users.

[0056] The calculated similarity values ​​provide a quantitative basis for the matching degree of each B-user video with the A-user. Based on these similarity values, the matching device generates a matching list in which each B-user video is sorted by similarity. The matching device can accurately recommend videos that best match the interests of the A-user, thereby increasing the success rate of video matching and user engagement.

[0057] S103, playing the video of the class B user to the class A user according to the matching list, initiating a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and pushing the video saved by the class A user in the video pool;

[0058] In this embodiment, the matching device plays the video content of the class B user to the class A user according to the matching list generated in the previous step. The class B users in the matching list are sorted according to the similarity of interests and preferences with the class A users, ensuring that the class A users first receive the video recommendations that best meet their needs. During the playback process, the matching device continuously receives the operation behaviors of the class A users on the video, such as playback time, pause, slide or other interaction methods. These operation behaviors not only help the matching device better understand the feedback of the class A users on the video content, but also dynamically update the preference information of the class A users.

[0059] When user A shows interest or operation intention while watching a video of user B, the matching device initiates a video chat request to the target user B based on the operation of user A. In this way, the matching device can automatically identify and respond to the behavior of user A, thereby facilitating a more likely successful video chat match. The video content pushed to the target user B comes from the videos previously uploaded by user A and saved in the video pool.

[0060] In addition, the Class B users in the matching list must be online users during this process. This requirement ensures that Class B users can respond to the matching requests of Class A users in a timely manner, thereby avoiding matching failures or delays caused by Class B users being offline. This restriction greatly improves the real-time nature of video chat and the success rate of matching.

[0061] S104: Establishing a video call connection between the target class B user and the class A user according to the target class B user's operation on pushing the video.

[0062] In this embodiment, the matching device intelligently establishes a video call connection between the A-type user and the B-type user according to the target B-type user's operation on the pushed video. When the A-type user plays the B-type user's video according to the matching result, the matching device will receive the B-type user's interactive behavior during the video playback process in real time, such as clicks, pauses, playback duration, etc. Based on these behavioral responses, the matching device can infer the B-type user's interest in the video content and whether they are willing to further video chat with the A-type user.

[0063] If user B shows positive interaction signals when watching user A's video, such as staying on the video for a long time, or performing regular operations such as play and pause, the matching device will automatically identify that user B may be interested in user A based on these signals, and will trigger the matching device to initiate a video chat request to user B.

[0064] In a possible implementation of the present invention, after playing the category B user video to the category A user according to the matching list, it also includes: obtaining the operation behavior of the category A user on the recommended video and updating the user portrait.

[0065] In this embodiment, the matching device updates the user portrait in real time by continuously monitoring the operation behavior of Class A users on the recommended videos. When Class A users watch the recommended videos, the matching device receives a series of their interactive behaviors, including but not limited to clicks, pauses, skips, play time, video likes, comments, and any other interactive behaviors. By analyzing these operation behaviors, the matching device can deeply understand Class A users' interest preferences, viewing habits, and emotional responses to video content.

[0066] For example, if a type A user stays on a specific video for a long time, or repeatedly watches a certain video segment, this may mean that the user has a high interest in the content of the video or certain specific parts. If a type A user quickly skips a video, the matching device can infer that the content of the video does not match his or her interests, and may even update the user profile to reduce the probability of recommending similar content in the future. This feedback mechanism based on user behavior enables the matching device to continuously optimize the video recommendation and matching process, thereby improving the accuracy of recommendations and enhancing the user experience.

[0067] By continuously updating user portraits, matching devices can not only respond to the immediate needs of Class A users in the short term, but also accumulate user preference data over a long period of time to form an accurate user behavior model. For example, a user portrait can include a user's viewing preferences (such as a preference for videos of a certain type or theme), emotional preferences (such as a tendency to watch humorous and touching videos), and interaction frequency (such as whether the user prefers to watch a full video for a long time or skip quickly). This information provides a reliable basis for matching devices in the future recommendation process, ensuring that the recommended content continues to be close to the actual needs of users.

[0068] See also Figure 2 The second aspect of the present invention provides a video chat matching device based on short videos, comprising:

[0069] The video review unit 201 is used to receive videos uploaded by category A users and category B users, review the videos and save the videos that have passed the review in the video pool;

[0070] A matching list generating unit 202 is configured to receive a video chat matching request initiated by a class A user in the video pool, obtain a multi-dimensional video feature of the class A user, and select a matching list of class B users in the video pool according to the multi-dimensional video feature;

[0071] The video push unit 203 is used to play the video of the class B user to the class A user according to the matching list, initiate a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and push the video of the class A user saved in the video pool;

[0072] The call connection establishing unit 204 is used to establish a video call connection between the target class B user and the class A user according to the operation of the target class B user on pushing the video.

[0073] A third aspect of the present invention provides a short video-based video chat matching device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the computer program can be executed by the processor to implement a short video-based video chat matching method as described in any one of the above.

[0074] A fourth aspect of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored therein, and the computer program can be executed by a processor of a device where the computer-readable storage medium is located to implement a short video-based video chat matching method as described in any one of the above items.

[0075] A short video-based video chat matching method, apparatus, device and storage medium provided by the present invention first receives videos uploaded by category A users and category B users, reviews the videos and saves the reviewed videos in a video pool; then, receives a video chat matching request initiated by a category A user in the video pool, obtains the multi-dimensional video features of the category A user, and selects a matching list of category B users in the video pool according to the multi-dimensional video features; then, plays the category B user video to the category A user according to the matching list, and initiates a video chat request to the target category B user based on the operation of the category A user on the category B user video, and pushes the video saved by the category A user in the video pool; finally, establishes a video call connection between the target category B and the category A user according to the operation of the target category B user on the pushed video, thereby solving the problem that traditional information display cannot effectively convey the personality characteristics of the user, resulting in difficulty in further communication after matching.

[0076] Exemplarily, the computer program described in the third and fourth aspects of the present invention may be divided into one or more modules, and the one or more modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the device for implementing a video chat matching device based on short videos. For example, the device described in the second aspect of the present invention.

[0077] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the video chat matching method based on short videos, and uses various interfaces and lines to connect the various parts of the entire implementation of the video chat matching method based on short videos.

[0078] The memory can be used to store the computer program and / or module, and the processor realizes various functions of a video chat matching method based on short video by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, a text conversion function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, text message data, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0079] Wherein, if the implemented module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0080] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0081] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A video chat matching method based on short videos, characterized in that: include: Receive videos uploaded by users of category A and users of category B, review the videos and save the videos that have passed the review in a video pool, specifically: extract key frames from the received videos, extract features from the key frames through a convolutional neural network (CNN) to generate spatial features, wherein the spatial features include visual features and semantic features, and the key frames are video frames whose image differences between adjacent video frames are greater than a preset value; input the spatial features of consecutive key frames into a recurrent neural network (RNN) to generate a temporal dependency relationship of the video content, separate the audio information in the video using a human voice activity detection model, convert the audio information into text, and extract audio features from the audio information; when it is determined that the behavior pattern is normal according to the temporal dependency relationship, and the audio information is normal according to the text and the audio features, save the video in the video pool; A multi-task cascade convolutional neural network model is used to detect the face area in the video frame, and the FaceNet model is used to generate a face feature vector for the detected face area; the DEX model is used to analyze the face features and generate an age prediction value, the prediction value is mapped to a predefined age group, and a one-hot encoding vector of the age group is generated; the ResNet50 model is used to detect the human body area from the video frame, and the clothing style of the human body area is predicted through multi-label classification, and the corresponding probability value is generated for each category, and the top N categories with the highest probability are selected as the clothing style vector; the human voice activity detection model is used to separate the paragraph containing speech from the audio signal of the video, and the CREPE model is used to classify the paragraph containing speech. Perform pitch estimation to generate pitch features; perform fast Fourier transform on the paragraph containing speech, and calculate MFCC coefficients through Mel filter and discrete cosine transform; perform sentiment classification analysis on the MFCC coefficients, use LSTM model to process feature sequences, and generate sentiment prediction values; use x-vector model to identify the paragraph containing speech to generate user's voiceprint features; use the face feature vector, the one-hot encoding vector of the age group, and the clothing style vector to construct portrait features, use the pitch features, MFCC coefficients, sentiment prediction values, and the user's voiceprint features to construct voice features, and add multi-dimensional labels to the video through the portrait features and the voice features; receiving a video chat matching request initiated by a class A user in the video pool, obtaining a multi-dimensional video feature of the class A user, and selecting a matching list of class B users in the video pool according to the multi-dimensional video feature; Play the video of the class B user to the class A user according to the matching list, initiate a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and push the video saved by the class A user in the video pool; According to the operation of the target class B user on pushing the video, a video call connection is established between the target class B user and the class A user.

2. A video chat matching method based on short videos according to claim 1, characterized in that: The category B users in the matching list are online users.

3. A video chat matching method based on short videos according to claim 1, characterized in that: The method of adding multi-dimensional labels to the video by using the portrait features and the voice features is specifically as follows: The portrait features and the vocal features are reduced in dimension by principal component analysis, and the reduced portrait features and vocal features are input into a multi-label neural network to generate probabilities of multiple labels, and when the probability value of a label exceeds a preset threshold, it is marked as a candidate label suitable for the current video; The cosine similarity is used to measure the similarity between the reduced-dimensional portrait features, voice features and candidate tags to generate initial weights, and the frequency factor is calculated based on the frequency of tag occurrence and the tag weight is adjusted; Tags with weights higher than a preset value are filtered into a final tag set, and the final tag set is added to the metadata of the video, and the updated metadata is stored in a database based on a unique identifier of the video.

4. A video chat matching method based on short videos according to claim 1, characterized in that: The receiving of a video chat matching request initiated by a class A user in the video pool, obtaining a multi-dimensional video feature of the class A user, and selecting a matching list of class B users in the video pool according to the multi-dimensional video feature, is specifically: The multi-dimensional video features of the category A user are obtained according to the video chat matching request, the similarity between the preference of the category A user and the category B user's video is calculated by cosine similarity, and a matching list is generated according to the similarity value, wherein the multi-dimensional video features include user portraits and multi-dimensional labels of videos.

5. A video chat matching method based on short videos according to claim 4, characterized in that: After playing the category B user video to the category A user according to the matching list, it also includes: obtaining the operation behavior of the category A user on the recommended video and updating the user portrait.

6. A video chat matching device based on short videos, characterized in that: include: The video review unit is used to receive videos uploaded by category A users and category B users, review the videos and save the videos that have passed the review in a video pool, specifically: extract key frames from the received videos, extract features from the key frames through a convolutional neural network (CNN) to generate spatial features, wherein the spatial features include visual features and semantic features, and the key frames are video frames whose image differences between adjacent video frames are greater than a preset value; input the spatial features of consecutive key frames into a recurrent neural network (RNN) to generate a temporal dependency of the video content, separate the audio information in the video using a human voice activity detection model, convert the audio information into text, and extract audio features from the audio information; when it is determined that the behavior pattern is normal based on the temporal dependency, and the audio information is normal based on the text and the audio features, the video is saved in the video pool; A multi-task cascade convolutional neural network model is used to detect the face area in the video frame, and the FaceNet model is used to generate a face feature vector for the detected face area; the DEX model is used to analyze the face features and generate an age prediction value, the prediction value is mapped to a predefined age group, and a one-hot encoding vector of the age group is generated; the ResNet50 model is used to detect the human body area from the video frame, and the clothing style of the human body area is predicted through multi-label classification, and the corresponding probability value is generated for each category, and the top N categories with the highest probability are selected as the clothing style vector; the human voice activity detection model is used to separate the paragraph containing speech from the audio signal of the video, and the CREPE model is used to classify the paragraph containing speech. Perform pitch estimation to generate pitch features; perform fast Fourier transform on the paragraph containing speech, and calculate MFCC coefficients through Mel filter and discrete cosine transform; perform sentiment classification analysis on the MFCC coefficients, use LSTM model to process feature sequences, and generate sentiment prediction values; use x-vector model to identify the paragraph containing speech to generate user's voiceprint features; use the face feature vector, the one-hot encoding vector of the age group, and the clothing style vector to construct portrait features, use the pitch features, MFCC coefficients, sentiment prediction values, and the user's voiceprint features to construct voice features, and add multi-dimensional labels to the video through the portrait features and the voice features; a matching list generating unit, configured to receive a video chat matching request initiated by a class A user in the video pool, obtain a multi-dimensional video feature of the class A user, and select a matching list of class B users in the video pool according to the multi-dimensional video feature; a video pushing unit, configured to play the video of the class B user to the class A user according to the matching list, initiate a video chat request to the target class B user based on the operation of the class A user on the video of the class B user, and push the video of the class A user saved in the video pool; The call connection establishing unit is used to establish a video call connection between the target class B user and the class A user according to the operation of the target class B user on pushing the video.

7. A video chat matching device based on short videos, characterized in that: It includes a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a short video-based video chat matching method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored, and the computer program can be executed by a processor of the device where the computer-readable storage medium is located to implement a short video-based video chat matching method as described in any one of claims 1 to 5.

Images