Live broadcast redisk processing method, apparatus and device, and storage medium

Abstract

The invention relates to a live broadcast redisk processing method, apparatus and device, and a storage medium. The method comprises the following steps: acquiring a live video and service data corresponding to the live video, wherein the service data comprises a plurality of service labels arranged according to a service time sequence; slicing the live video according to a set framing duration to obtain a plurality of video clips; performing feature type identification on the image frame in the video clip, and determining a feature tag of the image frame; based on the service time sequence, performing time alignment on the service data and the video clips to obtain a plurality of pieces of fusion data arranged according to the service time sequence; and performing correlation analysis on the service label and the feature label in the fused data to obtain a correlation analysis result, and obtaining live broadcast redisk data according to the correlation analysis result. According to the scheme provided by the invention, the live broadcast video can be digitalized, and the digitalized live broadcast video data and the service data are analyzed and processed to obtain the live broadcast data with a high conversion rate, so that the quantification of the live broadcast effect is realized, and the live broadcast redisk efficiency and confidence are improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a live broadcast replay processing method, device, equipment and storage medium. Background Art

[0002] With the rise of live streaming sales on the Internet, a large number of brand merchants and Internet celebrities have entered the industry. However, since the effect of live streaming content is different from thematic creations such as short videos or pictures and texts, it is difficult to quantify and track, resulting in most live streaming becoming stereotyped and routine performances, which can neither effectively interact with users nor improve the conversion rate of live streaming sales.

[0003] In related technologies, the review of live broadcast content is mainly carried out through statistical analysis of live broadcast trend data provided by the live broadcast platform and order data of the e-commerce platform, and then relying on staff to locate and analyze user interaction, anchor performance, live broadcast room atmosphere and commodity transaction status during the live broadcast time period with higher conversion rates.

[0004] However, live streaming videos are not only lengthy, but also require large file sizes. If we rely on staff to analyze live content, not only will the work be arduous and inefficient, but the analysis conclusions will also lack credibility. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related technology, the present application provides a live broadcast review processing method, device, equipment and storage medium, which can digitize the live broadcast video, and analyze and process the digitized live broadcast video data and business data to obtain live broadcast data with a high conversion rate, realize the quantification of the live broadcast effect, and improve the efficiency and confidence of the live broadcast review.

[0006] The first aspect of the present application provides a live broadcast replay processing method, comprising the following steps:

[0007] Acquire a live video and service data corresponding to the live video, wherein the service data includes a plurality of service tags arranged in service time sequence;

[0008] Slice the live video according to the set frame duration to obtain multiple video clips;

[0009] Performing feature type recognition on the image frames in the video clip to determine feature labels of the image frames;

[0010] Based on the service timing, aligning the service data and the video clip time to obtain a plurality of fused data arranged according to the service timing;

[0011] A correlation analysis is performed on the business tags and feature tags in the fused data to obtain a correlation analysis result, and live broadcast replay data is obtained based on the correlation analysis result.

[0012] As an optional embodiment, the performing feature type recognition on the image frames in the video clip to determine the feature labels of the image frames includes:

[0013] Based on at least two pre-configured feature types, a pre-established feature recognition model is used to perform feature type recognition on the image frames in the video clip to determine that the image frames in the video clip match feature labels of each feature type.

[0014] As an optional embodiment, the preconfigured at least two feature types may include a text type, an emotion type, and a behavior type.

[0015] As an optional embodiment, the time alignment of the service data and the video clip based on the service time sequence to obtain a plurality of fused data arranged according to the service time sequence includes:

[0016] Determining time unit parameters of the service data and the video clip according to the service timing and the set frame duration;

[0017] Normalizing the time unit parameters of the service data and the video clip to the same time unit;

[0018] If the normalized time unit parameter of the service data is an integer multiple a of the time unit parameter of the video clip, clustering the video clips according to a first preset number b based on the service timing, and fusing the clustered video clips with the corresponding service data to obtain fused data, wherein b is an integer multiple of a;

[0019] If the time unit parameter of the normalized business data is not an integer multiple of the time unit parameter of the video clip, a second preset number is determined based on a preset rule, and the video clips are clustered according to the second preset number based on the business timing, and the clustered video clips are fused with the corresponding business data to obtain fused data.

[0020] As an optional embodiment, performing correlation analysis on the service label and the feature label in the fused data to obtain the correlation analysis result includes:

[0021] Based on a pre-established correlation analysis model, filtering the business tags and feature tags in the fused data;

[0022] The correlation between the filtered business labels and feature labels in the fused data is analyzed and calculated to obtain the correlation coefficient of the fused data.

[0023] As an optional embodiment, obtaining the live broadcast replay data according to the correlation analysis result includes:

[0024] Determining a high correlation number in the fused data according to a correlation coefficient of the fused data;

[0025] Select a preset number of highly relevant data and locate the corresponding target time points;

[0026] Based on the target time point, the fusion data and correlation coefficient corresponding to the set review duration are obtained as high conversion rate data in the live review data.

[0027] As an optional embodiment, slicing the live video according to the set frame duration to obtain multiple video segments includes:

[0028] Acquire image data and audio data of the live video;

[0029] Slicing the image data according to a set frame duration to obtain a plurality of image segments, and slicing the audio data according to a set frequency to obtain a plurality of audio segments;

[0030] The multiple image segments and the multiple audio segments are time-aligned to obtain multiple video segments.

[0031] A second aspect of the present application provides a live broadcast replay processing device, comprising:

[0032] An acquisition module, used to acquire a live video and service data corresponding to the live video, wherein the service data includes a plurality of service tags arranged in service time sequence;

[0033] A slicing module, used for slicing the live video according to the set frame duration to obtain multiple video segments;

[0034] An identification module, used to identify the feature types of the image frames in the video clips and determine the feature labels of the image frames;

[0035] An alignment module, used for aligning the service data and the video clip time based on the service timing to obtain a plurality of fused data arranged according to the service timing;

[0036] The analysis module is used to perform correlation analysis on the business tags and feature tags in the fused data to obtain correlation analysis results, and obtain live broadcast replay data based on the correlation analysis results.

[0037] A third aspect of the present application provides an electronic device, including:

[0038] Processor; and

[0039] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.

[0040] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.

[0041] The technical solution provided by this application may have the following beneficial effects:

[0042] This application obtains live video and business data corresponding to the live video, divides the live video into frames to obtain multiple video clips, and then aligns the business data and video clips in time to obtain fused data arranged in time sequence. The fused data can quantify user interaction, anchor performance, live broadcast room atmosphere and commodity trading during the live broadcast process. Then, a correlation analysis is performed on the business tags and feature tags in the fused data to obtain the correlation analysis results. The correlation analysis results can display live broadcast clips with high conversion rates, especially live broadcast clips that are strongly correlated with business data, especially order data. Finally, based on the correlation analysis results, the live broadcast review data is obtained, so that the staff can efficiently locate the live broadcast clips with high conversion rates and their corresponding correlation data based on the live broadcast review data. The staff can use these data to adaptively adjust the live broadcast content and methods to achieve a higher live broadcast conversion rate.

[0043] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0045] Figure 1 It is a flowchart of a live broadcast replay processing method shown in an embodiment of the present application;

[0046] Figure 2 It is a structural diagram of a live broadcast replay processing device shown in an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0049] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0051] In related technologies, the review of live broadcast content is mainly carried out through statistical analysis of live broadcast trend data provided by the live broadcast platform and order data of the e-commerce platform, and then relying on staff to locate and analyze user interaction, anchor performance, live broadcast room atmosphere and commodity transaction status during the live broadcast time period with higher conversion rates.

[0052] However, live streaming videos are not only lengthy, but also require large file sizes. If we rely on staff to analyze live content, not only will the work be arduous and inefficient, but the analysis conclusions will also lack credibility.

[0053] In response to the above problems, an embodiment of the present application provides a live broadcast review processing method, which can digitize the live broadcast video, and analyze and process the digitized live broadcast video data and business data to obtain live broadcast data with a high conversion rate, realize the quantification of the live broadcast effect, and improve the efficiency and confidence of the live broadcast review.

[0054] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0055] Figure 1 It is a flow chart of the live broadcast replay processing method shown in an embodiment of the present application.

[0056] See also Figure 1 The present application provides a live broadcast replay processing method, comprising the following steps:

[0057] S1: Obtain a live video and business data corresponding to the live video, where the business data includes multiple business tags arranged in business time sequence.

[0058] In the embodiments of the present application, the live video may be any type of live video, including but not limited to consumer live video.

[0059] In this embodiment, the live video may be a live video being played in real time (which may be understood as a live video in the form of streaming segments during the live broadcast), or a live video that has been played in real time and is used for playback.

[0060] That is to say, the solution provided in the embodiment of the present application can generate live broadcast replay data not only for live broadcast videos for playback, but also for live broadcast videos being played in real time. Preferably, live broadcast replay data is generated for live broadcast videos for playback.

[0061] It should be noted that business data refers to various types of indicator data related to the content of live video collected according to the business sequence during the live video broadcast process, wherein the duration granularity of the business sequence can be adjusted according to actual business needs, such as 1 minute. The business tag corresponding to the business data is information that identifies the business data. The specific business information of the business data at a certain point in time can be quickly located through the business tag. Among them, the business tags include but are not limited to: current time, number of people online, number of people entering, number of people leaving, number of fans, sales, sales of goods, etc. Specifically, the business tag corresponding to a certain point in time can be: 22:00, 500 people online, 50 people entering, 20 people leaving, 100 fans, sales of 1,000 yuan, and sales of toothbrushes.

[0062] It should be noted that the embodiments of the present application can collect live broadcast effect trend data provided by live broadcast platforms such as Douyin, Juliang Baiying, and B station backend through crawlers. For example, data such as the number of online people, the number of people entering the venue, the number of people leaving and entering, the number of fans, etc. at a granularity of 1 minute during the live broadcast can be obtained. It can also collect and track barrage and comment data through browser plug-ins, etc., and order data can also be obtained through e-commerce platform interfaces such as JD.com and Taobao.

[0063] S2: Slice the live video according to the set frame duration to obtain multiple video segments.

[0064] In the embodiment of the present application, the live video can be sliced ​​according to the timeline of the live video and the set frame duration, for example, slicing at 3fps.

[0065] S3: Perform feature type recognition on the image frames in the video clip to determine feature labels of the image frames.

[0066] In the embodiment of the present application, a timestamp dimension can be added to identify the feature type of the image frame in the video clip, so that the feature labels corresponding to the image frame can be quickly obtained according to the timestamp dimension. Of course, the duration granularity corresponding to the timestamp dimension can be smaller than the duration granularity of the set frame duration. For example, if the frame duration is set to 3 seconds, the timestamp dimension can be accurate to 1 second.

[0067] It should be noted that the feature label of the image frame in the embodiment of the present application refers to the information that identifies the image frame. The feature information of the image frame can be quickly located through the label of the image frame to digitize the video information. Among them, the feature label of the image frame includes but is not limited to any combination of text labels, visual labels, emotional labels and behavioral labels. Among them, the features of the text label are used to describe the information obtained in text form; the features of the visual label are used to describe the video characters (including but not limited to the anchor, anchor, and guest); the features of the emotional label are used to describe the emotions of the video characters; the features of the behavioral label are used to describe the actions of the video characters.

[0068] In addition, since the image contents of two or more adjacent image frames usually do not change much, the embodiment of the present application can also adopt the method of taking image frames at intervals to identify some image frames in the video clip, which can reduce the amount of calculated data to a certain extent.

[0069] S4: Based on the service timing, the service data and the video clips are time-aligned to obtain a plurality of fused data arranged in the service timing.

[0070] Since the time parameters of business data and video clips in actual application scenarios are different, and in general, the duration granularity corresponding to the business timing of business data, such as 1 minute, is longer than the duration granularity of the frame duration of video clips, such as 3 seconds, it is necessary to align the business data and video clips in time. Moreover, the duration granularity corresponding to the business timing is difficult to adjust. Therefore, the business data and video clips can be aligned according to the duration granularity of the business timing to obtain multiple fused data arranged according to the business timing.

[0071] S5: Perform correlation analysis on the business tags and feature tags in the fused data to obtain correlation analysis results, and obtain live broadcast replay data based on the correlation analysis results.

[0072] The embodiment of the present application combines digitized live broadcast content (feature labels of image frames) with business data, and correlation analysis results can be obtained through correlation analysis. For example, live broadcast segments with high conversion rates (which can be live broadcast segments that are strongly correlated with business data, especially order data) can output the correlation data between the live broadcast content and conversion rate of such segments as conclusions and suggestions for reference by live broadcast review personnel.

[0073] The embodiment of the present application obtains live video and business data corresponding to the live video, divides the live video into frames to obtain multiple video clips, and then time-aligns and fuses the business data and video clips to obtain fused data arranged in time sequence. The fused data can quantify user interaction, anchor performance, live broadcast room atmosphere and commodity trading during the live broadcast process. Then, a correlation analysis is performed on the business tags and feature tags in the fused data to obtain a correlation analysis result. The correlation analysis result can display live broadcast clips with high conversion rates, especially live broadcast clips that are strongly correlated with business data, especially order data. Finally, based on the correlation analysis results, live broadcast review data is obtained, so that the staff can efficiently locate the live broadcast clips with high conversion rates and their corresponding correlation data based on the live broadcast review data. The staff can use these data to adaptively adjust the live broadcast content and method to achieve a higher live broadcast conversion rate.

[0074] As an optional embodiment, performing feature type recognition on image frames in the video clip to determine feature labels of the image frames includes:

[0075] Based on at least two pre-configured feature types, a pre-established feature recognition model is used to perform feature type recognition on image frames in the video clip to determine feature labels that match each feature type on the image frames in the video clip.

[0076] In the embodiment of the present application, at least two pre-configured feature types may include text type, emotion type and behavior type, and the corresponding pre-established feature recognition models may include text recognition model, emotion recognition model and behavior recognition model. The text type, emotion type and behavior type of the image frames in the video clip, for example, the 3 frames of images in the 3fps video clip, are respectively recognized by the text recognition model, emotion recognition model and behavior recognition model, and the text features, emotion features and behavior features of the 3 frames of images in each video clip are determined to realize the digitization of the live broadcast content and process.

[0077] As a preferred embodiment, the at least two pre-configured feature types may include a text type, an emotion type, and a behavior type.

[0078] In an embodiment of the present application, the text features corresponding to the text type are used to describe information that can be obtained in text form. For example, it can be text information obtained by recognizing the host's voice, it can be text information obtained by recognizing user comments and barrage, or it can be text information obtained by recognizing live video images.

[0079] The emotion features corresponding to the emotion type are used to describe the emotions of all characters appearing in the live video. For example, the features of this type may be the facial expressions, speaking speed, and intonation of the characters.

[0080] The behavior features corresponding to the behavior types are used to describe the hand movements, hand postures, and movement frequencies of all characters in the live video.

[0081] It should be noted that in the embodiments of the present application, in addition to being defined as text type, emotion type and behavior type in the above manner, feature types can be arbitrarily defined as needed as long as they can characterize the common characteristics of a class of features.

[0082] By pre-configuring multiple feature types, the embodiment of the present application can describe the characteristics of the live video from the perspective of text features, emotional features, and behavioral features. The high-conversion rate video clips obtained by combining the feature labels of the image frames with the business labels can reflect the conversion rate of the high-conversion rate video clips from various perspectives.

[0083] Specifically, the embodiment of the present application can identify the feature label of the image frame in the following manner:

[0084] 1. Identification of text features The text information can be obtained by, for example, using automatic speech recognition technology (ASR) to recognize the voice of the characters.

[0085] 2. Recognition of emotional features can be performed on image frames through face recognition models. For example, Intel's OpenVINO tool can be used to accelerate face-detection (face detection) and facenet (face recognition) based on ONNX format to recognize the faces of characters in image frames, and also recognize the emotions and expressions in faces through, for example, Emotion Recognition CNN or FER-2013. It can also be compared with the work cards of the staff in the live broadcast room at the same time to mark the basic information of the characters appearing in the image frame. Among them, any face recognition algorithm can be used for face recognition, including but not limited to recognition algorithms based on facial feature points (feature□basedrecognitional algorithms), recognition algorithms based on the entire face image (appearance□basedrecognitional algorithms), recognition algorithms based on templates (template□basedrecognitional algorithms), recognition algorithms using neural networks (recognitional algorithmsusingneuralnetwork), recognition algorithms using support vector machines (recognitional algorithmsusingSVM), etc.

[0086] 3. The recognition of behavioral features can be based on, for example, the human-pose-estimation-opencv model dataset of TensorFlow, using, for example, openCV's netform call model for recognition, to recognize the actions and postures of the characters in the image frames, and to analyze and determine the gesture interaction, posture interaction and other data between the characters and the user.

[0087] The above models are combined to form a multimodal model, which can realize multi-angle and multi-level recognition of image frames and obtain more comprehensive feature labels.

[0088] As an optional embodiment, in step S4, the service data and the video clip are time-aligned based on the service time sequence to obtain a plurality of fused data arranged in the service time sequence, including:

[0089] S41: Determine the time unit parameter of the service data according to the service sequence, and determine the time unit parameter of the video segment according to the set frame duration.

[0090] In the embodiment of the present application, the business data can be obtained from the live broadcast trend chart, the horizontal axis of the live broadcast trend chart is time, and the vertical axis corresponds to multiple business tags. The time unit parameter of the business data can refer to the time unit of the horizontal axis in the live broadcast trend chart, such as 1 minute.

[0091] In the embodiment of the present application, the time unit parameter of the video clip matches the set frame duration, for example, 3 seconds.

[0092] S42: Normalize the time unit parameters of the service data and the video clip to the same time unit.

[0093] Because the time unit parameter corresponding to the business data may be larger, such as minutes, and the frame length of the video clips may be smaller in order to meet the data accuracy, such as seconds, it is necessary to unify the time unit parameters of the business data and the video clips, for example, to unify them into seconds. Then the time unit parameter of the business data is normalized from the original 1 minute to 60 seconds to facilitate subsequent calculations.

[0094] S43: If the time unit parameter of the normalized business data is an integer multiple a of the time unit parameter of the video clip, cluster the video clips according to a first preset number b based on the business timing, and fuse the clustered video clips with the corresponding business data to obtain fused data, where b is an integer multiple of a.

[0095] In an embodiment of the present application, for example, the time unit parameter corresponding to the business data is 1 minute, the time unit parameter corresponding to the video clip is 3 seconds, and the time unit parameter of the business data after normalization is 60 seconds. Then the time unit parameter of the normalized business data is an integer multiple of the time unit parameter of the video clip, and a is 20.

[0096] In this case, based on the service time sequence, the video clips are clustered according to a first preset number b (b is an integer multiple of a, b can be 20, 40, 60...), that is, the video clips are clustered in a group of 20 / 40 / 60 or other multiples of a, and the clustered video clips are arranged according to the service time sequence and fused with the corresponding service data one by one, and multiple fused data are obtained after fusion. Preferably, b is equal to a, so that each fused data can be a data set at the minimum time granularity, which can improve the accuracy of data analysis.

[0097] Specifically, for example, the time axis of the live video is from 13:00 on May 1, 2024 to 14:00 on May 1, 2024, and the time unit parameter of the business data is 1 minute, that is, the business data includes the data at 13:0:00 on May 1, 2024 (that is, the initial data is generally 0), the data at 13:01:00 on May 1, 2024, the data at 13:01:00 on May 1, 2024, the data at 13:02:00 on May 1, 2024...the data at 13:59:00 on May 1, 2024, the ... the video clip is divided into 3fps, that is, the video clip contains the data from 13:0:00 on May 1, 2024 to 13:0:03 on May 1, 2024, the data from 13:0:03 on May 1, 2024 to 13:0:06 on May 1, 2024... the data from 13:0:57 on May 1, 2024 to 13:01:0 on May 1, 2024... the data from 13:59:57 on May 1, 2024 to 14:0:0 on May 1, 2024, etc. The business data and video clips are time-aligned to obtain fused data at intervals of 1 minute. The fused data includes the business data and video clips at 13:0:00 on May 1, 2024, the business data and video clips at 13:01:00 on May 1, 2024, the business data and video clips at 13:01:00 on May 1, 2024, the business data and video clips at 13:02:00 on May 1, 2024...the business data and video clips at 13:59:00 on May 1, 2024, and the business data and video clips at 14:0:00 on May 1, 2024.

[0098] S44: If the time unit parameter of the normalized business data is not an integer multiple of the time unit parameter of the video clip, a second preset number is determined based on a preset rule, and the video clips are clustered according to the second preset number based on the business timing, and the clustered video clips are fused with the corresponding business data to obtain fused data.

[0099] In an embodiment of the present application, for example, the time unit parameter corresponding to the business data is 1 minute, the time unit parameter corresponding to the video clip is 7 seconds, and the time unit parameter of the business data after normalization is 60 seconds. Then, the time unit parameter of the business data after normalization is not an integer multiple of the time unit parameter of the video clip, and is cd (for example, 8.5).

[0100] In this case, the second preset number is determined based on the preset rule, and the cd can be removed from zero or rounded up as the second preset number, for example, 8 or 9 is used as the second preset number, and then the video clips are clustered according to the second preset number (8 or 9) based on the business timing, that is, the video clips are clustered into groups of 8 or 9, and the clustered video clips are arranged according to the business timing and fused with the corresponding business data one by one to obtain multiple fused data.

[0101] As an optional embodiment, in step S5, correlation analysis is performed on the service label and the feature label in the fused data to obtain a correlation analysis result, including:

[0102] S51: Filter the business labels and feature labels in the fused data based on the pre-established correlation analysis model.

[0103] The correlation model in the embodiment of the present application may be a statistical model, such as the corr method of the pandas library of Python.

[0104] Specifically, for example, the fusion data at 13:01:00 on May 1, 2024 in the embodiment of the present application includes the business data at 13:01:00 on May 1, 2024, and the video clips from 13:00:00 on May 1, 2024 to 13:01:00 on May 1, 2024, for example, including 200 characters, 200 happy emotions, 10 angry emotions, sales and other labels. Since the number of samples of 10 angry emotions is too small, when performing correlation analysis, the data with too few samples will be filtered to improve the analysis efficiency and accuracy.

[0105] S52: Analyze and calculate the correlation between the filtered business labels and feature labels in the fused data to obtain the correlation coefficient of the fused data.

[0106] For example, after filtering, the fused data at 13:01:00 on May 1, 2024 contains 200 characters, 200 happy emotions, sales and other labels. The correlation model is used to analyze and calculate the correlation between the 200 characters and 200 happy emotions and sales at that time point, and express it as a correlation coefficient, such as a percentage or score.

[0107] As a preferred embodiment, in step S5, the live broadcast replay data is obtained according to the correlation analysis result, including:

[0108] S53: Determine high-correlation data in the fused data according to the correlation coefficient of the fused data.

[0109] In the embodiment of the present application, fused data with a high correlation coefficient can be used as high-correlation data. For example, data with a correlation coefficient of not less than 80% can be used as high-correlation data. The evaluation criteria for high-correlation data can also be adjusted according to actual business needs, and the present application does not limit this.

[0110] S54: Select a preset number of highly relevant data and locate the corresponding target time point.

[0111] The embodiment of the present application can directly select preset data from the high-correlation data, and can also sort the high-correlation data from high to low according to the correlation coefficient, and select a preset number of data from high to low according to the sorting, such as TOP10 data, and based on these data, the corresponding target time point, that is, the time point with high conversion rate can be located.

[0112] S55: Based on the target time point, obtain the fusion data and correlation coefficient corresponding to the set review duration as the high conversion rate data in the live review data.

[0113] Since the fused data corresponding to the target time point is not dynamic enough, it is necessary to extend the time period to the set review period based on the target time point to obtain the fused data within the target time period, and use the fused data and correlation coefficient within the target time period as high-conversion rate data in the live review data.

[0114] Specifically, for example, the target time point of the embodiment of the present application is 13:08:00 on May 1, 2024. The review duration can be set according to actual business needs. For example, it is set to 6 minutes. Then, based on 13:08:00 on May 1, 2024, the fused data of the three minutes before and three minutes after the target time point is selected as the high-conversion rate data in the live review data, that is, the fused data from 13:05:00 on May 1, 2024 to 13:11:00 on May 1, 2024.

[0115] According to the fusion data and correlation coefficients of the above selected time periods, the live broadcast can be reviewed digitally. For example, in the live broadcast video on May 1, 2024, the number of people who bought toothbrushes was the largest from 13:05:00 to 13:11:00. Among them, a certain anchor holding the product with a smile on his face was highly correlated with the sales results, and compared with the number of people interacting and commenting in the live broadcast room, the number of people online in the live broadcast room was more correlated with the sales results. The live broadcast review data given in this way makes it easier for staff to directly locate specific video clips, which helps staff to conduct purposeful and planned analysis and review. It can also reduce violations and blocked words during the live broadcast process, and can quickly improve the live broadcast conversion rate after training new employees.

[0116] As an optional embodiment, in step S2, the live video is sliced ​​according to the set frame duration to obtain multiple video segments, including:

[0117] S21: Acquire the image data and audio data of the live video.

[0118] During the live broadcast process, the terminal device of the live broadcast will continuously push the live video stream to the server; for example, during the live broadcast process, the terminal device of the live broadcast will collect image data and audio data in real time, and output video data that meets the video encoding requirements based on the collected image data and audio data, such as video data in flv format; then the terminal device of the live broadcast can use the preset encoding method and packaging method to encode and package the output video data that meets the video encoding requirements to obtain the live video stream, and upload the live video stream to the server. Accordingly, the acquired live video contains image data and audio data.

[0119] It should be noted that the embodiment of the present application can obtain the stream address of the live video through the live broadcast platform or the live broadcast analysis page, then download the video stream in flv format, and convert it into a video in mp4 format. The video in mp4 format can then be archived locally to the hard disk and archived online to cloud storage.

[0120] S22: Slice the image data according to the set frame duration to obtain a plurality of image segments, and slice the audio data according to the set frequency to obtain a plurality of audio segments.

[0121] To facilitate analysis, the embodiment of the present application slices the stored image data into 3fps image segments using a tool such as ffmpeg, and divides the audio data into segments according to a set frequency such as 16Khz and stores them independently, and marks the corresponding timing according to the timeline of the live video.

[0122] S23: Time-aligning the multiple image segments and the multiple audio segments to obtain multiple video segments.

[0123] Since the image segments and the audio segments are segmented at different time granularities, in order to merge the image segments and the audio segments into a video segment, it is necessary to align the image segments and the audio segments in time.

[0124] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a live broadcast replay processing device, an electronic device and corresponding embodiments.

[0125] Figure 2 It is a structural diagram of a live broadcast replay processing device shown in an embodiment of the present application.

[0126] See also Figure 2 The embodiment of the present application provides a live broadcast replay processing device, including an acquisition module 210, a slicing module 220, an identification module 230, an alignment module 240 and an analysis module 250:

[0127] The acquisition module 210 is used to acquire the live video and the business data corresponding to the live video, and the business data includes a plurality of business tags arranged in business time sequence.

[0128] The slicing module 220 is used to slice the live video according to the set frame duration to obtain multiple video segments. The slicing module 220 is used to obtain the image data and audio data of the live video; slice the image data according to the set frame duration to obtain multiple image segments, and slice the audio data according to the set frequency to obtain multiple audio segments; and align the multiple image segments and the multiple audio segments in time to obtain multiple video segments.

[0129] The recognition module 230 is used to perform feature type recognition on the image frames in the video clip and determine the feature labels of the image frames. The recognition module 230 is used to perform feature type recognition on the image frames in the video clip based on at least two pre-configured feature types and using a pre-established feature recognition model to determine that the image frames in the video clip match the feature labels of each feature type.

[0130] The alignment module 240 is used to align the time of the business data and the video clips based on the business timing to obtain a plurality of fused data arranged according to the business timing. The alignment module 240 is used to determine the time unit parameter of the business data according to the business timing, and determine the time unit parameter of the video clip according to the set frame duration; normalize the time unit parameters of the business data and the video clip to the same time unit; if the time unit parameter of the normalized business data is an integer multiple a of the time unit parameter of the video clip, cluster the video clips according to a first preset number b based on the business timing, and fuse the clustered video clips with the corresponding business data to obtain fused data, wherein b is an integer multiple of a; if the time unit parameter of the normalized business data is not an integer multiple of the time unit parameter of the video clip, determine a second preset number based on a preset rule, cluster the video clips according to the second preset number based on the business timing, and fuse the clustered video clips with the corresponding business data to obtain fused data.

[0131] The analysis module 250 is used to perform a correlation analysis on the business tags and feature tags in the fused data, obtain the correlation analysis results, and obtain the live broadcast review data based on the correlation analysis results. Among them, the analysis module 250 is used to filter the business tags and feature tags in the fused data based on a pre-established correlation analysis model; analyze and calculate the correlation between the filtered business tags and feature tags in the fused data to obtain the correlation coefficient of the fused data. The analysis module 250 is also used to determine the number of high correlations in the fused data based on the correlation coefficient of the fused data; select a preset number of high correlation data and locate the corresponding target time point; based on the target time point, obtain the fused data and correlation coefficient corresponding to the set review duration as the high conversion rate data in the live broadcast review data.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0133] Figure 3 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.

[0134] See also Figure 3 , the electronic device 300 includes a memory 310 and a processor 320.

[0135] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0136] The memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 320 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 310 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 310 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0137] The memory 310 stores executable codes, and when the executable codes are processed by the processor 320 , the processor 320 can execute part or all of the methods described above.

[0138] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0139] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0140] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A live broadcast replay processing method, characterized in that: The steps include: Acquire a live video and service data corresponding to the live video, wherein the service data includes a plurality of service tags arranged in service time sequence; Slice the live video according to the set frame duration to obtain multiple video clips; Performing feature type recognition on the image frames in the video clip to determine feature labels of the image frames; Based on the service timing, aligning the service data and the video clip time to obtain a plurality of fused data arranged according to the service timing; A correlation analysis is performed on the business tags and feature tags in the fused data to obtain a correlation analysis result, and live broadcast replay data is obtained based on the correlation analysis result.

2. The method according to claim 1, characterized in that The step of identifying the feature type of the image frame in the video clip and determining the feature label of the image frame includes: Based on at least two pre-configured feature types, a pre-established feature recognition model is used to perform feature type recognition on the image frames in the video clip to determine that the image frames in the video clip match feature labels of each feature type.

3. The method according to claim 2, characterized in that The preconfigured at least two feature types may include a text type, a sentiment type, and a behavior type.

4. The method according to claim 1, characterized in that The step of aligning the service data and the video clip time based on the service time sequence to obtain a plurality of fused data arranged according to the service time sequence includes: Determining time unit parameters of the service data and the video clip according to the service timing and the set frame duration; Normalizing the time unit parameters of the service data and the video clip to the same time unit; If the normalized time unit parameter of the service data is an integer multiple a of the time unit parameter of the video clip, clustering the video clips according to a first preset number b based on the service timing, and fusing the clustered video clips with the corresponding service data to obtain fused data, wherein b is an integer multiple of a; If the time unit parameter of the normalized business data is not an integer multiple of the time unit parameter of the video clip, a second preset number is determined based on a preset rule, and the video clips are clustered according to the second preset number based on the business timing, and the clustered video clips are fused with the corresponding business data to obtain fused data.

5. The method according to claim 1, characterized in that The performing correlation analysis on the service label and the feature label in the fused data to obtain the correlation analysis result includes: Based on a pre-established correlation analysis model, filtering the business tags and feature tags in the fused data; The correlation between the filtered business labels and feature labels in the fused data is analyzed and calculated to obtain the correlation coefficient of the fused data.

6. The method according to claim 5, characterized in that The live broadcast replay data is obtained according to the correlation analysis results, including: Determining a high correlation number in the fused data according to a correlation coefficient of the fused data; Select a preset number of highly relevant data and locate the corresponding target time points; Based on the target time point, the fusion data and correlation coefficient corresponding to the set review duration are obtained as high conversion rate data in the live review data.

7. The method according to claim 1, characterized in that The step of slicing the live video according to the set frame duration to obtain a plurality of video segments includes: Acquire image data and audio data of the live video; Slicing the image data according to a set frame duration to obtain a plurality of image segments, and slicing the audio data according to a set frequency to obtain a plurality of audio segments; The multiple image segments and the multiple audio segments are time-aligned to obtain multiple video segments.

8. A live broadcast replay processing device, characterized in that: include: An acquisition module, used to acquire a live video and service data corresponding to the live video, wherein the service data includes a plurality of service tags arranged in service time sequence; A slicing module, used for slicing the live video according to the set frame duration to obtain multiple video segments; An identification module, used to identify the feature types of the image frames in the video clips and determine the feature labels of the image frames; An alignment module, used for aligning the service data and the video clip time based on the service timing to obtain a plurality of fused data arranged according to the service timing; The analysis module is used to perform correlation analysis on the business tags and feature tags in the fused data to obtain correlation analysis results, and obtain live broadcast replay data based on the correlation analysis results.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.

Images