Data processing method, device and equipment and computer readable storage medium

Abstract

The invention provides a data processing method, apparatus and device, and a computer readable storage medium. The method comprises the steps of obtaining audio data and video data corresponding to a working behavior; splitting the audio data and the video data to obtain a plurality of audio clips and a plurality of video clips; analyzing the target audio clip to obtain text data and target frequency corresponding to the target audio clip; intercepting the target video clip based on the text data corresponding to the target audio clip and the target frequency to obtain a key frame image set corresponding to the target video clip; and detecting the working behavior according to the plurality of audio clips, the text data and the target frequency corresponding to the plurality of audio clips, the plurality of video clips and the key frame image sets corresponding to the plurality of video clips, and determining a compliance result. According to the invention, the accuracy of behavior detection is improved, the normalization of working behaviors is improved, and the employee management efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, equipment and computer-readable storage medium. Background Art

[0002] Compliance refers to the state or process of an enterprise or organization's operations being consistent with applicable laws and regulations, industry standards, and internal corporate rules and regulations. Ensuring compliance is an important part of corporate management. It not only helps avoid legal risks and financial losses, but also maintains the company's reputation and promotes long-term stable development.

[0003] As the main body of the enterprise, the work behavior of employees is directly related to the compliance risk level of the enterprise. Therefore, regulating the work behavior of employees is one of the important means to reduce the compliance risk of the enterprise. At present, most of the work behaviors are detected by manually identifying the video recording of the work process to find and correct the non-compliant behavior of employees. However, manual detection requires a lot of manpower costs, and manual detection has subjective factors, and the recognition accuracy is low. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a data processing method, device, equipment and computer-readable storage medium, which improves the accuracy of behavior detection and helps to improve the standardization of work behavior. The specific technical solution is as follows:

[0005] In a first aspect, the present application provides a data processing method, the method comprising:

[0006] Obtain audio data and video data corresponding to the work behavior;

[0007] Splitting the audio data and the video data to obtain a plurality of audio segments and a plurality of video segments;

[0008] Analyze the target audio segment to obtain text data and a target frequency corresponding to the target audio segment, wherein the target frequency is the frequency with the highest frequency of occurrence in the target audio segment, and the target audio segment is any one of the multiple audio segments;

[0009] Based on the text data and the target frequency corresponding to the target audio segment, a target video segment is intercepted to obtain a set of key frame images corresponding to the target video segment, wherein the target video segment is one of the plurality of video segments, and the target audio segment corresponds to the target video segment;

[0010] The work behavior is detected based on multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, multiple video clips, and a set of key frame images corresponding to each of the multiple video clips to determine a compliance result, which characterizes the compliance degree of the work behavior.

[0011] In a possible implementation, analyzing the target audio segment to obtain text data and a target frequency corresponding to the target audio segment includes:

[0012] Performing text recognition on the target audio segment to obtain text data corresponding to the target audio segment;

[0013] Analyze the target audio segment to obtain a target waveform graph;

[0014] A target frequency corresponding to the target audio segment is determined from the target waveform.

[0015] In a possible implementation, the step of intercepting a target video segment based on the text data corresponding to the target audio segment and the target frequency to obtain a set of key frame images corresponding to the target video segment includes:

[0016] Perform keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency to obtain a target keyword of the target audio segment;

[0017] The target video segment is intercepted according to the target keyword to obtain a key frame image set corresponding to the target video segment.

[0018] In a possible implementation, performing keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency to obtain a target keyword of the target audio segment includes:

[0019] The text data corresponding to the target audio segment and the target frequency are input into a keyword recognition model to obtain the target keyword, and the keyword recognition model is trained based on a keyword sample set.

[0020] In a possible implementation, the detecting the work behavior according to the plurality of audio clips, the text data and target frequency corresponding to each of the plurality of audio clips, the plurality of video clips, and the key frame image sets corresponding to each of the plurality of video clips to determine the compliance result includes:

[0021] Detecting the work behavior according to the target audio segment, the text data and target frequency corresponding to the target audio segment, the target video segment, and the key frame image set corresponding to the target video segment, to obtain a primary compliance result;

[0022] The plurality of primary compliance results are input into a detection model to obtain the compliance result, and the detection model is trained based on a historical training set.

[0023] In a second aspect, the present application further provides a data processing device, the device comprising:

[0024] An acquisition module, used to acquire audio data and video data corresponding to the work behavior;

[0025] A splitting module, used for splitting the audio data and the video data to obtain multiple audio segments and multiple video segments;

[0026] An analysis module, configured to analyze a target audio segment to obtain text data and a target frequency corresponding to the target audio segment, wherein the target frequency is the frequency with the highest frequency of occurrence in the target audio segment, and the target audio segment is any one of the multiple audio segments;

[0027] an interception module, configured to intercept a target video segment based on the text data corresponding to the target audio segment and the target frequency, and obtain a set of key frame images corresponding to the target video segment, wherein the target video segment is one of the plurality of video segments, and the target audio segment corresponds to the target video segment;

[0028] A detection module is used to detect the work behavior based on multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, multiple video clips and the key frame image sets corresponding to each of the multiple video clips, and determine a compliance result, wherein the compliance result represents the compliance degree of the work behavior.

[0029] In a possible implementation, the analysis module includes:

[0030] A text recognition unit, used to perform text recognition on the target audio segment to obtain text data corresponding to the target audio segment;

[0031] An analysis unit, configured to analyze the target audio segment to obtain a target waveform graph;

[0032] A determination unit is used to determine a target frequency corresponding to the target audio segment from the target waveform.

[0033] In a possible implementation, the interception module includes:

[0034] A keyword recognition unit, configured to perform keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency, to obtain a target keyword of the target audio segment;

[0035] The interception unit is used to intercept the target video segment according to the target keyword to obtain a key frame image set corresponding to the target video segment.

[0036] In a third aspect, the present application further provides a computer device, including: a memory and a processor;

[0037] Wherein, the memory is used to store computer programs;

[0038] The processor is used to execute the computer program in the memory to implement the method described in the first aspect or any one of the first aspects.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any one of the first aspects.

[0040] In this application, audio data and video data corresponding to the work behavior are obtained; the audio data and video data are split to obtain multiple audio clips and multiple video clips; the target audio clip is analyzed to obtain the text data and target frequency corresponding to the target audio clip; the target video clip is intercepted based on the text data and target frequency corresponding to the target audio clip to obtain a set of key frame images corresponding to the target video clip; the work behavior is detected according to multiple audio clips, the text data and target frequencies corresponding to each of the multiple audio clips, the multiple video clips, and the key frame image sets corresponding to each of the multiple video clips, and the compliance results are determined. This application refines the analysis content by splitting the audio data and video data, which can make the analysis results more accurate, combines the audio data and video data for detection, and can determine the key information in the work behavior through the target frequency, thereby improving the accuracy of behavior detection, helping to improve the standardization of work behavior, and improving employee management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0042] Figure 1A flow chart of a data processing method provided by an embodiment of the present application is shown;

[0043] Figure 2 A flowchart of another data processing method provided by an embodiment of the present application is shown;

[0044] Figure 3 A schematic diagram of data splitting provided in an embodiment of the present application is shown;

[0045] Figure 4 A structural schematic diagram of a data processing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0047] Employee work behavior refers to the various behaviors exhibited by employees in the enterprise, including work attitude, work efficiency, teamwork and other aspects.

[0048] Compliance guides employees' work behavior. A company that focuses on compliance will formulate clear employee codes of conduct, which usually include behavioral requirements in terms of integrity, respect, cooperation, responsibility, etc., as well as prohibited behaviors such as theft, fraud, harassment, discrimination, etc. These codes provide employees with clear behavioral guidance, so that employees can clearly know which behaviors are in line with corporate requirements and which behaviors are prohibited at work. By following these codes, employees can regulate their work behavior and improve work efficiency and teamwork ability.

[0049] As the main body of the enterprise, the work behavior of employees is directly related to the compliance risk level of the enterprise. For example, sales behavior that does not comply with regulations may lead to legal disputes, customer complaints, economic losses and damage to the company's reputation.

[0050] If sales personnel mislead customers by exaggerating product effects, concealing product defects or providing false information in order to achieve sales goals, such behavior violates the principle of good faith. If sales personnel arbitrarily adjust product prices, engage in price discrimination or price fraud during the sales process, such behavior not only violates price regulations, but may also harm the interests of customers.

[0051] Therefore, the sales process needs to be strictly controlled to prevent any non-compliant behavior from occurring during the sales process.

[0052] Currently, sales behaviors are mostly detected by manually identifying video recordings of the sales process in order to discover non-compliant behaviors of sales personnel. However, manual detection requires a lot of manpower costs, and manual detection has subjective factors, resulting in low recognition accuracy.

[0053] The embodiment of the present application provides a data processing method, which can replace manual work process detection, realize automated detection of the work process, ensure the accuracy of the detection results, help to improve the standardization of employee work behavior, and improve employee management efficiency.

[0054] See also Figure 1 , shows a flow chart of a data processing method provided by an embodiment of the present application, and the embodiment of the present application at least includes the following steps:

[0055] S11, obtaining audio data and video data corresponding to the work behavior.

[0056] When it is necessary to perform normative detection on the work behavior of employees, audio data and video data corresponding to the work behavior are obtained. The work behavior can be sales behavior, management behavior, research and development behavior, etc. The work behavior is determined according to the specific occupation type. The embodiment of the present application takes sales behavior as an example for explanation.

[0057] S12, split the audio data and the video data to obtain multiple audio segments and multiple video segments.

[0058] After obtaining the audio data and the video data, the audio data and the video data may be split to obtain a plurality of audio segments and a plurality of video segments. For example, the audio data and the video data may be split according to a preset time interval, which may be set according to actual conditions. For another example, the audio data and the video data may be split evenly according to a preset number of splits, which may be set according to actual conditions.

[0059] It should be understood that in addition to the two splitting methods mentioned above, other splitting methods can be set according to actual needs, and the splitting methods are not specifically limited here.

[0060] S13, analyzing the target audio segment to obtain text data and target frequency corresponding to the target audio segment.

[0061] After obtaining multiple audio segments, the target audio segment is analyzed to obtain text data and a target frequency corresponding to the target audio segment, wherein the target audio segment is any one of the multiple audio segments, and the target frequency is the frequency with the highest frequency in the target audio segment.

[0062] It can be understood that, for any target audio segment among multiple audio segments, the target audio segment is analyzed to obtain text data and target frequency corresponding to the target audio segment, and multiple audio segments are traversed to obtain text data and target frequencies corresponding to each of the multiple audio segments.

[0063] In a possible implementation, analyzing the target audio segment to obtain text data and a target frequency corresponding to the target audio segment may include the following steps:

[0064] S131, performing text recognition on the target audio segment to obtain text data corresponding to the target audio segment.

[0065] In the embodiment of the present application, the text data corresponding to the target audio segment can be obtained by performing text recognition on the target audio segment through speech recognition technology.

[0066] S132, analyzing the target audio segment to obtain a target waveform graph.

[0067] Audio is an information carrier with regular frequency and amplitude changes of sound waves with voice, music and sound effects. Therefore, the embodiment of the present application can obtain a target waveform graph by analyzing the target audio segment. The target waveform graph is an audio waveform graph of the target audio segment.

[0068] The embodiment of the present application can sample and quantize the target audio segment, and after sampling and quantization, a series of numerical values ​​are obtained, which represent the amplitude of the audio segment at different time points. These numerical values ​​are waveform data, which are the basis for drawing a waveform graph. According to the waveform data, corresponding points or lines are drawn in the coordinate system to form a waveform graph. The waveform graph can intuitively show the change of the amplitude of the audio signal over time.

[0069] S133: Determine a target frequency corresponding to the target audio segment from the target waveform.

[0070] The frequency with the highest occurrence frequency is determined from the target waveform, and the frequency with the highest occurrence frequency is used as the target frequency.

[0071] In one possible implementation, the target frequency may be determined from the target waveform through Fourier transform.

[0072] The embodiment of the present application analyzes the audio clip to obtain text data and target frequency, which can provide a data basis for the subsequent capture of video clips and determination of compliance results.

[0073] S14, intercepting the target video segment based on the text data corresponding to the target audio segment and the target frequency, and obtaining a key frame image set corresponding to the target video segment.

[0074] After obtaining the text data and target frequency corresponding to the target audio segment, the target video segment can be intercepted to obtain a key frame image set corresponding to the target video segment, wherein the target video segment is one of multiple video segments, and there is a corresponding relationship between the target audio segment and the target video segment.

[0075] It can be understood that after the audio data and the video data are split, a correspondence between the audio segments and the video segments can be established so that the audio segments correspond to the video segments one by one.

[0076] In S13, for any target audio segment among the multiple audio segments, the target audio segment is analyzed to obtain text data and target frequency corresponding to the target audio segment, and the multiple audio segments are traversed to obtain text data and target frequency corresponding to each of the multiple audio segments. In S14, the target video segment corresponding to the target audio segment is intercepted to obtain a key frame image set corresponding to the target video segment, and by traversing the multiple audio segments and the multiple video segments, a key frame image set corresponding to the multiple video segments can be obtained.

[0077] In a possible implementation, intercepting a target video segment based on text data corresponding to a target audio segment and a target frequency to obtain a set of key frame images corresponding to the target video segment may include the following steps:

[0078] S141, performing keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency, to obtain a target keyword of the target audio segment.

[0079] The embodiment of the present application can identify the keywords in the target audio segment according to the text data corresponding to the target audio segment and the target frequency to obtain the target keyword. The target keyword can be one or more.

[0080] In a possible implementation, the text data corresponding to the target audio segment and the target frequency may be input into a keyword recognition model to obtain the target keyword. The keyword recognition model is trained based on a keyword sample set.

[0081] The training process of the keyword recognition model may include: obtaining a keyword sample set, which may include a text data sample, a frequency corresponding to the text data sample, and a keyword corresponding to the text data sample; inputting the text data sample and the frequency corresponding to the text data sample into the first model to be trained to obtain a predicted keyword; calculating a loss function based on the difference between the predicted keyword and the keyword corresponding to the text data sample, adjusting the model parameters of the first model to be trained based on the loss function, and obtaining a keyword recognition model. The first model to be trained may be a neural network model or other machine learning models.

[0082] In another possible implementation, the text data corresponding to the target audio segment and the target frequency can be input into a keyword recognition model to obtain an initial keyword set, and the initial keyword set can be input into an importance recognition model to obtain target keywords. The importance recognition model is trained based on a keyword training sample set.

[0083] The training process of the importance recognition model may include: obtaining a keyword training sample set, which may include multiple audio samples and a keyword sample set corresponding to each audio sample, and the keyword sample set may include one or more keyword samples; obtaining the proximity, frequency and value of each keyword sample in the keyword sample set, the proximity of the keyword sample represents the time interval between the time when the keyword sample appears and the cut-off collection time of the audio sample, the frequency of the keyword sample represents the total frequency of the keyword sample appearing in the target time period, and the value of the keyword sample represents the value contribution and importance of the keyword sample in the target time period; determining the importance label of each keyword sample according to the comparison results of the proximity, frequency and value of each keyword sample with the proximity mean, frequency mean and value mean of the keyword training sample set; generating a target training set according to the importance label of each keyword sample in multiple keyword sample sets and each keyword sample set; training a second model to be trained based on the target training set to obtain an importance recognition model. The second model to be trained may be a neural network model or other machine learning models.

[0084] The calculation formula of the recency (R) of a keyword sample is as follows:

[0085]

[0086] T is the cut-off time point of audio sample collection, t i is the i-th time point at which the keyword sample appears in the audio sample, and n is the total number of time points at which the keyword sample appears in the audio sample, which can also be understood as the total number of times the keyword sample appears in the audio sample.

[0087] The calculation formula of the frequency (Frequency, F) of the keyword sample is as follows:

[0088]

[0089] f i is the frequency of keyword samples appearing at the i-th time point.

[0090] The calculation formula of the monetary value (M) of keyword samples is as follows:

[0091]

[0092] m i Indicates the value or importance of the keyword sample when it appears at the i-th time point. For example, in the sales process, if the keyword is a product, the higher the sales amount of the product, the more important the keyword is.

[0093] The three indicators R, F, and M of each keyword sample are compared with the mean values ​​of the indicators of the keyword training sample set, and the difference is the indicator of the keyword sample minus the mean value of each indicator, forming 5 categories. The embodiment of the present application adopts the classification method to analyze the importance of keyword samples.

[0094] Table 1

[0095]

[0096] Table 1 shows five categories provided in the embodiment of the present application, and the importance label of each keyword sample can be determined through Table 1. In Table 1, the specific values ​​of A1, A2, B1, B2, C1, and C2 can be set according to the actual scenario.

[0097] The embodiment of the present application performs keyword recognition through a keyword recognition model and an importance recognition model, which can improve recognition accuracy and recognition efficiency.

[0098] S142, intercepting the target video segment according to the target keyword to obtain a key frame image set corresponding to the target video segment.

[0099] After determining the target keyword, the target video segment can be intercepted according to the target keyword to obtain a set of key frame images corresponding to the target video segment. For example, the target video segment can be intercepted according to the time point corresponding to the target keyword.

[0100] S15, detecting the work behavior according to the multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, the multiple video clips, and the key frame image sets corresponding to each of the multiple video clips, and determining the compliance result.

[0101] After the above steps, the text data and target frequency corresponding to each of the multiple audio clips and the key frame image set corresponding to each of the multiple video clips can be obtained, and the work behavior is detected by combining the multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, the multiple video clips and the key frame image set corresponding to each of the multiple video clips to determine the compliance result. The compliance result represents the compliance degree of the work behavior.

[0102] The embodiment of the present application can detect the work behavior according to the preset rules to determine the compliance result. The compliance result can include compliance, non-compliance and suspicion.

[0103] For example, the preset rules corresponding to sales behavior may include whether the customer himself participates in the sales process and whether the sales language is standardized.

[0104] The embodiments of the present application can predict the products currently being sold based on the target keywords, and determine whether the customer's personal participation is required based on the products currently being sold. If the customer's personal participation is required, the key frame image is identified. If the customer's personal participation is present, the illegal keywords corresponding to the products currently being sold are obtained, and the degree of standardization of the sales talk is determined based on the illegal keywords and text data. If the degree of standardization of the sales talk is greater than 80%, the compliance result is compliant; if the degree of standardization of the sales talk is greater than 60% and less than or equal to 80%, the compliance result is suspicious; if the degree of standardization of the sales talk is less than or equal to 60%, the compliance result is non-compliant; if the customer himself does not participate, the compliance result is non-compliant.

[0105] The preset rules corresponding to the sales behavior may also include video clarity and video duration. The above are only examples, and the preset rules can be set according to actual scenarios, and the embodiments of this application are not limited thereto.

[0106] In a possible implementation, determining compliance results according to a plurality of audio clips, text data and target frequencies corresponding to each of the plurality of audio clips, a plurality of video clips, and a set of key frame images corresponding to each of the plurality of video clips may include the following steps:

[0107] S151, detecting the work behavior according to the target audio segment, the text data corresponding to the target audio segment and the target frequency, the target video segment and the key frame image set corresponding to the target video segment, and obtaining a primary compliance result.

[0108] The embodiment of the present application can detect each audio clip and the video clip corresponding to each audio clip according to preset rules to obtain a primary compliance result, and traverse multiple audio clips to obtain multiple primary compliance results.

[0109] S152, inputting multiple primary compliance results into the detection model to obtain compliance results.

[0110] Multiple initial compliance results are input into the detection model for integration and analysis to obtain compliance results for audio data and video data. The detection model is trained based on historical training sets.

[0111] The training process of the detection model may include:

[0112] S101, generate an initial training set S based on historical audio data, historical video data, historical primary compliance results and historical compliance results, the number of historical audio data, historical video data, historical primary compliance results and historical compliance results are all N, N is a positive integer. Using the self-service sampling method, n groups of samples are extracted each time with replacement as a training subset, n is a positive integer. A group of samples includes historical audio data, historical video data, historical primary compliance results and historical compliance results, and there is a corresponding relationship between the historical audio data, historical video data, historical primary compliance results and historical compliance results in a group of samples. The historical training set includes an initial training set S and a test set T. Using the self-service sampling method can increase the diversity of the data set and help reduce overfitting.

[0113] S102, randomly extract d features without replacement from the data indicator system as the basis for whether each node on the decision tree is split, and generate a decision tree without pruning from top to bottom. The training subset is used as a sample of the root node of the decision tree, and training starts from the root node until all nodes of the decision tree are trained or marked as leaf nodes, and the leaf nodes are classification results. The embodiment of the present application can determine the data indicator system based on the enterprise business division indicators and the indicator dimensions. Feature randomness is introduced, that is, when each node is split, only a part of the randomly selected features are considered, rather than all features, which helps to increase the diversity between decision trees.

[0114] S103, repeat S101 and S102 N times to obtain n training subsets, namely S1, S2, ..., Sn, and generate n decision trees, namely R1, R2, ..., Rn, and combine the n decision trees to form a random forest model.

[0115] S104, divide the test set T into x sample sets Tx, input the sample set Tx into the random forest model, let the decision trees R1, R2, ..., Rn make decisions on any sample t in the sample set Tx, obtain n decision results, use the majority voting method to vote on the decision results, and finally determine the classification result of t. The classification results may include compliance, non-compliance and suspicion.

[0116] S105, repeat S104 until all samples in the test set T are classified.

[0117] In S101 to S103, the performance of the random forest model can be optimized by adjusting the number of decision trees in the random forest model and the number of features selected when each node is split. After S101 to S103, a trained random forest model can be obtained, and the trained random forest model can be evaluated by a test set. Evaluation indicators can generally include accuracy, precision, recall, F1 value, etc. If the trained random forest model passes the evaluation, the trained random forest model can be used as a detection model.

[0118] The embodiment of the present application uses the random forest model as a detection model, which can improve detection accuracy and stability.

[0119] In an embodiment of the present application, audio data and video data corresponding to the work behavior are obtained; the audio data and video data are split to obtain multiple audio clips and multiple video clips; the target audio clip is analyzed to obtain text data and target frequency corresponding to the target audio clip, the target frequency is the frequency with the highest frequency in the target audio clip, and the target audio clip is any one of the multiple audio clips; based on the text data and target frequency corresponding to the target audio clip, the target video clip is intercepted to obtain a key frame image set corresponding to the target video clip, the target video clip is one of the multiple video clips, and there is a corresponding relationship between the target audio clip and the target video clip; according to multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, the multiple video clips, and the key frame image set corresponding to each of the multiple video clips, the work behavior is detected to determine the compliance result, and the compliance result characterizes the compliance degree of the work behavior. The embodiment of the present application refines the analysis content by splitting the audio data and video data, can make the analysis result more accurate, combines the audio data and video data for detection, and can determine the key information in the work behavior through the target frequency, improves the accuracy of behavior detection, helps to improve the standardization of work behavior, and improves the efficiency of employee management.

[0120] See also Figure 2 , shows a flow chart of a data processing method provided by an embodiment of the present application, and the embodiment of the present application at least includes the following steps:

[0121] S21, obtaining a set of audio data, video data, and touch data corresponding to the work behavior.

[0122] There is a corresponding relationship between audio data, video data and touch data. The time period corresponding to the audio data is the same as the time period corresponding to the video data. The touch data set includes the touch data received by the touch screen during the time period corresponding to the audio data, and each touch data has a corresponding time point. The touch data can be used to clarify the work content of employees, such as the products sold by sales staff.

[0123] S22, split the audio data, video data and touch data set to obtain multiple audio segments, multiple video segments and multiple touch data.

[0124] In the embodiment of the present application, the audio data, video data and touch data set can be split to obtain multiple audio clips, multiple video clips and multiple touch data. For example, the audio data, video data and touch data can be split according to a preset time interval. For another example, the audio data, video data and touch data can be evenly split according to a preset number of splits.

[0125] In one possible implementation, after obtaining the audio data, video data and touch data of the sales link, the audio data is converted into an audio signal, and the audio signal is split based on a preset time interval (such as 5s) so that every 5s corresponds to an audio segment, the video data is split based on 5s so that every 5s corresponds to a video segment, and the touch data is split based on 5s so that every 5s corresponds to a touch data.

[0126] like Figure 3 As shown, the audio data can be divided into audio segment A, audio segment B, audio segment C and audio segment D, the video data can be divided into video segment E, video segment F, video segment G and video segment H, and the touch data set can be divided into touch data X, touch data Y, touch data Z and touch data W. Touch data X is a set of touch data generated in the time period corresponding to audio segment A, and may include one or more touch data; touch data Y is a set of touch data generated in the time period corresponding to audio segment B, and may include one or more touch data; touch data Z is a set of touch data generated in the time period corresponding to audio segment C, and may include one or more touch data; touch data W is a set of touch data generated in the time period corresponding to audio segment D, and may include one or more touch data.

[0127] S23, analyzing the target audio segment to obtain text data and target frequency corresponding to the target audio segment.

[0128] S24, intercepting the target video segment based on the text data corresponding to the target audio segment and the target frequency, and obtaining a key frame image set corresponding to the target video segment.

[0129] S23 to S24 can refer to Figure 1 S13 to S14 in the corresponding embodiment can be understood and will not be repeated here.

[0130] S25, detecting the work behavior according to multiple audio clips, text data and target frequencies corresponding to each of the multiple audio clips, multiple video clips, key frame image sets corresponding to each of the multiple video clips, and multiple touch data, and determining compliance results.

[0131] After the above steps, the text data and target frequencies corresponding to the multiple audio clips and the key frame image sets corresponding to the multiple video clips can be obtained. The work behavior is detected by combining the multiple audio clips, the text data and target frequencies corresponding to the multiple audio clips, the multiple video clips, the key frame image sets corresponding to the multiple video clips and the multiple touch data to determine the compliance results.

[0132] The embodiment of the present application can predict the current work content based on the text data and touch data, and determine the compliance result according to the preset rules corresponding to the current work content. For example, the current selling product can be predicted based on the text data and touch data, and the compliance result can be determined according to the preset rules corresponding to the current selling product.

[0133] In a possible implementation, determining the compliance result according to multiple audio clips, text data and target frequencies corresponding to each of the multiple audio clips, multiple video clips, key frame image sets corresponding to each of the multiple video clips, and multiple touch data may include the following steps:

[0134] S251, detecting the work behavior according to the target audio segment, the text data and target frequency corresponding to the target audio segment, the target video segment, the key frame image set corresponding to the target video segment, and the touch data corresponding to the target audio segment to obtain a primary compliance result.

[0135] The embodiment of the present application can detect each audio clip and the video clip corresponding to each audio clip according to preset rules to obtain a primary compliance result, and traverse multiple audio clips to obtain multiple primary compliance results.

[0136] S252, inputting multiple primary compliance results into the detection model to obtain compliance results.

[0137] Multiple initial compliance results are input into the detection model for integration and analysis to obtain compliance results for audio data and video data.

[0138] Combined with touch data, work behaviors can be detected in a targeted manner, narrowing the scope of preset rules and improving the efficiency and accuracy of determining compliance results.

[0139] Next, a data processing device provided in an embodiment of the present application is introduced. The data processing device introduced below and the data processing method introduced above can refer to each other.

[0140] See also Figure 4 , shows a schematic diagram of the structure of a data processing device provided in an embodiment of the present application, the device comprising:

[0141] An acquisition module 401 is used to acquire audio data and video data corresponding to the work behavior;

[0142] A splitting module 402, configured to split the audio data and the video data to obtain a plurality of audio segments and a plurality of video segments;

[0143] An analysis module 403 is used to analyze a target audio segment to obtain text data and a target frequency corresponding to the target audio segment, wherein the target frequency is the frequency with the highest frequency of occurrence in the target audio segment, and the target audio segment is any one of the multiple audio segments;

[0144] The interception module 404 is used to intercept the target video segment based on the text data corresponding to the target audio segment and the target frequency to obtain a key frame image set corresponding to the target video segment, wherein the target video segment is one of the multiple video segments, and there is a corresponding relationship between the target audio segment and the target video segment;

[0145] The detection module 405 is used to detect the work behavior based on the multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, the multiple video clips and the key frame image sets corresponding to each of the multiple video clips, and determine the compliance result, wherein the compliance result represents the compliance degree of the work behavior.

[0146] In the embodiment of the present application, the analysis module 403 includes:

[0147] A text recognition unit, used to perform text recognition on the target audio segment to obtain text data corresponding to the target audio segment;

[0148] An analysis unit, configured to analyze the target audio segment to obtain a target waveform graph;

[0149] A determination unit is used to determine a target frequency corresponding to the target audio segment from the target waveform.

[0150] In the embodiment of the present application, the interception module 404 includes:

[0151] A keyword recognition unit, configured to perform keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency, to obtain a target keyword of the target audio segment;

[0152] The interception unit is used to intercept the target video segment according to the target keyword to obtain a key frame image set corresponding to the target video segment.

[0153] In the embodiment of the present application, the keyword recognition unit is specifically used to:

[0154] The text data corresponding to the target audio segment and the target frequency are input into a keyword recognition model to obtain the target keyword, and the keyword recognition model is trained based on a keyword sample set.

[0155] In the embodiment of the present application, the detection module 405 includes:

[0156] a detection unit, configured to detect the work behavior according to the target audio segment, the text data and target frequency corresponding to the target audio segment, the target video segment, and a set of key frame images corresponding to the target video segment, to obtain a primary compliance result;

[0157] An input unit is used to input the plurality of primary compliance results into a detection model to obtain the compliance result, wherein the detection model is trained based on a historical training set.

[0158] The embodiment of the present application also provides a computer device, including: a memory and a processor;

[0159] Wherein, the memory is used to store computer programs;

[0160] The processor is used to execute the computer program in the memory to implement the method described in the above method embodiment.

[0161] The embodiment of the present application also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enables the computer to execute the method described in the above method embodiment.

[0162] In an embodiment of the present application, an acquisition module acquires audio data and video data corresponding to the work behavior; the audio data and video data are split to obtain multiple audio clips and multiple video clips; the analysis module analyzes the target audio clip to obtain text data and target frequency corresponding to the target audio clip; the interception module intercepts the target video clip based on the text data and target frequency corresponding to the target audio clip to obtain a set of key frame images corresponding to the target video clip; the detection module detects the work behavior based on multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, the multiple video clips, and the key frame image set corresponding to each of the multiple video clips, and determines the compliance result. The embodiment of the present application refines the analysis content by splitting the audio data and video data, which can make the analysis result more accurate, combines the audio data and video data for detection, and can determine the key information in the work behavior through the target frequency, thereby improving the accuracy of behavior detection, helping to improve the standardization of work behavior, and improving employee management efficiency.

[0163] It should be noted that the same or similar parts between the various embodiments can be referred to each other. As for the device embodiment and the system embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] For the above-mentioned embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0165] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0166] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0167] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain audio data and video data corresponding to the work behavior; Splitting the audio data and the video data to obtain a plurality of audio segments and a plurality of video segments; Analyze the target audio segment to obtain text data and a target frequency corresponding to the target audio segment, wherein the target frequency is the frequency with the highest frequency of occurrence in the target audio segment, and the target audio segment is any one of the multiple audio segments; Based on the text data and the target frequency corresponding to the target audio segment, a target video segment is intercepted to obtain a set of key frame images corresponding to the target video segment, wherein the target video segment is one of the plurality of video segments, and the target audio segment corresponds to the target video segment; The work behavior is detected based on multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, multiple video clips, and a set of key frame images corresponding to each of the multiple video clips to determine a compliance result, which characterizes the compliance degree of the work behavior.

2. The method according to claim 1, characterized in that The step of analyzing the target audio segment to obtain text data and a target frequency corresponding to the target audio segment includes: Performing text recognition on the target audio segment to obtain text data corresponding to the target audio segment; Analyze the target audio segment to obtain a target waveform graph; A target frequency corresponding to the target audio segment is determined from the target waveform.

3. The method according to claim 1, characterized in that The step of intercepting a target video segment based on the text data corresponding to the target audio segment and the target frequency to obtain a set of key frame images corresponding to the target video segment includes: Perform keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency to obtain a target keyword of the target audio segment; The target video segment is intercepted according to the target keyword to obtain a key frame image set corresponding to the target video segment.

4. The method according to claim 3, characterized in that: The step of performing keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency to obtain a target keyword of the target audio segment includes: The text data corresponding to the target audio segment and the target frequency are input into a keyword recognition model to obtain the target keyword, and the keyword recognition model is trained based on a keyword sample set.

5. The method according to any one of claims 1 to 4, characterized in that: The detecting the working behavior according to the plurality of audio clips, the text data and target frequency corresponding to each of the plurality of audio clips, the plurality of video clips, and the key frame image sets corresponding to each of the plurality of video clips to determine the compliance result includes: Detecting the work behavior according to the target audio segment, the text data and target frequency corresponding to the target audio segment, the target video segment, and the set of key frame images corresponding to the target video segment, to obtain a primary compliance result; The plurality of primary compliance results are input into a detection model to obtain the compliance result, and the detection model is trained based on a historical training set.

6. A data processing device, characterized in that: The device comprises: An acquisition module, used to acquire audio data and video data corresponding to the work behavior; A splitting module, used for splitting the audio data and the video data to obtain multiple audio segments and multiple video segments; An analysis module, configured to analyze a target audio segment to obtain text data and a target frequency corresponding to the target audio segment, wherein the target frequency is the frequency with the highest frequency of occurrence in the target audio segment, and the target audio segment is any one of the multiple audio segments; an interception module, configured to intercept a target video segment based on the text data corresponding to the target audio segment and the target frequency, and obtain a set of key frame images corresponding to the target video segment, wherein the target video segment is one of the plurality of video segments, and the target audio segment corresponds to the target video segment; A detection module is used to detect the work behavior based on multiple audio clips, the text data and target frequency corresponding to each of the multiple audio clips, multiple video clips and the key frame image sets corresponding to each of the multiple video clips, and determine a compliance result, wherein the compliance result represents the compliance degree of the work behavior.

7. The device according to claim 6, characterized in that The analysis module comprises: A text recognition unit, used to perform text recognition on the target audio segment to obtain text data corresponding to the target audio segment; An analysis unit, configured to analyze the target audio segment to obtain a target waveform graph; A determination unit is used to determine a target frequency corresponding to the target audio segment from the target waveform.

8. The device according to claim 6, characterized in that The interception module comprises: A keyword recognition unit, configured to perform keyword recognition on the target audio segment according to the text data corresponding to the target audio segment and the target frequency, to obtain a target keyword of the target audio segment; The interception unit is used to intercept the target video segment according to the target keyword to obtain a key frame image set corresponding to the target video segment.

9. A computer device, characterized in that: include: Memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program in the memory to implement the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The device stores instructions which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 5.

Images