Data processing method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a data processing method, device, electronic device and storage medium, the method comprising: obtaining live broadcast data and determining sub-data corresponding to each analyzer; obtaining data features of target sub-data and historical analysis results of historical target sub-data, and determining action information of the target analyzer; controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data; and being able to improve the processing efficiency of live broadcast data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a data processing device, an electronic device and a storage medium. Background Art

[0002] With the development of live streaming, more and more live streaming are appearing in front of people, such as e-commerce live streaming, online live streaming, entertainment live streaming, etc.

[0003] In order to ensure the health and compliance of live broadcast content, analyzers containing various recognition algorithms are currently used to analyze live broadcast content in order to manage content that may violate regulations. For example, a voice recognition algorithm can be used to identify whether the host has used insults, false propaganda, and other content for appropriate management. For example, an image recognition algorithm can be used to identify whether the host has made indecent movements and other content for appropriate management.

[0004] However, when analyzing and managing the live broadcast content in the above manner, the server needs to analyze a large amount of data, and the server has low efficiency in processing the live broadcast data. Summary of the invention

[0005] The embodiment of the present application provides a data processing method to improve the processing efficiency of live broadcast data.

[0006] Correspondingly, the embodiment of the present application also provides an electronic device and a storage medium to ensure the implementation and application of the above system.

[0007] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: obtaining live broadcast data and determining the sub-data corresponding to each analyzer; obtaining data characteristics of the target sub-data and historical analysis results of the historical target sub-data, and determining the action information of the target analyzer; controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data.

[0008] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: collecting target sub-data corresponding to a target analyzer in live broadcast data; determining historical analysis results of historical target sub-data based on a data detector; uploading the target sub-data and the historical analysis results to a server, so that the server can determine action information of the target analyzer based on data features of the target sub-data and the historical analysis results of the historical target sub-data, so as to control the target analyzer based on the action information and determine the analysis results of the target sub-data.

[0009] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: acquiring multimedia data and determining sub-data corresponding to each analyzer; acquiring data characteristics of target sub-data and historical analysis results of historical target sub-data, and determining action information of the target analyzer; controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data.

[0010] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: obtaining object description data of the target object, and determining the sub-data corresponding to each analyzer; obtaining data characteristics of the target sub-data and historical analysis results of the historical target sub-data, and determining the action information of the target analyzer; controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data.

[0011] In order to solve the above problems, an embodiment of the present application discloses a data processing method, which includes: obtaining live broadcast data and determining sub-data corresponding to each analyzer; obtaining data characteristics of target sub-data and historical analysis results of historical target sub-data, and determining action information of the target analyzer; controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data; determining the review results of the live broadcast data according to the analysis results of each sub-data, and processing the live broadcast data according to the review results.

[0012] In order to solve the above problems, an embodiment of the present application discloses an electronic device, characterized in that it includes: a processor; and a memory on which executable code is stored, and when the executable code is executed, the processor executes one or more methods described in the above embodiments.

[0013] In order to solve the above problems, the embodiments of the present application disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the above embodiments.

[0014] Compared with the prior art, the embodiments of the present application have the following advantages:

[0015] In an embodiment of the present application, live broadcast data and a corresponding analyzer can be obtained, and then the target sub-data corresponding to the target analyzer in the live broadcast data can be obtained, and the action information corresponding to the target analyzer can be determined based on the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data. After determining the action information corresponding to the target analyzer, the target analyzer can be controlled based on the action information to determine the analysis result corresponding to the target sub-data through the target analyzer. The embodiment of the present application can determine the action information by analyzing the data characteristics of the sub-data and the historical analysis results of the historical sub-data, so as to control the analyzer to analyze or ignore the sub-data based on the action information, thereby reducing the amount of data analyzed by the analyzer, improving the processing efficiency of the live broadcast data, and thus completing the transmission of the live broadcast data in a more timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A It is a flowchart of a data processing method according to an embodiment of the present application;

[0017] Figure 1B is a flowchart of a data processing method according to another embodiment of the present application;

[0018] Figure 2A is a flowchart of a data processing method according to another embodiment of the present application;

[0019] Figure 2B is a flowchart of a data processing method according to another embodiment of the present application;

[0020] Figure 2C is a flowchart of a data processing method according to another embodiment of the present application;

[0021] Figure 3 is a flowchart of a data processing method according to another embodiment of the present application;

[0022] Figure 4 is a flowchart of a data processing method according to another embodiment of the present application;

[0023] Figure 5 is a flowchart of a data processing method according to another embodiment of the present application;

[0024] Figure 6 is a flowchart of a data processing method according to another embodiment of the present application;

[0025] Figure 7 is a structural schematic diagram of a data processing device according to an embodiment of the present application;

[0026] Figure 8 is a structural schematic diagram of a data processing device according to another embodiment of the present application;

[0027] Fig. 9 It is a schematic diagram of the structure of an exemplary device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] The embodiments of the present application can be applied to the field of live broadcasting, which refers to publishing the collected on-site data to the Internet for viewers to watch. The live broadcasting data of the embodiments of the present application can be live broadcasting data of various scenes, such as e-commerce live broadcasting data, entertainment live broadcasting data, network live broadcasting data, education live broadcasting data, medical live broadcasting data, conference live broadcasting data, competition live broadcasting data, etc.

[0030] like Figure 1A As shown, in the embodiment of the present application, the transmission process of live data involves three end sides, namely: a first end for collecting live data, a server end for analyzing live data, and a second end for watching live data. The embodiment of the present application can obtain the live data of the scene through the first end, and upload the live data to the server end, so as to analyze the live data through the server end, and then the server end transmits the compliant live data to the second end so that users who watch the live broadcast can watch it. Among them, the server end can use a variety of analyzers to analyze the live data to determine whether the live data is compliant and the risks of non-compliant live data. For non-compliant live data, the embodiment of the present application can process it and feed it back to the second end, and feed back warning information to the first end to prompt the anchor. In addition, the embodiment of the present application can also send prompt information to the managers of the live broadcast management platform to prompt the managers to manually intervene in the management. Compliance can be understood as whether it meets the predetermined standards, and the predetermined standards can be set based on needs and relevant regulations.

[0031] Specifically, the first end is used to collect live data, which may include at least one of audio data, video data, text data, etc. In the embodiment of the present application, the first end may be a device with multiple collection functions, or may be multiple devices, each of which collects corresponding data. Figure 1BAs shown, in an embodiment of the present application, the server can configure a data detector at the first end to perform a preliminary analysis on the data collected by the first end through the data detector. Specifically, the first end may include a data acquisition component, and the data acquisition component is used to collect sub-data of the live data. For example, for an image acquisition terminal, the data acquisition component can be understood as an image acquisition component. For example, for an audio acquisition terminal, the data acquisition component can be understood as an audio acquisition component. After the data acquisition component collects the sub-data, on the one hand, it can upload the data stream of the sub-data to the server, and on the other hand, it can be analyzed through the data detector deployed at the first end to determine the analysis result, wherein the data detector can analyze the data correlation between the target sub-data and the historical target sub-data based on the data features of the target sub-data of the target frame and the historical analysis results of the historical target sub-data (the sub-data of at least one frame before the target sub-data of the target frame), so as to determine the analysis result, such as analyzing whether the features between the target sub-data and the historical target sub-data are the same, so as to determine whether the data is repeated, as the corresponding analysis result, and upload it to the server. Among them, under normal circumstances, the data computing capability of the first end is usually lower than that of the server end. Therefore, the embodiment of the present application can configure a lightweight data detector on the first end according to needs to perform simple data calculations.

[0032] In the embodiment of the present application, after the data detector deployed at the terminal analyzes the sub-data and determines the analysis result, a corresponding timestamp can be added to the analysis result so that the server can align the analysis result with the sub-data. After receiving the live broadcast data and analysis result uploaded by the first end, the server can align the analysis result with each frame of sub-data through the data alignment module according to the timestamp in the analysis result and the timestamp of each frame of sub-data, so as to perform subsequent analysis.

[0033] In the present application embodiment, Figure 1AAs shown, the server can configure multiple analyzers to perform different analyses. The analyzer is used to analyze the live data to determine the corresponding analysis results. The analyzer can be a neural network model or a rule-based analysis method, which can be configured according to the needs. The analyzer can analyze whether the live data is compliant, thereby determining the analysis results. For example, the analyzer can be divided according to the data type contained in the live data, such as the live data can include image data streams, audio data streams and character data streams. Accordingly, the analyzer can be mainly divided into three types, image analyzers corresponding to image data streams, audio analyzers corresponding to audio data streams, and text analyzers corresponding to character data streams. For the sub-data of the live data, the embodiment of the present application can further divide the sub-data, and accordingly, the analyzer can also be further divided. For example, for images, images can include face images, target images of target objects (such as commodities in e-commerce live data), etc. Accordingly, the image analyzer can be further divided into a face analyzer for analyzing face images and a target object analyzer for analyzing target images.

[0034] The embodiment of the present application can divide the live broadcast data into sub-data. Figure 1A In the example shown, this embodiment can divide the live broadcast data into image data, text data, facial images and audio data, and perform corresponding analysis through the decision model to determine the action information. Afterwards, the analyzer is controlled to analyze the sub-data according to the action information to determine the analysis result. Then, the analysis results of each sub-data are combined to determine the audit result of the live broadcast data, and corresponding processing is performed according to the audit result. In the embodiment of the present application, the audit result may include compliance and non-compliance. For compliant live broadcast data, the embodiment of the present application can transmit the live broadcast data to the second end. For non-compliant live broadcast data (there is a risk), on the one hand, a prompt (such as a warning message) can be issued to the anchor, and on the other hand, the risky content can be processed and the processed live broadcast data can be transmitted to the second end.

[0035] The following describes the analysis process of sub-data by taking the target analyzer analyzing the target sub-data in the live broadcast data as an example. The target analyzer can be understood as any of the above analyzers (such as face analyzer, target object analyzer, text analyzer, audio analyzer). Figure 1BAs shown, the server can extract features from the sub-data of the target frame through the feature processing module, thereby determining the data features of the sub-data of the target frame and inputting them into the decision model. The decision model can also obtain the historical analysis results of the historical sub-data before the target frame, thereby determining the corresponding action information, so as to control the target analyzer based on the corresponding action information. Among them, the historical analysis results can be obtained after the analyzer analyzes the historical sub-data of at least one frame before the target frame, or can be obtained after the data detector deployed at the first end performs the analysis. In this embodiment, the target analyzer can be controlled to analyze or ignore the sub-data of the target frame based on the action information, thereby reducing the amount of data analyzed by some analyzers, thereby improving the processing efficiency of the live data, so that the live data can be published to the second end in a more timely manner.

[0036] In the embodiment of the present application, the decision model can be configured as a reinforcement learning (RL) model. Reinforcement learning is also called reinforcement learning, evaluation learning or enhanced learning, which is used to describe and solve the problem of an agent learning a strategy to maximize rewards or achieve a specific goal during the interaction with the environment. Reinforcement learning is a learning method in which an agent learns by "trial and error", and the reward obtained by interacting with the environment guides the behavior, with the goal of making the agent obtain the maximum reward. In this way, the reinforcement learning model obtains knowledge from the environment and improves the action plan to adapt to the environment. The embodiments of the present application can determine action information through a decision model, and control the analyzer based on the action information to determine the analysis results, and determine the impact on the environment based on the analysis results to obtain environmental impact information. The environmental impact information may include positive impact, negative impact, etc., wherein the impact on the environment can be configured according to the scenario, such as determining the environmental impact information based on the impact on the calculation speed and the impact on the accuracy of the analysis results, and then determining the corresponding action reward information based on the environmental impact information. The action reward information may include positive rewards, negative rewards, etc., so as to continuously optimize the decision-making method of the decision model through rewards, so that the decision model can better adapt to the environment (determine more reasonable action information) and achieve the purpose of reinforcement learning.

[0037] For example, for an image analyzer, when the decision model determines that the image data is a repeated image based on the data features of the image data and the historical analysis results of the historical image data, the action information of the image analyzer can be determined to be the first image analysis action. When a live broadcast user uses repeated images (such as photos as live broadcast images) for live broadcast, the features of the image data in the live broadcast data and the features of the previous historical image data are repeated features (repeated images), then the decision model can be used to determine that the action information of the image analyzer is the first image analysis action. The first image analysis action can be an action to control the image analyzer to analyze the image data, or it can be an action to control the image analyzer to ignore the image data. This embodiment can perform reinforcement learning through a "trial and error" method to obtain more reasonable actions.

[0038] For another example, for a face analyzer, when the decision model determines that the face image is a repeated face image based on the data features of the face image and the historical analysis results of the historical face images, the action information of the face analyzer can be determined to be the first face image analysis action. For live broadcast data of the face of the same live broadcast user for a long time, the features of the face image in the live broadcast data and the features of the previous historical face images are repeated features (repeated faces), then the decision model can be used to determine that the action information of the face analyzer is the first face image analysis action, and the first face image analysis action can be an action to control the face analyzer to analyze the face image, or it can be an action to control the face analyzer to ignore the face image.

[0039] For another example, for the target object analyzer, when the decision model determines that the target image is a repeated target based on the data features of the target image and the historical analysis results of the historical target images, the action information of the target object analyzer can be determined to be the first target image analysis action. In live broadcast scenarios such as e-commerce live broadcasts, the commodities corresponding to some time periods in the live broadcast data may be the same commodity (repeated target objects), then the decision model can be used to determine that the action information of the target object analyzer is the first target image analysis action. The first target image analysis action can be an action to control the target object analyzer to analyze the target image, or it can be an action to control the target object analyzer to ignore the target image.

[0040] For another example, for a text analyzer, when the decision model determines that the text data is repeated text based on the data features of the text data and the historical analysis results of the historical text data, it can determine that the action information of the text analyzer is the first text analysis action. In a live broadcast scene, the text data can include input text (such as bullet comments) and can also include text extracted from images (such as text in live broadcast images). In live broadcast images, the host may add some fixed text in the live broadcast screen (such as Figure 1AIn the example shown, for example, "Welcome everyone to follow us"), during the live broadcast, similar fixed text usually does not change. Therefore, the decision model can be used to determine that the action information of the text analyzer is the first text analysis action. The first text analysis action can be an action to control the text analyzer to analyze the text data, or it can be an action to control the text analyzer to ignore the text data.

[0041] For another example, for an audio analyzer, when the decision model can determine that the audio data is background audio based on the data features of the audio data and the historical analysis results of the historical audio data, the action information of the audio analyzer is determined to be the first audio analysis action. In a live broadcast scenario, some anchors may only play background music for a period of time. Therefore, the audio data and historical audio data can be analyzed by the decision model. When it is determined that the audio data is background audio, the action information of the text analyzer can be determined to be the first audio analysis action. The first audio analysis action can be an action to control the audio analyzer to analyze the audio data, or it can be an action to control the audio analyzer to ignore the audio data.

[0042] The embodiment of the present application can be applied to the scene of analyzing the data to be analyzed containing multiple sub-data. The analysis method corresponding to each sub-data in the data to be analyzed (such as full analysis, partial analysis or ignoring, etc.) can be determined by decision-making, and then analyzed, which can improve the analysis efficiency of the data to be analyzed. For example, the embodiment of the present application can be applied to the scene of analyzing live data, and the analysis method (action information) corresponding to the audio data, image data (or further subdivided data) and text data in the live data can be decided, and analyzed by the analyzer, which can improve the data processing efficiency. In the live broadcast scene, after the service end receives the live broadcast data uploaded by the first end, it can be analyzed with a higher processing efficiency to determine whether the live broadcast data is compliant, and transmit the compliant live broadcast data to the second end. It should be noted that the live broadcast data in the embodiment of the present application can be the live broadcast data of each scene, such as the live broadcast data can be e-commerce live broadcast data, entertainment live broadcast data, network live broadcast data, education live broadcast data, medical live broadcast data, conference live broadcast data, road video, etc.

[0043] For another example, the embodiment of the present application can also be applied to the scene of analyzing multimedia data. The embodiment of the present application can divide the multimedia data into image data, audio data, etc., and determine the corresponding action information through the decision model, so as to control the analyzer according to the action information to obtain the analyzed multimedia data. The embodiment of the present application can analyze the pre-stored multimedia data (such as long and short videos, live video recordings, etc.) to determine the analysis results. For example, the embodiment of the present application can be specifically applied to the scene of preliminary screening of multimedia data, so as to perform efficient preliminary screening through a simple analyzer, screen out some multimedia data that need to be complex analyzed, and further perform complex analysis on the screened multimedia data. The embodiment of the present application can also be applied to the scene of copyright infringement analysis of multimedia data, so as to analyze whether the multimedia data infringes the copyright of other multimedia data, or whether other multimedia data infringes the copyright of the multimedia data.

[0044] For another example, the embodiment of the present application can also be applied to the scene of managing the object description data of a certain target object, such as the scene of analyzing the vehicle description data of the transport vehicle (which may include sub-data such as vehicle speed, driving route, driving video, driver video, etc.), and can also be applied to the scene of analyzing the item description data of the storage items (which may include various sensor data and other related data). For example, the embodiment of the present application can receive the vehicle description data uploaded by the vehicle terminal, and determine the sub-data corresponding to each analyzer (such as driver facial image, driver behavior image, vehicle driving image, and other vehicle description data), and then make a decision based on the data characteristics of each sub-data and the analysis results of the historical sub-data, determine the corresponding action information, and control the corresponding analyzer to analyze based on the action information, so as to obtain the corresponding analysis results. The embodiment of the present application can analyze the risk events such as whether the driver has non-compliant behavior, so as to manage it, such as analyzing whether the driver makes a phone call while driving, so as to manage the driver through the management personnel of the management platform. Among them, in the analysis results, the analyzer can be controlled by action information, so as to reduce the amount of data to be analyzed, improve the analysis efficiency, and improve the safety of the vehicle transportation process. For example, if the driver of a vehicle is the same driver over a period of time, the embodiment of the present application can ignore part of the analysis of the driver's facial image in the vehicle description data, thereby reducing the amount of data to be analyzed, thereby improving the analysis efficiency, obtaining the analysis results more quickly, and promptly notifying the management personnel to manage the vehicle based on the analysis results.

[0045] The present application embodiment provides a data processing method, which can be applied to a server. The server can be understood as a device that receives data to be analyzed and analyzes the data to be analyzed. This embodiment is described by analyzing live broadcast data as an example. The server can receive live broadcast data uploaded by a first end (anchor end), and after performing compliance analysis on the live broadcast data, transmit the compliant live broadcast data to a second end (viewer end). Specifically, Figure 2A As shown, the method includes:

[0046] Step 202, obtain live data, and determine the sub-data corresponding to each analyzer. The embodiment of the present application can provide a service for analyzing live data through the server, and feedback the corresponding analysis results. Correspondingly, the embodiment of the present application can obtain live data based on the data access interface through the interface, and after performing compliance analysis (or risk analysis), feedback the corresponding analysis results through the interface issued by the data, so as to determine whether the live data is compliant and whether to transmit the live data based on the analysis results. The embodiment of the present application can process live data of various scenarios, such as analyzing live data of conference scenarios, entertainment live scenarios, and game live scenarios. Specifically, as an optional embodiment, the live data includes e-commerce live data, entertainment live data, network live data, education live data, medical live data, and conference live data.

[0047] Live broadcast data can be understood as data composed of multiple sub-data (audio, image, text, etc.). The embodiment of the present application can configure a corresponding analyzer for the live broadcast data to analyze each sub-data in the live broadcast data. Specifically, as an optional embodiment, the determination of the sub-data corresponding to each analyzer includes at least one of the following steps: extracting the image data in the live broadcast data according to the image analyzer; extracting the face image in the live broadcast data according to the face analyzer; extracting the target image related to the target object in the live broadcast data according to the target object analyzer; extracting the text data in the live broadcast data according to the text analyzer; extracting the audio data in the live broadcast data according to the audio analyzer. The embodiment of the present application can configure an image analyzer, a face analyzer, a target object analyzer, a text analyzer, an audio analyzer, etc. to analyze the corresponding sub-data. It should be noted that in addition to the analyzer mentioned, the embodiment of the present application can also adopt other analyzers for corresponding analysis.

[0048] The embodiments of the present application can configure the corresponding identifier according to the specific scenario for which live broadcast data is analyzed. For example, for the live broadcast scenario of an online anchor, the embodiments of the present application can configure the analyzer to analyze the image to determine whether the anchor's behavior is compliant (such as whether it contains indecent behavior), and can also configure the analyzer to analyze the audio to determine whether the anchor's speech is compliant (such as whether it contains insults, verbal abuse, etc.). For another example, for a conference live broadcast scenario, the conference live broadcast scenario usually communicates through text and voice. Therefore, a text analyzer can be configured to analyze the text data in the image, and an audio analyzer can be configured to analyze the audio data.

[0049] After extracting the sub-data from the live broadcast data, the data features of the target sub-data and the historical analysis results of the historical target sub-data can be obtained in step 204, and the action information of the target analyzer can be determined. In step 206, the target analyzer is controlled according to the action information to determine the analysis result corresponding to the target sub-data.

[0050] The data features of the sub-data are used to represent the content carried by the sub-data. For different types of sub-data, the embodiments of the present application can extract the corresponding features in different ways. For example, for image data, the embodiments of the present application can determine the corresponding image data features based on the pixel value corresponding to each image pixel point; for example, for audio data, the embodiments of the present application can separate the corresponding acoustic features to form data features. The embodiments of the present application can configure a feature processing module on the server side to extract the data features of the sub-data of the live broadcast data. It should be noted that the embodiments of the present application can also configure the feature processing module (lightweight) of some sub-data on the first end for collecting data, which can be set according to the needs.

[0051] For the historical analysis results of historical sub-data, the embodiment of the present application can configure an analyzer on the server side for analysis, or it can be determined by configuring a data detector on the first end. Specifically, as an optional embodiment, the step of obtaining the historical analysis results includes at least one of the following steps: obtaining the historical analysis results obtained by the analyzer; receiving the historical analysis results uploaded by the first end that uploads the live broadcast data, and the first end determines the historical analysis results based on the data detector deployed on the first end.

[0052] For sub-data that need to be calculated by complex calculations, the embodiment of the present application can configure the analyzer on the server side, so as to perform corresponding calculations through the larger computing power of the server side, obtain the analysis results, and feed back to the decision model as the historical analysis results to make the next decision. For sub-data with simple calculations, the embodiment of the present application can configure a data detector on the first end. Specifically, as an optional embodiment, the method also includes: configuring a data detector and deploying the data detector to the first end to analyze based on the data detector at the first end. The server side can determine some lightweight (such as simple calculations and low computing power requirements) data detectors and send them to the first end to perform simple data processing at the first end to determine the historical analysis results. The data detector can extract the historical data features of the historical sub-data and the data features of the sub-data, and match them to determine the historical analysis results. After the first end determines the historical analysis results, it can add a timestamp to the historical analysis results and upload the historical analysis results to the server side. The server side aligns the historical analysis results and the sub-data according to the timestamp to make a decision.

[0053] In the embodiment of the present application, the data features of the target sub-data and the historical analysis results of the historical target sub-data can be input into the decision model, and the decision model is used as a basis. Specifically, as an optional embodiment, the step of determining the action information of the target analyzer includes at least one of the following steps: when the image data is determined to be a repeated image based on the data features of the image data and the historical analysis results of the historical image data, the action information of the image analyzer is determined to be the first image analysis action; when the face image is determined to be a repeated face image based on the data features of the face image and the historical analysis results of the historical face images, the action information of the face analyzer is determined to be the first face image analysis action; when the target image is determined to be a repeated target based on the data features of the target image and the historical analysis results of the historical target images, the action information of the target object analyzer is determined to be the first target image analysis action; when the text data is determined to be a repeated text based on the data features of the text data and the historical analysis results of the historical text data, the action information of the text analyzer is determined to be the first text analysis action; when the audio data is determined to be background audio based on the data features of the audio data and the historical analysis results of the historical audio data, the action information of the audio analyzer is determined to be the first audio analysis action.

[0054] For example, if Figure 2BAs shown, the embodiment of the present application can extract the data features of each sub-data through the feature processing module, and can also obtain the historical analysis results of the historical sub-data, and input the data features and historical analysis results into the decision model, determine the action information of the pornographic analysis unit, the face analysis unit and the text analysis unit according to the decision model, and control the corresponding analyzer to analyze the data according to the action information, determine the analysis results, and feed the analysis results back to the decision model so that the decision model can perform the next analysis based on the analysis results. In the case of repeated faces, repeated texts, and low pornographic identification segments, the embodiment of the present application can control the corresponding analyzer to ignore the corresponding sub-data, which can reduce the amount of data analyzed by the analyzer and improve data processing efficiency.

[0055] In addition, multiple analyzers can also analyze the sub-data of live broadcast data in a collaborative manner. For example, Figure 2C As shown, the analysis can be performed in a collaborative manner between a word analyzer and a semantic analyzer. Specifically, the data features of the text data can be extracted through the feature processing module, and the action information of the word analysis can be determined through the decision model, so that the audio data in the live broadcast data can be converted into text information based on the word analyzer, and the text information can be analyzed to see whether it contains illegal words (such as abusive words). It should be noted that in the embodiment of the present application, the decision model can make decisions based on the data features of the sub-data and the historical analysis results of the historical sub-data, wherein the historical analysis results of the historical sub-data can be determined on the server side or on the terminal side, such as Figure 2C In the example shown, calculation is performed at the acquisition end (first end), the end calculation result is determined, and the end calculation result and the collected data are uploaded to the server end, so that the server end makes a decision based on the uploaded data stream and the end calculation result. After the word analyzer determines the analysis result, for the illegal data in the analysis result, some illegal words may be part of the anchor's mood particles, or the words do not belong to illegal words in semantics. Therefore, the present application can also extract the data features corresponding to the text information through the feature processing model (such as adding the character features of the context to the character features of the characters), and determine the action information of the semantic analyzer through the decision model, so as to perform semantic analysis through the semantic analyzer, determine whether the semantic information contains illegal information, and then determine the analysis result through the collaboration of the two analyzers. Among them, when the text information in the image is repeated text, the embodiment of the present application can control the word analyzer to ignore the corresponding text data, thereby reducing the amount of data analyzed by the analyzer and improving data processing efficiency.

[0056] After determining the action information, the analyzer can be controlled according to the action information to analyze the data to be analyzed. Specifically, as an optional embodiment, the control of the target analyzer to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: controlling the target analyzer to identify at least part of the content of the target sub-data to determine the analysis result; controlling the target analyzer to ignore the target sub-data, and using the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result. The embodiment of the present application can control the analyzer through action information, and can control the analyzer to analyze the sub-data to be analyzed, or control the analyzer to analyze part of the sub-data to be analyzed, and can also control the analyzer to ignore the sub-data to be analyzed, and output the historical analysis result as the analysis result. The embodiment of the present application controls the analyzer to analyze or ignore by decision-making, thereby reducing the amount of data analyzed by the analyzer, thereby improving the processing efficiency of live data.

[0057] In addition to determining action information based on data features and historical analysis results, embodiments of the present application can also combine action reward information corresponding to the action information to make decisions on action information. Specifically, as an optional embodiment, the method further includes: determining the environmental impact information corresponding to the action information based on the analysis results; determining the action reward information corresponding to the action information of the target analyzer based on the environmental impact information, and determining the subsequent action information of the target analyzer based on the action reward information. In embodiments of the present application, the decision model can be a reinforcement learning model, which can determine the impact on the environment (such as the impact on the calculation speed, the impact on the calculation results, etc.) by taking different action information (such as analysis or ignoring), and then adjust the decision of the decision model based on the impact on the environment to obtain the maximum reward.

[0058] The risks of live broadcast data may include pornography, gambling, drugs, violence, terrorism, false propaganda, and fakes. In the scenario of transmitting live broadcast data, the embodiment of the present application can analyze whether the live broadcast data is compliant (whether there is a risk) through an analyzer, and transmit the compliant data to the user watching the live broadcast. Specifically, as an optional embodiment, the method also includes at least one of the following steps: when the live broadcast data is determined to be compliant according to the analysis result, the live broadcast data is transmitted; when the live broadcast data is determined to contain a risk event according to the analysis result, the corresponding risk processing is performed. For compliant live broadcast data, the embodiment of the present application can transmit it to the second end so that the user watching the live broadcast can watch it. For non-compliant (risky) live broadcast data, the embodiment of the present application can process it. For example, corresponding risk warnings can be issued to managers, anchors, etc., so as to adjust the live broadcast content. Specifically, as an optional embodiment, the corresponding risk processing includes at least one of the following steps: sending risk warnings to the live broadcast management platform; sending warning information to the first end uploading the live broadcast data; sending a live broadcast closure instruction to the second end watching the live broadcast data to close the live broadcast data; processing the content related to the risk event in the live broadcast data, and outputting the processed live broadcast data to the second end watching the live broadcast data.

[0059] In the embodiment of the present application, when it is determined that there is a risk in the live broadcast data, a risk reminder can be sent to the management personnel of the management platform so that the management personnel can intervene, process the live broadcast data or warn the anchor. A warning message can also be sent to the first end of the anchor so that the anchor can adjust his live broadcast content in time. A live broadcast closing instruction can also be sent to the second end, and the second end closes the live broadcast data according to the live broadcast closing instruction to prevent the live broadcast data with risks from being displayed to users watching the live broadcast. After the risks in the live broadcast data have been processed, the service end can send a live broadcast opening instruction to the second end to display compliant live broadcast data to users watching the live broadcast. The embodiment of the present application can also remove the content related to the risk event in the live broadcast data, and transmit the processed live broadcast data to the second end. If there is risk content in the speech of the anchor, the anchor can be temporarily muted to remove the risk. The embodiment of the present application can also perform risk processing on images, texts, etc. (such as adding mosaics, deleting texts, etc.), which can be set according to needs.

[0060] In addition, in order to enable the decision model to be updated as the live broadcast data is updated, the embodiment of the present application can also record the relevant data and train the decision model based on the recorded data. Specifically, as an optional embodiment, the action information is determined based on the action decision model, and the method also includes: obtaining live broadcast data, action information, analysis results, and action reward information as training data; based on the training data, the decision model is trained and the action decision model is updated.

[0061] In the embodiment of the present application, live data, action information corresponding to each sub-data of the live data, analysis results determined by the analyzer according to the action information, and action reward information corresponding to the analysis results can be recorded to train the decision model. The main focus of the training process of the decision model in the embodiment of the present application is the accuracy of the decision model, and the data processing efficiency is not high. Therefore, the embodiment of the present application can train the decision model through the recorded data during the idle period. The embodiment of the present application can also analyze each frame of the recorded live data and determine the corresponding analysis results, so as to more accurately determine the impact of the action information of the control analyzer on the environment, determine more accurate action reward information, and then obtain a more suitable decision model.

[0062] In an embodiment of the present application, live data and each analyzer corresponding to the live data can be obtained, and then the action information corresponding to the target analyzer can be determined based on the data characteristics of the target sub-data corresponding to the target analyzer in the live data and the historical analysis results of the historical target sub-data. Among them, the historical target sub-data is the sub-data of at least one frame before the target sub-data. After determining the action information corresponding to the target analyzer, the target analyzer can be controlled according to the action information to determine the analysis result corresponding to the target sub-data through the target analyzer. The embodiment of the present application can determine the action information by analyzing the data characteristics of the sub-data and the historical analysis results of the historical sub-data, so as to control the analyzer to analyze the sub-data or ignore the sub-data according to the action information, thereby reducing the amount of data analyzed by the analyzer, improving the analysis efficiency of the live data, and thus completing the transmission of the live data in a more timely manner.

[0063] The present application embodiment provides a data processing method, which can be applied on the server side, such as Figure 3 As shown, the method includes:

[0064] Step 302, obtain live broadcast data, and determine the sub-data corresponding to each analyzer. As an optional embodiment, the determination of the sub-data corresponding to each analyzer includes at least one of the following steps: extracting image data from the live broadcast data according to the image analyzer; extracting face images from the live broadcast data according to the face analyzer; extracting target images related to the target object from the live broadcast data according to the target object analyzer; extracting text data from the live broadcast data according to the text analyzer; extracting audio data from the live broadcast data according to the audio analyzer.

[0065] Step 304: Obtain data features of the target sub-data and historical analysis results of the historical target sub-data, and determine the action information of the target analyzer. As an optional embodiment, the step of obtaining the historical analysis results includes at least one of the following steps: obtaining the historical analysis results obtained by the analyzer; receiving the historical analysis results uploaded by the first end that uploads the live broadcast data, and the first end determines the historical analysis results based on the data detector deployed at the first end. As an optional embodiment, the method also includes: configuring the data detector and deploying the data detector to the first end, so as to perform analysis at the first end based on the data detector. As an optional embodiment, the step of determining the action information of the target analyzer includes at least one of the following steps: when it is determined that the image data is a repeated image based on the data characteristics of the image data and the historical analysis results of the historical image data, the action information of the image analyzer is determined to be a first image analysis action; when it is determined that the face image is a repeated face image based on the data characteristics of the face image and the historical analysis results of the historical face images, the action information of the face analyzer is determined to be a first face image analysis action; when it is determined that the target image is a repeated target based on the data characteristics of the target image and the historical analysis results of the historical target images, the action information of the target object analyzer is determined to be a first target image analysis action; when it is determined that the text data is a repeated text based on the data characteristics of the text data and the historical analysis results of the historical text data, the action information of the text analyzer is determined to be a first text analysis action; when it is determined that the audio data is background audio based on the data characteristics of the audio data and the historical analysis results of the historical audio data, the action information of the audio analyzer is determined to be a first audio analysis action.

[0066] Step 306: Control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data. As an optional embodiment, the control of the target analyzer to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: controlling the target analyzer to identify at least part of the content of the target sub-data to determine the analysis result; controlling the target analyzer to ignore the target sub-data and taking the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result.

[0067] Step 308: When the live broadcast data is determined to be compliant according to the analysis result, the live broadcast data is transmitted.

[0068] Step 310: When it is determined that the live broadcast data contains a risk event based on the analysis result, corresponding risk processing is performed. As an optional embodiment, the corresponding risk processing includes at least one of the following steps: sending a risk reminder to the live broadcast management platform; sending a warning message to the first end that uploads the live broadcast data; sending a live broadcast closing instruction to the second end that views the live broadcast data to close the live broadcast data; processing the content related to the risk event in the live broadcast data, and outputting the processed live broadcast data to the second end that views the live broadcast data.

[0069] Step 312: Determine the environmental impact information corresponding to the action information based on the analysis result.

[0070] Step 314: Determine the action reward information corresponding to the action information of the target analyzer based on the environmental impact information, so as to determine the subsequent action information of the target analyzer based on the action reward information. After determining the action reward information, the action reward information can be fed back to the decision model of step 304 to determine the subsequent action information.

[0071] The embodiment of the present application can be completed by the server, which can provide services for analyzing (or auditing) live data. The server can provide an interface to obtain live data and separate the sub-data corresponding to each analyzer in the live data, such as image data corresponding to an image analyzer, face image corresponding to a face analyzer, target image corresponding to a target object analyzer, text data corresponding to a text analyzer, audio data corresponding to an audio analyzer, etc. Afterwards, the target sub-data corresponding to the target analyzer can be obtained, and the action information of the target analyzer can be determined based on the data features of the target sub-data and the historical analysis results of the historical target sub-data. Among them, the historical analysis results can be determined based on the analyzer configured on the server, and can also be determined based on the data detector configured on the terminal.

[0072] After determining the action information, the target analyzer can be controlled according to the action information. The embodiment of the present application can determine the action information corresponding to each analyzer, thereby controlling some analyzers to work (analyze) and some analyzers not to work (ignore data), thereby reducing the total amount of data analysis and improving data processing efficiency. The embodiment of the present application can analyze the sub-data of the live data by the analyzer to determine whether the live data is compliant. For compliant live data, the embodiment of the present application can transmit the live data to the second end for viewing. For non-compliant (risky) live data, the embodiment of the present application can perform corresponding risk processing. In addition, after determining the analysis results, the embodiment of the present application can also determine the action reward information based on the environmental impact information of the analysis results for the environment, so as to determine the subsequent actions of the target analyzer based on the action reward information.

[0073] The embodiment of the present application provides a data processing method, which can be applied to a first end. The first end can be understood as a device that collects sub-data of live broadcast data and uploads the sub-data to a server. The server can configure a data detector at the first end to perform simple calculations through the data detector and upload the calculation results to the server, so that the server can determine the analysis results based on the data uploaded by the first end and the calculation results, and then perform risk analysis on the live broadcast data. Specifically, Figure 4 As shown, the method includes:

[0074] Step 402: collect target sub-data corresponding to the target analyzer in the live broadcast data.

[0075] Step 404: Determine the historical analysis result of the historical target sub-data based on the data detector.

[0076] Step 406, upload the target sub-data and historical analysis results to the server, so that the server can determine the action information of the target analyzer based on the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data, so as to control the target analyzer based on the action information and determine the analysis results of the target sub-data.

[0077] The implementation of this embodiment is similar to that of the above embodiment. The specific implementation may refer to the specific implementation of the above embodiment, which will not be described again here.

[0078] The embodiment of the present application can be applied to the first end of data collection. The first end can collect corresponding sub-data through the data collection component, and analyze the sub-data according to the data detector to determine the corresponding historical analysis results, and then upload the data stream and historical analysis results of the collected sub-data to the server. After the server receives the data stream and historical analysis results of the sub-data, it can align the data and extract the data features of each sub-data in the data stream, and then input the data features and historical analysis results (calculation results) into the decision model, and determine the corresponding action information through the decision model. Afterwards, the analyzer is controlled to perform corresponding analysis based on the action information to determine the analysis results, and then the compliant live data can be transmitted to the second end. The embodiment of the present application can complete part of the data calculation on the end side by deploying a data detector on the first end, so that the data processing efficiency can be improved by collaborative calculation between the end side and the server, and the live data can be transmitted more timely.

[0079] In addition to being applied to the scenario of analyzing live broadcast data, the embodiments of the present application can also be applied to the scenario of analyzing pre-stored multimedia data, and can analyze the analysis actions (action information) corresponding to each sub-data in the multimedia data, thereby performing corresponding analysis and determining the analysis results of the multimedia data. Specifically, Figure 5 As shown, the method includes:

[0080] Step 502: Acquire multimedia data and determine sub-data corresponding to each analyzer.

[0081] Step 504: Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer.

[0082] Step 506: Control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data.

[0083] The implementation of this embodiment is similar to that of the above embodiment. The specific implementation may refer to the specific implementation of the above embodiment, which will not be described again here.

[0084] The embodiments of the present application can be applied in the scenario of analyzing pre-stored multimedia data, such as the scenario of analyzing long videos, short videos, audio, etc. Specifically, the embodiments of the present application can obtain multimedia data and determine multiple analyzers corresponding to the multimedia data. After that, the multimedia data is divided into sub-data corresponding to each analyzer, and the action information corresponding to the analyzer is determined through the decision model. After that, the analyzer is controlled according to the action information, so as to perform analysis and determine the corresponding analysis results. The embodiments of the present application can be applied in the scenario of analyzing whether multimedia data infringes copyright. The sub-data in the multimedia data can be analyzed to determine whether it infringes copyright. For example, it can be analyzed whether a video in the video data infringes copyright, whether a frame of image in the video data infringes copyright, and whether the audio data infringes copyright, so as to determine the corresponding analysis results and perform corresponding processing.

[0085] The embodiments of the present application can also be applied to the scenario of managing a certain target object, such as managing the transportation process of a transport vehicle, such as managing the storage process of a storage object, and can analyze the sub-data of the object description data to determine the corresponding analysis action (action information), thereby performing corresponding analysis and determining the analysis result of the object description data. Specifically, Figure 6 As shown, the method includes:

[0086] Step 602: Obtain object description data of the target object, and determine sub-data corresponding to each analyzer.

[0087] Step 604: Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer.

[0088] Step 606: Control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data.

[0089] The implementation of this embodiment is similar to that of the above embodiment. The specific implementation may refer to the specific implementation of the above embodiment, which will not be described again here.

[0090] The embodiments of the present application can be applied in scenarios where target objects are to be managed, such as scenarios where the transportation process of transport vehicles is to be managed, and scenarios where the storage process of storage objects is to be managed. The object description data of the target object can be obtained, and the sub-data corresponding to each analyzer can be determined. The action information is then determined through a decision model, and the analyzer is controlled to analyze the sub-data based on the action information to determine the corresponding analysis results. For example, risk events such as driving videos during transportation and whether there are non-compliant behaviors on the vehicle's driving route can be analyzed for management. For another example, the video data of storage objects during the warehousing process and the data transmitted by various sensors can be analyzed to determine whether the storage objects are stored normally.

[0091] On the basis of the above embodiments, the embodiments of the present application also provide a data processing method, which can be applied to the scenario of reviewing live broadcast data and processing the live broadcast data according to the review results. Specifically, the method includes: obtaining live broadcast data and determining the sub-data corresponding to each analyzer. Obtaining the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data, and determining the action information of the target analyzer. Controlling the target analyzer according to the action information to determine the analysis results corresponding to the target sub-data. Determine the review results of the live broadcast data based on the analysis results of each sub-data, so as to process the live broadcast data according to the review results.

[0092] Among them, the audit results of the live broadcast data may include two results: compliance and non-compliance (including risks). For different results, the embodiments of the present application may adopt different processing methods. As an optional embodiment, the method also includes: when the live broadcast data is determined to be compliant based on the audit results, the live broadcast data is transmitted; when the live broadcast data is determined to contain risk events based on the audit results, corresponding risk processing is performed. As an optional embodiment, the corresponding risk processing includes at least one of the following steps: sending a risk warning to the live broadcast management platform; sending a warning message to the first end that uploads the live broadcast data; sending a live broadcast closure instruction to the second end that watches the live broadcast data to close the live broadcast data; processing the content related to the risk event in the live broadcast data, and outputting the processed live broadcast data to the second end that watches the live broadcast data.

[0093] The implementation of this embodiment is similar to that of the above embodiment. The specific implementation may refer to the specific implementation of the above embodiment, which will not be described again here.

[0094] The embodiment of the present application can obtain live data and the corresponding analyzer, and then obtain the target sub-data corresponding to the target analyzer in the live data, and determine the action information of the target analyzer according to the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data. After that, the target analyzer can be controlled according to the action information to determine the analysis result of the target sub-data. Then the analysis results of each sub-data can be counted to determine the audit result of the live data. The audit result of the live data can include two results: compliance and non-compliance. For compliant live data, the embodiment of the present application can transmit the live data to the second end. For non-compliant live data (there is a risk), on the one hand, a prompt (such as a warning message) can be issued to the anchor, and on the other hand, the risky content can be processed and the processed live data can be transmitted to the second end. Among them, it should be noted that in the embodiment of the present application, the risk processing process for non-compliant live data can be completed on the server side, and the audit result can also be sent to other devices (such as a live management platform) for processing by other devices.

[0095] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0096] Based on the above embodiment, this embodiment further provides a data processing device, referring to Figure 7 , specifically it can include the following modules:

[0097] The to-be-analyzed data acquisition module 702 is used to acquire live broadcast data and determine sub-data corresponding to each analyzer.

[0098] The action information determination module 704 is used to obtain data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer.

[0099] The analysis result acquisition module 706 is used to control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data.

[0100] In summary, in an embodiment of the present application, live data and each analyzer corresponding to the live data can be obtained, and then the action information corresponding to the target analyzer can be determined based on the data characteristics of the target sub-data corresponding to the target analyzer in the live data and the historical analysis results of the historical target sub-data. After determining the action information corresponding to the target analyzer, the target analyzer can be controlled based on the action information to determine the analysis result corresponding to the target sub-data through the target analyzer. The embodiment of the present application can determine the action information by analyzing the data characteristics of the sub-data and the historical analysis results of the historical sub-data, so as to control the analyzer to analyze the sub-data or ignore the sub-data based on the action information, thereby reducing the amount of data analyzed by the analyzer, improving the processing efficiency of the live data, and thus completing the transmission of the live data in a more timely manner.

[0101] On the basis of the above embodiment, this embodiment further provides a data processing device, which may specifically include the following modules:

[0102] The data acquisition processing module to be analyzed is used to acquire live broadcast data and determine the sub-data corresponding to each analyzer. As an optional embodiment, the data acquisition processing module to be analyzed includes at least one of the following modules: an image data separation processing submodule, used to extract image data from live broadcast data based on an image analyzer; a face image separation processing submodule, used to extract face images from live broadcast data based on a face analyzer; a target image separation processing submodule, used to extract target images related to target objects from live broadcast data based on a target object analyzer; a text data separation processing submodule, used to extract text data from live broadcast data based on a text analyzer; and an audio data separation processing submodule, used to extract audio data from live broadcast data based on an audio analyzer.

[0103] An action information acquisition processing module is used to acquire data features of target sub-data and historical analysis results of historical target sub-data, and determine the action information of the target analyzer. As an optional embodiment, the action information acquisition processing module includes at least one of the following modules: a first historical analysis result acquisition processing module is used to acquire historical analysis results obtained by the analyzer; a second historical analysis result acquisition processing module is used to receive historical analysis results uploaded by the first end that uploads live broadcast data, and the first end determines the historical analysis results based on the data detector deployed at the first end. As an optional embodiment, the device also includes: a data monitor sending processing module, which is used to configure the data detector and deploy the data detector to the first end, so as to perform analysis at the first end based on the data detector. As an optional embodiment, the action information acquisition processing module includes at least one of the following modules: an image data analysis action determination processing submodule, which is used to determine that the action information of the image analyzer is a first image analysis action when the image data is determined to be a repeated image based on the data features of the image data and the historical analysis results of the historical image data; a face image analysis action determination processing submodule, which is used to determine that the action information of the face analyzer is a first face image analysis action when the face image is determined to be a repeated face image based on the data features of the face image and the historical analysis results of the historical face images; a target image analysis action determination processing submodule, which is used to determine that the action information of the face analyzer is a first face image analysis action when the face image is determined to be a repeated face image based on the data features of the face image and the historical analysis results of the historical face images The processing submodule for determining the action information of the target object analyzer as the first target image analysis action when the target image is determined to be a repeated target based on the data features of the text data and the historical analysis results of the historical text data; the processing submodule for determining the action information of the text analyzer as the first text analysis action when the text data is determined to be repeated text based on the data features of the text data and the historical analysis results of the historical text data; the processing submodule for determining the action information of the audio analyzer as the first audio analysis action when the audio data is determined to be background audio based on the data features of the audio data and the historical analysis results of the historical audio data.

[0104] The analysis result acquisition processing module is used to control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data. As an optional embodiment, the analysis result acquisition processing module includes at least one of the following modules: a first analysis result acquisition processing submodule, which is used to control the target analyzer to identify at least part of the content of the target sub-data to determine the analysis result; a second analysis result acquisition processing submodule, which is used to control the target analyzer to ignore the target sub-data and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result.

[0105] The live data transmission processing module is used to transmit the live data when it is determined that the live data is compliant according to the analysis result.

[0106] The live data risk processing module is used to perform corresponding risk processing when it is determined that the live data contains risk events based on the analysis results. As an optional embodiment, the live data risk processing module includes at least one of the following modules: a first risk processing submodule, used to send risk prompts to the live management platform; a second risk processing submodule, used to send warning information to the first end that uploads the live data; a third risk processing submodule, used to send a live broadcast closure instruction to the second end that watches the live data to close the live data; a fourth risk processing submodule, used to process the content related to the risk event in the live data, and output the processed live data to the second end that watches the live data.

[0107] The environmental impact determination processing module is used to determine the environmental impact information corresponding to the action information based on the analysis result.

[0108] The action reward determination processing module is used to determine the action reward information corresponding to the action information of the target analyzer based on the environmental impact information, so as to determine the subsequent action information of the target analyzer based on the action reward information.

[0109] The embodiment of the present application can be completed by the server, which can provide services for analyzing (or auditing) live data. The server can provide an interface to obtain live data and separate the sub-data corresponding to each analyzer in the live data, such as image data corresponding to an image analyzer, face image corresponding to a face analyzer, target image corresponding to a target object analyzer, text data corresponding to a text analyzer, audio data corresponding to an audio analyzer, etc. Afterwards, the target sub-data corresponding to the target analyzer can be obtained, and the action information of the target analyzer can be determined based on the data features of the target sub-data and the historical analysis results of the historical target sub-data. Among them, the historical analysis results can be determined based on the analyzer configured on the server, and can also be determined based on the data detector configured on the terminal.

[0110] After determining the action information, the target analyzer can be controlled according to the action information. The embodiment of the present application can determine the action information corresponding to each analyzer, thereby controlling some analyzers to work (analyze) and some analyzers not to work (ignore data), thereby reducing the total amount of data analysis and improving data processing efficiency. The embodiment of the present application can analyze the sub-data of the live data by the analyzer to determine whether the live data is compliant. For compliant live data, the embodiment of the present application can transmit the live data to the second end for viewing. For non-compliant (risky) live data, the embodiment of the present application can perform corresponding risk processing. In addition, after determining the analysis results, the embodiment of the present application can also determine the action reward information based on the environmental impact information of the analysis results for the environment, so as to determine the subsequent actions of the target analyzer based on the action reward information.

[0111] Based on the above embodiment, this embodiment further provides a data processing device, referring to Figure 8 , specifically it can include the following modules:

[0112] The sub-data collection module 802 is used to collect the target sub-data corresponding to the target analyzer in the live broadcast data.

[0113] The sub-data analysis module 804 is used to determine the historical analysis results of the historical target sub-data according to the data detector.

[0114] The data uploading module 806 for analyzing is used to upload the target sub-data and the historical analysis results to the server, so that the server can determine the action information of the target analyzer based on the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data, so as to control the target analyzer based on the action information and determine the analysis results of the target sub-data.

[0115] In summary, the embodiments of the present application can be applied to the first end of data collection. The first end can collect corresponding sub-data through the data collection component, and analyze the sub-data according to the data detector to determine the corresponding historical analysis results, and then upload the data stream of the collected sub-data and the historical analysis results to the server. After the server receives the data stream of the sub-data and the historical analysis results, it can align the data, extract the data features of each sub-data in the data stream, and input them into the decision model, and determine the corresponding action information through the decision model. Afterwards, the analyzer is controlled to perform corresponding analysis based on the action information to determine the analysis results, and then the compliant live data can be transmitted to the second end. The embodiments of the present application can complete part of the data calculation on the end side by deploying a data detector on the first end, so that the data processing efficiency can be improved by collaborative calculation between the end side and the server, and the live data can be transmitted more timely.

[0116] Based on the above embodiment, this embodiment further provides a data processing device, which can be applied to the scenario of analyzing multimedia data. Specifically,

[0117] The to-be-analyzed data acquisition module 702 is specifically used to acquire multimedia data and determine sub-data corresponding to each analyzer.

[0118] The action information determination module 704 is specifically used to obtain data features of the target sub-data and historical analysis results of the historical target sub-data, and determine the action information of the target analyzer.

[0119] The analysis result acquisition module 706 is specifically used to control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data.

[0120] The embodiments of the present application can be applied in the scenario of analyzing pre-stored multimedia data, such as the scenario of analyzing long videos, short videos, audio, etc. Specifically, the embodiments of the present application can obtain multimedia data and determine multiple analyzers corresponding to the multimedia data. After that, the multimedia data is divided into sub-data corresponding to each analyzer, and the action information corresponding to the analyzer is determined through the decision model. After that, the analyzer is controlled according to the action information, so as to perform analysis and determine the corresponding analysis results. The embodiments of the present application can be applied in the scenario of analyzing whether multimedia data infringes copyright. The sub-data in the multimedia data can be analyzed to determine whether it infringes copyright. For example, it can be analyzed whether a video in the video data infringes copyright, whether a frame of image in the video data infringes copyright, and whether the audio data infringes copyright, so as to determine the corresponding analysis results and perform corresponding processing.

[0121] Based on the above embodiment, this embodiment further provides a data processing device, which can be applied to the scenario of managing the target object. Specifically,

[0122] The to-be-analyzed data acquisition module 702 is specifically used to acquire object description data of the target object and determine sub-data corresponding to each analyzer.

[0123] The action information determination module 704 is specifically used to obtain data features of the target sub-data and historical analysis results of the historical target sub-data, and determine the action information of the target analyzer.

[0124] The analysis result acquisition module 706 is specifically used to control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data.

[0125] The embodiments of the present application can be applied in scenarios where target objects are to be managed, such as scenarios where the transportation process of transport vehicles is to be managed, and scenarios where the storage process of storage objects is to be managed. The object description data of the target object can be obtained, and the sub-data corresponding to each analyzer can be determined. The action information is then determined through a decision model, and the analyzer is controlled to analyze the sub-data based on the action information to determine the corresponding analysis results. For example, risk events such as driving videos during transportation and whether there are non-compliant behaviors on the vehicle's driving route can be analyzed for management. For another example, the video data of storage objects during the warehousing process and the data transmitted by various sensors can be analyzed to determine whether the storage objects are stored normally.

[0126] Based on the above embodiment, this embodiment further provides a data processing device, which can be applied to the scenario of reviewing live broadcast data. Specifically,

[0127] The to-be-analyzed data acquisition module 702 is specifically used to acquire live broadcast data and determine sub-data corresponding to each analyzer.

[0128] The action information determination module 704 is specifically used to obtain data features of the target sub-data and historical analysis results of the historical target sub-data, and determine the action information of the target analyzer.

[0129] The analysis result acquisition module 706 is specifically used to control the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data. According to the analysis result of each sub-data, the audit result of the live data is determined to process the live data according to the audit result.

[0130] The embodiment of the present application can obtain live data and the corresponding analyzer, and then obtain the target sub-data corresponding to the target analyzer in the live data, and determine the action information of the target analyzer according to the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data. After that, the target analyzer can be controlled according to the action information to determine the analysis result of the target sub-data. Then the analysis results of each sub-data can be counted to determine the audit result of the live data. The audit result of the live data can include two results: compliance and non-compliance. For compliant live data, the embodiment of the present application can transmit the live data to the second end. For non-compliant live data (there is a risk), on the one hand, a prompt (such as a warning message) can be issued to the anchor, and on the other hand, the risky content can be processed and the processed live data can be transmitted to the second end. Among them, it should be noted that in the embodiment of the present application, the risk processing process for non-compliant live data can be completed on the server side, and the audit result can also be sent to other devices (such as a live management platform) for processing by other devices.

[0131] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0132] The present application embodiment provides one or more machine-readable media, on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more methods as described in the above embodiments. In the present application embodiment, the electronic device includes a server, a terminal device, and the like.

[0133] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as a server (cluster), a terminal, etc. Fig. 9 An exemplary apparatus 900 that can be used to implement various embodiments described in this application is schematically illustrated.

[0134] For one embodiment, Fig. 9 An exemplary apparatus 900 is shown having one or more processors 902, a control module (chip set) 904 coupled to at least one of the processor(s) 902, a memory 906 coupled to the control module 904, a non-volatile memory (NVM) / storage device 908 coupled to the control module 904, one or more input / output devices 910 coupled to the control module 904, and a network interface 912 coupled to the control module 904.

[0135] The processor 902 may include one or more single-core or multi-core processors, and the processor 902 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 900 can be used as a server, terminal, or other device described in the embodiments of the present application.

[0136] In some embodiments, the apparatus 900 may include one or more computer-readable media (e.g., memory 906 or NVM / storage device 908) having instructions 914 and one or more processors 902 combined with the one or more computer-readable media and configured to execute the instructions 914 to implement a module to perform the actions described in the present disclosure.

[0137] For one embodiment, the control module 904 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 902 and / or any suitable device or component in communication with the control module 904 .

[0138] The control module 904 may include a memory controller module to provide an interface to the memory 906. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0139] The memory 906 may be used, for example, to load and store data and / or instructions 914 for the device 900. For one embodiment, the memory 906 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 906 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0140] For one embodiment, control module 904 may include one or more input / output controllers to provide an interface to NVM / storage device 908 and input / output device(s) 910 .

[0141] For example, NVM / storage 908 may be used to store data and / or instructions 914. NVM / storage 908 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0142] NVM / storage device 908 may include storage resources that are part of the device on which apparatus 900 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 908 may be accessed via (one or more) input / output devices 910 over a network.

[0143] (One or more) input / output devices 910 may provide an interface for the apparatus 900 to communicate with any other appropriate device, and the input / output device 910 may include a communication component, an audio component, a sensor component, etc. The network interface 912 may provide an interface for the apparatus 900 to communicate through one or more networks, and the apparatus 900 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0144] For one embodiment, at least one of the processor(s) 902 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 904. For one embodiment, at least one of the processor(s) 902 may be packaged together with the logic of one or more controllers of the control module 904 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 902 may be integrated on the same die with the logic of one or more controllers of the control module 904. For one embodiment, at least one of the processor(s) 902 may be integrated on the same die with the logic of one or more controllers of the control module 904 to form a system-on-chip (SoC).

[0145] In various embodiments, the device 900 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 900 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 900 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0146] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0147] An embodiment of the present application further provides an electronic device, comprising: a processor; and a memory, on which executable code is stored, and when the executable code is executed, the processor executes one or more methods described in the embodiments of the present application.

[0148] The embodiments of the present application also provide one or more machine-readable media on which executable codes are stored. When the executable codes are executed, the processor executes one or more methods described in the embodiments of the present application.

[0149] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0150] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0151] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0152] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0154] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0155] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0156] The data processing method, the data processing device, the electronic device and the storage medium provided by the present application are introduced in detail above. The principles and implementation methods of the present application are explained in this article by using specific examples. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method includes: Obtain live broadcast data and determine the sub-data corresponding to each analyzer; Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer, wherein the action information includes at least one of the following: full analysis, partial analysis, and ignore; Controlling the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: Controlling the target analyzer to identify at least part of the content of the target sub-data and determine an analysis result; Control the target analyzer to ignore the target sub-data, and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result; Determining environmental impact information corresponding to the action information based on the analysis result; Based on the environmental impact information, action reward information corresponding to the action information of the target analyzer is determined, so as to determine subsequent action information of the target analyzer based on the action reward information.

2. The method according to claim 1, characterized in that The determining of the sub-data corresponding to each analyzer comprises at least one of the following steps: Extracting image data from the live broadcast data based on the image analyzer; According to the face analyzer, extract the face image in the live broadcast data; According to the target object analyzer, a target image related to the target object in the live broadcast data is extracted; Extracting text data from the live broadcast data based on the text analyzer; The audio data in the live broadcast data is extracted according to the audio analyzer.

3. The method according to claim 1, characterized in that The step of obtaining the historical analysis result includes at least one of the following steps: Get the historical analysis results obtained by the analyzer; Receive the historical analysis results uploaded by the first end that uploads the live broadcast data, where the first end determines the historical analysis results based on a data detector deployed at the first end.

4. The method according to claim 2, characterized in that: The step of determining the action information of the target analyzer comprises at least one of the following steps: When it is determined that the image data is a repeated image based on the data features of the image data and the historical analysis results of the historical image data, determining the action information of the image analyzer to be a first image analysis action; When it is determined that the facial image is a repeated facial image based on the data features of the facial image and the historical analysis results of the historical facial images, determining the action information of the facial analyzer to be a first facial image analysis action; When it is determined that the target image is a repeated target based on the data features of the target image and the historical analysis results of the historical target images, the action information of the target object analyzer is determined to be a first target image analysis action; When it is determined that the text data is a repeated text according to the data characteristics of the text data and the historical analysis result of the historical text data, the action information of the text analyzer is determined to be a first text analysis action; When the audio data is determined to be background audio based on the data features of the audio data and the historical analysis results of the historical audio data, the action information of the audio analyzer is determined to be a first audio analysis action.

5. The method according to claim 1, characterized in that The method further comprises: When it is determined according to the analysis result that the live broadcast data is compliant, transmitting the live broadcast data; When it is determined according to the analysis result that the live broadcast data contains risk events, corresponding risk processing is performed.

6. The method according to claim 5, characterized in that The corresponding risk management includes at least one of the following steps: Send risk warnings to the live broadcast management platform; Send a warning message to the first end that uploads the live broadcast data; Sending a live broadcast closing instruction to the second end watching the live broadcast data to close the live broadcast data; The content related to the risk event in the live broadcast data is processed, and the processed live broadcast data is output to the second end that watches the live broadcast data.

7. A data processing method, characterized in that: The method comprises: Collect target sub-data corresponding to the target analyzer in the live broadcast data; Determine historical analysis results of historical target sub-data based on the data detector; Upload the target sub-data and the historical analysis results to the server, so that the server can determine the action information of the target analyzer according to the data characteristics of the target sub-data and the historical analysis results of the historical target sub-data, and control the target analyzer according to the action information to determine the analysis result of the target sub-data, wherein the action information includes at least one of the following: full analysis, partial analysis and ignore, and the control of the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: control the target analyzer to identify at least part of the content of the target sub-data to determine the analysis result; control the target analyzer to ignore the target sub-data, and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result; Determining environmental impact information corresponding to the action information based on the analysis result; Based on the environmental impact information, action reward information corresponding to the action information of the target analyzer is determined, so as to determine subsequent action information of the target analyzer based on the action reward information.

8. A data processing method, characterized in that: The method comprises: Acquire multimedia data and determine sub-data corresponding to each analyzer; Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer, wherein the action information includes at least one of the following: full analysis, partial analysis, and ignore; Controlling the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: Controlling the target analyzer to identify at least part of the content of the target sub-data and determine an analysis result; Control the target analyzer to ignore the target sub-data, and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result; Determining environmental impact information corresponding to the action information based on the analysis result; Based on the environmental impact information, action reward information corresponding to the action information of the target analyzer is determined, so as to determine subsequent action information of the target analyzer based on the action reward information.

9. A data processing method, characterized in that: The method comprises: Obtain object description data of the target object and determine the sub-data corresponding to each analyzer; Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer, wherein the action information includes at least one of the following: full analysis, partial analysis, and ignore; Controlling the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: Controlling the target analyzer to identify at least part of the content of the target sub-data and determine an analysis result; Control the target analyzer to ignore the target sub-data, and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result; Determining environmental impact information corresponding to the action information based on the analysis result; Based on the environmental impact information, action reward information corresponding to the action information of the target analyzer is determined, so as to determine subsequent action information of the target analyzer based on the action reward information.

10. A data processing method, characterized in that: The method comprises: Obtain live broadcast data and determine the sub-data corresponding to each analyzer; Acquire data features of the target sub-data and historical analysis results of the historical target sub-data, and determine action information of the target analyzer, wherein the action information includes at least one of the following: full analysis, partial analysis, and ignore; Controlling the target analyzer according to the action information to determine the analysis result corresponding to the target sub-data includes at least one of the following steps: Controlling the target analyzer to identify at least part of the content of the target sub-data and determine an analysis result; Control the target analyzer to ignore the target sub-data, and use the historical analysis result of the target sub-analyzer on the historical target sub-data as the analysis result; Determining environmental impact information corresponding to the action information based on the analysis result; Determine action reward information corresponding to the action information of the target analyzer according to the environmental impact information, so as to determine subsequent action information of the target analyzer according to the action reward information; Based on the analysis results of each sub-data, the audit results of the live broadcast data are determined, and the live broadcast data is processed according to the audit results.

11. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, causes the processor to execute the method according to any one of claims 1 to 10.

12. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to perform the method according to any one of claims 1 to 10.

Images