Data processing method and device and storage medium

Abstract

The invention discloses a data processing method and device and a storage medium, and belongs to the field of data security. The method is applied to a first device, the first device comprises a first rule, and the first rule is used for defining attribute information of normal data and / or attribute information of abnormal data. In the method, attribute information of to-be-detected first data is acquired; and under the condition that the attribute information of the first data is matched with the first rule, determining that the first data is normal data or abnormal data. And under the condition that the attribute information of the first data is not matched with the first rule, detecting whether the first data is abnormal data based on a first abnormal data detection model. According to the invention, the accuracy of abnormal data detection can be improved, and the calculation overhead and time delay of abnormal data detection can be reduced.

Description

Technical Field

[0001] This application relates to the field of data security, and in particular to a method, apparatus and storage medium for processing data. Background Technology

[0002] In recent years, data security has received significant attention. Abnormal data is considered to have lower security and may pose a security threat to society and the public. For example, telecommunications and online fraud cases have been on the rise in recent years. Users may receive audio, video, text, images, and / or web pages related to fraud. This data is abnormal and may result in financial losses for users.

[0003] To improve data security, related technologies receive and inspect data sent by the communicating party, issuing alarms when anomalies are detected. However, the detection methods provided by these technologies may result in false positives or false negatives, leading to low accuracy. Alternatively, the detection methods may have high computational costs, prolonging the detection process. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for processing data to improve the accuracy of detecting abnormal data and reduce the computational overhead and latency of detecting abnormal data. The technical solution is as follows:

[0005] In a first aspect, this application provides a method for processing data, the method being applied to a first device, the first device including a first rule, the first rule being used to define attribute information of normal data and / or attribute information of abnormal data. In the method, attribute information of first data to be detected is acquired. If the attribute information of the first data matches the first rule, the first data is determined to be normal data or abnormal data. If the attribute information of the first data does not match the first rule, whether the first data is abnormal data is detected based on a first abnormal data detection model.

[0006] When the attribute information of the first data matches the first rule, it can be accurately determined whether the first data is abnormal or normal. This way, while maintaining the accuracy of detecting whether the first data is abnormal, a model can be omitted, reducing computational overhead and latency. Conversely, when the attribute information of the first data does not match the first rule, it means that the first rule cannot accurately determine whether the first data is normal or abnormal. In this case, the first abnormal data detection model is used to detect the first data, avoiding false positives and false negatives, and improving the accuracy of detecting whether the first data is abnormal.

[0007] In one possible implementation, the first rule includes a first sub-rule and / or a second sub-rule. The first sub-rule defines the attribute information of normal data, and the second sub-rule defines the attribute information of abnormal data. If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data. If the attribute information of the first data matches the second sub-rule, the first data is determined to be abnormal data. In this way, the first or second sub-rule can be used to detect whether the first data is abnormal or normal, reducing computational overhead and the latency required to detect abnormal data.

[0008] In another possible implementation, the attribute information of the first data includes the source information of the first data, the first sub-rule includes the source information of at least one normal data, and the second sub-rule includes the source information of at least one abnormal data; or, the attribute information of the first data includes the identification information of the first data, the first sub-rule includes the identification information of at least one normal data, and the second sub-rule includes the identification information of at least one abnormal data. Thus, if the attribute information of the first data matches the first sub-rule, it can be accurately determined that the first data is normal data; or, if the attribute information of the first data matches the second sub-rule, it can be accurately determined that the first data is abnormal data.

[0009] In another possible implementation, the first device further includes a second rule for defining an abnormal region. The attribute information of the first data includes the location of the sending object. If the location belongs to an abnormal region, a second abnormal data detection model is obtained by increasing the sensitivity of the first abnormal data detection model. The first data is then used to determine whether it is abnormal. This allows for focused detection of first data originating from an abnormal region, thereby improving the accuracy of detecting whether the first data is abnormal.

[0010] In another possible implementation, the first anomaly detection model includes a first threshold. This model is used to obtain the anomaly confidence level of the first data and determines the first data as an anomaly when the anomaly confidence level is greater than or equal to the first threshold. Adjusting the first threshold of the first anomaly detection model to a second threshold, where the second threshold is less than the first threshold, yields a second anomaly detection model. Thus, by adjusting the threshold, a more sensitive anomaly detection model can be obtained, simplifying the adjustment complexity.

[0011] In another possible implementation, the first data is preprocessed to obtain second data, which includes a portion of the first data. The first anomaly detection model and the second data are then used to determine whether the first data is an anomaly. This reduces the amount of data to be detected using the anomaly detection model, decreases computational overhead, and improves detection efficiency.

[0012] In another possible implementation, the second data includes a portion of the first data that meets the quality criteria. This improves the accuracy of using the first anomaly detection model to determine whether the first data is an anomaly.

[0013] In another possible implementation, the first data is audio data, and the second data is the portion of the first data whose noise intensity is below a third threshold; or...

[0014] The first data is video data, and the second data is the portion of the first data where the contrast is below the fourth threshold, the illumination intensity is below the fifth threshold, and / or includes the unoccluded portion of the target image; or...

[0015] The first data is the image, and the second data includes the target image in the image or data obtained by expanding the edges of the target image.

[0016] In another possible implementation, the first data is text data containing multiple characters, and the second data includes the first M characters in the first data, where M is an integer greater than 1.

[0017] In another possible implementation, a detection command is sent to a second device, which includes a first data anomaly detection model. The detection command instructs the second device to detect whether the first data is anomaly based on the first anomaly detection model. This deploys the first rule on the first device, and when the first rule needs to be upgraded, the upgrade is centralized on the first device, ensuring the first rule maintains high timeliness.

[0018] In another possible implementation, the first device is a device located on the cloud side, and the second device is a terminal device; or, the first device is a terminal device, and the second device is a device located on the cloud side.

[0019] In another possible implementation, the content of the anomalous data involves fraud.

[0020] Secondly, this application provides a data processing apparatus for performing the method in the first aspect or any possible implementation thereof. Specifically, the apparatus includes units for performing the method in the first aspect or any possible implementation thereof.

[0021] Thirdly, this application provides a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor and a memory;

[0022] The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of the first aspect or any possible implementation thereof.

[0023] Fourthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of the first aspect or any possible implementation thereof.

[0024] Fifthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method of the first aspect or any possible implementation thereof.

[0025] In a sixth aspect, this application provides a chip including a memory and a processor, wherein the memory is used to store computer instructions, and the processor is used to retrieve and execute the computer instructions from the memory to perform the method in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of another network architecture provided in an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of another network architecture provided in an embodiment of this application;

[0029] Figure 4 This is a schematic diagram of a system architecture provided in an embodiment of this application;

[0030] Figure 5 This is a flowchart of a data processing method provided in an embodiment of this application;

[0031] Figure 6 This is a flowchart of another data processing method provided in an embodiment of this application;

[0032] Figure 7 This is a schematic diagram of a data processing component structure provided in an embodiment of this application;

[0033] Figure 8 This is a flowchart of another data processing method provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram of another data processing component structure provided in an embodiment of this application;

[0035] Figure 10 This is a schematic diagram of a data processing device provided in an embodiment of this application;

[0036] Figure 11 This is a schematic diagram of a computing device structure provided in an embodiment of this application;

[0037] Figure 12 This is a schematic diagram of a cluster structure for processing data provided in an embodiment of this application;

[0038] Figure 13 This is a schematic diagram of another data processing cluster structure provided in an embodiment of this application. Detailed Implementation

[0039] In recent years, crimes such as telecommunications fraud and hacking attacks have been on the rise, becoming a prominent problem seriously endangering social security and the safety of people's property. Users may receive abnormal data involving fraud or hacking attacks, resulting in financial losses. Therefore, there is an urgent need for technical means to detect whether users are receiving abnormal data, and to issue alerts and take other actions if abnormal data is detected.

[0040] In order to detect data, embodiments of this application provide a first device, which may include a first rule. The first rule is used to define attribute information of normal data and / or attribute information of abnormal data.

[0041] In some embodiments, the first device may further include a first abnormal data detection model. Thus, the first device can acquire attribute information of the first data to be detected; if the attribute information of the first data matches a first rule, determine whether the first data is normal or abnormal data; if the attribute information of the first data does not match the first rule, detect whether the first data is abnormal data based on the first abnormal data detection model.

[0042] In some embodiments, the first device may further include a second rule for defining an abnormal area, which is an area with a risk level of medium or high.

[0043] If the attribute information of the first data does not match the first rule and the location of the first data's sender belongs to the abnormal area defined by the second rule, the first device can improve the sensitivity of the first abnormal data detection model to detect abnormal data, thereby obtaining a second abnormal data detection model. The device then uses this second abnormal data detection model to determine whether the first data is abnormal. The fact that the location of the first data's sender belongs to the abnormal area defined by the second rule indicates that the attribute information of the first data matches the second rule.

[0044] If the attribute information of the first data does not match the first rule and the location of the sending object of the first data does not belong to the abnormal area defined by the second rule, the first abnormal data detection model is used to detect whether the first data is abnormal data. If the location of the sending object of the first data does not belong to the abnormal area defined by the second rule, it means that the attribute information of the first data also does not match the second rule.

[0045] In this way, the sensitivity of the anomaly detection model is increased to carefully detect the first data from the abnormal region, thereby accurately determining whether the first data is abnormal and improving detection accuracy.

[0046] Optionally, the sensitivity of the first anomaly detection model in detecting anomalous data can be improved by lowering the threshold of the first anomaly detection data model, thus obtaining a second anomaly detection model.

[0047] For example, the first anomaly detection model includes a first threshold. This model is used to obtain the anomaly confidence level of the first data and determines that the first data is anomaly when the anomaly confidence level is greater than or equal to the first threshold. Therefore, by adjusting the first threshold of the first anomaly detection model to a second threshold, which is less than the first threshold, a second anomaly detection model with higher sensitivity for detecting anomaly data is obtained.

[0048] In some embodiments, before using the first anomaly detection model or the second anomaly detection model to detect the first data, the first data may be preprocessed to obtain the second data, which includes a portion of the data in the first data. Then, the first data is detected based on the second data and the first anomaly detection model, or the first data is detected based on the second data and the second anomaly detection model.

[0049] Optionally, the first device can be a server or terminal device, and the server can be a cloud-based computing device. The aforementioned first or second abnormal data detection model is a unimodal or multimodal model.

[0050] In some embodiments, see Figure 1 The first device can communicate with the management device. The administrator can configure the first rule on the management device, and the management device can send the first rule to the first device.

[0051] Administrators can update the first rule periodically or irregularly, and management devices can obtain the latest version of the first rule and update the first rule on the first device to the latest version.

[0052] Optionally, the management device can be a device located on the cloud side, such as a server located on the cloud side.

[0053] In some embodiments, see Figure 2 The first device can communicate with the second device. The first device includes a first rule, and the second device includes a first anomaly detection model. Thus, the first device can obtain attribute information of the first data to be detected; if the attribute information of the first data matches the first rule, it determines whether the first data is normal or abnormal; if the attribute information of the first data does not match the first rule, it sends a detection command to the second device. After receiving the detection command, the second device detects whether the first data is abnormal based on the first anomaly detection model.

[0054] In some embodiments, Figure 2 The network architecture shown can include one or more second devices, meaning the network architecture includes at least one second device. Figure 2 In the network architecture shown, the first device can be a server, and at least one second device can be a terminal device, such as a server located on the cloud side. Alternatively, the first device can be an edge device of a wireless local area network (WLAN).

[0055] Administrators may frequently need to upgrade the primary rule, which resides on the primary device. This allows the management device to upgrade only the primary rule on the primary device, ensuring its timeliness. However, if the primary rule is deployed to each secondary device, during upgrades, the management device might upgrade the primary rule on some devices first, then others, or it might miss upgrading the primary rule on some devices. This delay in upgrading the primary rule on those secondary devices results in poor timeliness.

[0056] In some embodiments, see Figure 3 The number of first devices can be one or more, meaning the network architecture includes at least one first device. Figure 3 In the network architecture shown, at least one first device is a terminal device, and the second device can be a server, such as a server located on the cloud side. Alternatively, the second device can be an edge device of a WLAN.

[0057] Thus, for each first device, if the attribute information of the first data does not match the first rule, the first device sends a detection command to the second device. Upon receiving the detection command, the second device detects whether the first data is abnormal based on the first abnormal data detection model. Since the computational cost of detecting the first data using the first abnormal data detection model is high, while the second device has abundant computing resources, having the second device use the first abnormal data detection model to detect the first data can meet the required computational cost and improve detection efficiency.

[0058] Specifically, if the attribute information of the first data matches the attribute information of normal data defined by the first rule, the first data can be accurately determined to be normal data; if the attribute information of the first data matches the attribute information of abnormal data defined by the first rule, the first data can be accurately determined to be abnormal data. The computational complexity of matching the attribute information of the first data with the first rule is very low, so the latency required for matching the attribute information of the first data with the first rule is very short. This not only accurately detects whether the first data is abnormal data, but also reduces computational overhead and the latency of detecting the first data.

[0059] If the attribute information of the first data does not match the first rule, it means that the first rule may not be able to accurately detect whether the first data is normal or abnormal. In order to avoid false detection or missed detection, the first abnormal data detection model is used to detect whether the first data is abnormal, thereby improving the accuracy of the first data detection.

[0060] In some embodiments, the present application can be applied to the field of anti-telecom network fraud technology, where the content of the abnormal data involves fraud.

[0061] For example, see Figure 4 The first abnormal data detection model can be an artificial intelligence (AI) fraud detection module. In order to detect abnormal data, it can also include a rule filtering module, a data preprocessing module, and / or a dynamic threshold module.

[0062] The rule filtering module includes a fraud-related rule base and a rule engine. The fraud-related rule base includes the aforementioned first rule and / or second rule. The rule engine matches the attribute information of the first data to be detected against the rules in the fraud-related rule base. If the attribute information of the first data matches the first rule, the first data is determined to be normal or abnormal data. If the attribute information of the first data does not match any of the first rules but matches the second rule, the first data is preprocessed using a data preprocessing module to obtain the second data. A dynamic threshold module is used to adjust the sensitivity of the AI ​​fraud detection module in detecting abnormal data. The adjusted AI fraud detection module is then used to detect the second data to determine whether the first data is abnormal data.

[0063] Optionally, the data preprocessing module obtains the second data by cleaning and / or filtering the first data. The AI ​​fraud detection module includes a voice AI module, an image AI module, a webpage AI module, and a behavior AI module, which are used to detect whether the first data is abnormal.

[0064] Optionally, the dynamic threshold module adjusts the threshold of the AI ​​fraud detection module through dynamic threshold adjustment. Furthermore, when the first piece of data is detected as abnormal, a threshold grading strategy can be used to issue an alert for that data.

[0065] In some embodiments, the first data is data received by the user when communicating with a communication partner who may be engaged in activities such as fraud or hacking, making the first data data that involves fraud or hacking.

[0066] For example, when a user uses a mobile phone or other communication tool to make a voice call, the first data is the audio data sent by the other party during the voice call.

[0067] For example, when a user uses a mobile phone or other communication tool to make a video call with another party, the first data is the video data sent by the other party during the video call.

[0068] For example, when a user communicates with another party using a chat application or email, the first data is the data sent by the other party using the chat application or email. This data can include one or more of the following: text data, images, video data, or audio data, etc. Text data includes data containing multiple characters, and can be chat messages, web pages, email content, or text files, etc.

[0069] To improve the efficiency of detection data, reduce the latency and overhead of detection data, avoid missed detections and false detections, and improve the accuracy of detection data, this application can process data through any of the following embodiments.

[0070] See Figure 5 This application provides a data processing method 500, which can be applied to a first device. The first device includes a first rule and a first abnormal data detection module. The first rule is used to define attribute information of normal data and / or attribute information of abnormal data. The first device can be... Figure 1 The first device in the network architecture shown. If the attribute information of the first data to be detected matches the first rule, the first device determines that the first data is normal data or abnormal data; if the attribute information of the first data to be detected does not match the first rule, the first device detects whether the first data is abnormal data based on the first abnormal data detection model. The method 500 includes the following process.

[0071] Step 501: The first device acquires the attribute information of the first data to be detected.

[0072] In some embodiments, the first device may be a user's terminal device, and the first device may receive first data sent by the communication device corresponding to the communication counterpart.

[0073] In some embodiments, the first device can be a device other than a terminal device, such as a server (e.g., a server located in the cloud). The user's terminal device receives first data sent by the communication device corresponding to the communication counterpart, sends first data to the first device, and the first device receives the first data.

[0074] In some embodiments, the attribute information of the first data includes one or more of the following: source information of the first data, identification information of the first data, or the location of the object to which the first data is sent, etc.

[0075] Optionally, see Figure 6 The first data is either audio data from a voice call between the user and the other party, or video data from a video call between the user and the other party. The attribute information of the first data may include source information and / or the location of the recipient of the first data. The source information includes the telephone number of the other party, the recipient of the first data can be the telephone number of the other party, and the location of the recipient of the first data is the location of that telephone number.

[0076] Optionally, the first data is data used by the user to communicate with the other party using a chat application. The first data can be chat information and includes one or more of the following: text data, images, video data, or audio data, etc. The attribute information of the first data may include the source information of the first data and / or the location of the recipient of the first data. The source information of the first data includes the chat account of the other party, the recipient of the first data is the other party, and the location of the recipient of the first data is the location of the address used by the recipient, which can be an Internet Protocol (IP) address or a Media Access Control (MAC) address, etc.

[0077] Optionally, the first data is data from communication between the user and the other party using email. The first data can be emails, including email content, etc. The attribute information of the first data may include source information, such as the email address of the other party.

[0078] Optionally, the first data may be a webpage, etc., and the attribute information of the first data may include the identification information of the first data, which may be the address of the webpage.

[0079] Step 502: The first device matches the attribute information of the first data with the first rule included in the first device. If the attribute information of the first data matches the first rule, then proceed to step 503. If the attribute information of the first data does not match the first rule, then proceed to step 504.

[0080] The first rule in the first device may be issued by the management device. The first rule is used to define the attribute information of normal data and / or the attribute information of abnormal data.

[0081] In some embodiments, the first rule includes a first sub-rule and / or a second sub-rule, wherein the first sub-rule is used to define the attribute information of normal data and the second sub-rule is used to define the attribute information of abnormal data.

[0082] Optionally, the first sub-rule includes source information of at least one normal data, and the second sub-rule includes source information of at least one abnormal data; or, the first sub-rule includes identification information of at least one normal data, and the second sub-rule includes identification information of at least one abnormal data.

[0083] For example, the first sub-rule is a number whitelist, including attribute information for at least one normal data item. For instance, the first sub-rule could be a number whitelist that includes at least one phone number and / or at least one chat account. The at least one phone number could include the numbers of the user's relatives or friends and / or numbers from business directories, and the at least one chat account could include the chat accounts of the user's relatives or friends. Alternatively, the first sub-rule could be an address whitelist that includes the address of at least one webpage, which could be the address of a webpage from a major portal website.

[0084] For example, the second sub-rule could be a number blacklist, including attribute information of at least one abnormal data. For instance, the second sub-rule could be a number blacklist including at least one phone number and / or at least one chat account, where the at least one phone number might be a phone number collected by the management device for engaging in illegal activities, and the at least one chat account might be a chat account collected by the management device for engaging in illegal activities. Alternatively, the second sub-rule could be an address blacklist including the address of at least one webpage, where the at least one webpage might be a webpage collected by the management device for containing illegal content, etc.

[0085] In some embodiments, the management device may also issue a second rule to the first device, which is used to define abnormal regions. The second rule can be a region blacklist, including at least one abnormal region.

[0086] In some embodiments, the management device may periodically or irregularly update the first rule and / or the second rule in the first device. For example, by adding attribute information of normal data and / or attribute information of abnormal data as defined in the first rule to obtain the latest version of the first rule, the management device can update the first rule in the first device to the latest version of the first rule, and / or by adding abnormal regions as defined in the second rule to obtain the latest version of the second rule, the management device can update the second rule in the first device to the latest version of the second rule.

[0087] In step 502, the following process can be used to determine whether the attribute information of the first data matches the first rule.

[0088] 5021: The first device determines whether the attribute information of the normal data defined by the first rule includes the attribute information of the first data. If the attribute information of the normal data defined by the first rule includes the attribute information of the first data, it can be concluded that the attribute information of the first data matches the first rule, that is, the first data is normal data.

[0089] In 5021, the first device determines whether the attribute information of the normal data defined by the first sub-rule includes the attribute information of the first data. If the attribute information of the normal data defined by the first sub-rule includes the attribute information of the first data, then it can be concluded that the first data is normal data.

[0090] Optionally, the attribute information of the first data includes the source information of the first data. The first device determines whether the source information of the normal data defined by the first sub-rule includes the source information of the first data. If the source information of the normal data defined by the first sub-rule includes the source information of the first data, then it can be concluded that the first data is normal data.

[0091] For example, see Figure 7 The first sub-rule is a whitelist of phone numbers (such as a business directory). The attribute information of the first data includes the phone number or chat account corresponding to the first data. If the whitelist of phone numbers includes the phone number or chat account corresponding to the first data, it can be concluded that the first data is normal data.

[0092] Optionally, the attribute information of the first data includes the identification information of the first data. The first device determines whether the identification information of the normal data defined by the first sub-rule includes the identification information of the first data. If the identification information of the normal data defined by the first sub-rule includes the identification information of the first data, then it can be concluded that the first data is normal data.

[0093] For example, the first sub-rule is an address whitelist. The attribute information of the first data includes the address of the first data (such as a webpage). If the address whitelist includes the address of the first data, it can be concluded that the first data is normal data.

[0094] 5022: If the attribute information of normal data defined by the first rule does not include the attribute information of the first data, the first device determines whether the attribute information of abnormal data defined by the first rule includes the attribute information of the first data. If the attribute information of abnormal data defined by the first rule includes the attribute information of the first data, it can be concluded that the attribute information of the first data matches the first rule, that is, the first data is abnormal data.

[0095] In 5022, the first device determines whether the attribute information of the abnormal data defined by the second sub-rule includes the attribute information of the first data. If the attribute information of the abnormal data defined by the second sub-rule includes the attribute information of the first data, then it can be concluded that the first data is abnormal data.

[0096] Optionally, the attribute information of the first data includes the source information of the first data. The first device determines whether the source information of the abnormal data defined by the second sub-rule includes the source information of the first data. If the source information of the abnormal data defined by the second sub-rule includes the source information of the first data, then it can be concluded that the first data is abnormal data.

[0097] For example, see Figure 6 or Figure 7The second sub-rule is the number blacklist. The attribute information of the first data includes the phone number or chat account corresponding to the first data. If the number blacklist includes the phone number or chat account corresponding to the first data, it can be concluded that the first data is abnormal data.

[0098] Optionally, the attribute information of the first data includes the identification information of the first data. The first device determines whether the identification information of the abnormal data defined by the second sub-rule includes the identification information of the first data. If the identification information of the abnormal data defined by the second sub-rule includes the identification information of the first data, then it can be concluded that the first data is abnormal data.

[0099] For example, the second sub-rule is an address blacklist. The attribute information of the first data includes the address of the first data (such as a webpage). If the address blacklist includes the address of the first data, it can be concluded that the first data is abnormal data.

[0100] 5023: If the attribute information of the abnormal data defined by the first rule does not include the attribute information of the first data, then it can be concluded that the attribute information of the first data does not match the first rule.

[0101] If the attribute information of the first data does not match the first rule, it is impossible to determine whether the first data is normal or abnormal. Therefore, it is necessary to use the first abnormal data detection model to detect the first data in order to determine whether the first data is normal or abnormal.

[0102] Step 503: The first device determines whether the first data is normal or abnormal data, and then ends.

[0103] If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data. If the attribute information of the first data matches the second sub-rule, the first data is determined to be abnormal data.

[0104] Step 504: The first device detects whether the first data is abnormal based on the first abnormal data detection model, and then ends.

[0105] The first abnormal data detection model includes a first threshold.

[0106] In step 504, the first device inputs the first data into the first anomaly detection model. The first anomaly detection model infers the anomaly confidence level of the first data. If the anomaly confidence level of the first data is greater than or equal to a first threshold, the first data is determined to be anomaly data; if the anomaly confidence level of the first data is less than the first threshold, the first data is determined to be normal data. Then, the first device obtains the detection result output by the first anomaly detection module, which indicates whether the first data is normal data or anomaly data.

[0107] In some embodiments, when the first data is abnormal, the first device is the user's terminal device, and the first device can issue an alarm to the user. Alternatively, the first device is a device other than the user's terminal device (such as a server or a WLAN edge device), and the first device can notify the user's terminal device, which can then issue an alarm to the user. Optionally, a hierarchical alarm method can be used.

[0108] Optionally, the alert method may include one or more of the following: reminder method, pop-up method, or blocking method.

[0109] The reminder method refers to the first device displaying a reminder message in the notification bar, which is used to indicate that the first data is abnormal data.

[0110] The pop-up method refers to the first device display window, which includes processing operations. These operations can add the phone number or chat number corresponding to the first data to the number blacklist, or add the address corresponding to the first data to the address blacklist, etc.

[0111] The blocking method refers to the first device being able to send a notification message to the user's relatives and friends, which is used to notify the relatives and friends to block the user from communicating with the other party.

[0112] In some embodiments, the first device further includes a second rule for defining at least one abnormal region. The attribute information of the first data also includes the location of the sending object corresponding to the first data.

[0113] If the attribute information of the first data does not match the first rule, the first device further determines whether the attribute information of the first data matches the second rule. If the attribute information of the first data matches the second rule, the sensitivity of the first anomaly detection model in detecting anomaly data is increased to obtain a second anomaly detection model. The first data is then used to detect whether it is anomaly data based on the second anomaly detection model. Alternatively, if the attribute information of the first data does not match the second rule, the first anomaly detection model is used to detect whether the first data is anomaly data.

[0114] Optionally, the first device determines whether the location of the sending object of the first data belongs to the abnormal area defined by the second rule. If the location belongs to the abnormal area defined by the second rule, it is concluded that the attribute information of the first data matches the second rule; if the location does not belong to the abnormal area defined by the second rule, it is concluded that the attribute information of the first data does not match the second rule.

[0115] In some embodiments, the first device can adjust the first threshold of the first anomaly detection model to a second threshold to obtain a second anomaly detection model. The second threshold is less than the first threshold, thus achieving a second anomaly detection model with higher sensitivity for detecting anomaly data. In step 504, the first device inputs the first data into the second anomaly detection model. The second anomaly detection model infers the anomaly confidence level of the first data. If the anomaly confidence level of the first data is greater than or equal to the second threshold, the first data is determined to be anomaly data; if the anomaly confidence level is less than the second threshold, the first data is determined to be normal data. Then, the first device obtains the detection result output by the second anomaly detection module, which indicates whether the first data is normal data or anomaly data.

[0116] The first device obtains a second anomaly detection model with higher sensitivity for detecting anomaly data by lowering the threshold of the first anomaly detection model. Optionally, the software version can be obtained; this software can be communication software used by the user, and the first data is data received using this communication software. The threshold of the first anomaly detection model is lowered based on the software version.

[0117] For example, the higher the software version, the greater the reduction in threshold, and the lower the software version, the smaller the reduction in threshold.

[0118] In some embodiments, before detecting the first data, the first data may be preprocessed to obtain the second data, the second data including a portion of the first data, and the first data may be detected as abnormal based on the first abnormal data detection model and the second data, or the first data may be detected as abnormal based on the second abnormal data detection model and the second data.

[0119] In some embodiments, the second data includes a portion of the first data in which the quality meets the quality conditions.

[0120] For example, the first data is audio data, and the second data is the portion of the first data where the noise intensity is below a third threshold.

[0121] Optionally, in implementation, the first device can detect the noise intensity in the audio data and extract a segment of audio data with a noise intensity lower than a third threshold as the second data.

[0122] Optionally, the first device may extract a segment of audio data with a noise intensity lower than a third threshold and a length greater than or equal to N from the audio data as the second data, where N is an integer greater than 1.

[0123] Optionally, the first device can convert the audio data into multiple texts and extract a segment of audio data with noise intensity below a third threshold and the number of texts included being greater than or equal to N as the second data.

[0124] For example, the first data is video data, and the second data is the data in the first data where the contrast is below the fourth threshold, the light intensity is below the fifth threshold, and / or includes the portion of the target image that is not occluded.

[0125] Optionally, in implementation, the first device may detect the contrast, light intensity and / or image content of each frame of video image in the video data, and extract multiple frames of video images from the video data whose contrast is lower than a fourth threshold, whose light intensity is lower than a fifth threshold and / or whose target image is not occluded, and use these multiple frames of video images as the second data.

[0126] Optionally, the target image can be an image of the communicating party, etc.

[0127] For example, the first data is an image, and the second data includes the target image in the image or data obtained by expanding the edges of the target image.

[0128] The image is large, while the target image within it is small. The first device can crop the image to obtain the target image and use it as the second data. Alternatively, the edges of the target image can be expanded, and the expanded target image can be used as the second data.

[0129] In some embodiments, the first data is text data (such as a webpage) including multiple characters, and the second data includes the first M characters in the first data, where M is an integer greater than 1. Optionally, M can be equal to a value such as 200, 210, or 220.

[0130] In some embodiments, the content of the aforementioned abnormal data involves fraud or hacking attacks. That is, the embodiments of this application can be applied to the field of anti-telecom network fraud technology. The first device adopts a lightweight fixed-rule filtering mechanism to quickly return non-fraudulent scenarios and quickly alert users to fraudulent scenarios. The first device uses a front-end filtering mechanism and data preprocessing as the detection entry point for the abnormal data detection model. That is, the first device can filter through front-end rules and low-quality data, and the data does not need to be sent to the back-end abnormal data detection model for deep detection every time. Thus, through multi-module collaborative detection between the front-end rules (first rule and / or second rule) and the back-end abnormal data detection model to determine whether the first data is abnormal (whether it involves fraud or hacking attacks), the accuracy of detecting abnormal data can be improved, the computational overhead required for detection can be reduced, the latency required for detection can be reduced, and resource waste can be reduced. The risk of fraud is comprehensively determined using rules and the abnormal data detection model. The risk threshold (first threshold and second threshold) is adjustable on the cloud side. Multi-module collaborative fraud determination allows the risk threshold to be adjusted in real time according to operational conditions.

[0131] In some embodiments, the first abnormal data detection model is obtained by training an AI model with at least one training sample, the at least one training sample including at least one first training sample, each first training sample including third data and first labeling information, the first labeling information being used to label the third data as abnormal data.

[0132] Optionally, the at least one training sample further includes at least one second training sample, each second training sample including fourth data and second labeling information, the second labeling information being used to label the fourth data as normal data.

[0133] In this embodiment, the first device includes a first rule and a first abnormal data detection model. The first device may be a terminal device or a server (such as a server located on the cloud side). The first device acquires the attribute information of the first data and matches the attribute information of the first data with the first rule. If the attribute information of the first data matches the first rule, the first device determines whether the first data is normal or abnormal. This not only improves the accuracy of detecting whether the first data is abnormal, but also reduces the latency required for detection, improves detection efficiency, and reduces the overhead required for detecting the first data. If the attribute information of the first data does not match the first rule, the first device detects whether the first data is normal or abnormal based on the first abnormal data detection model. This improves the accuracy of detecting whether the first data is abnormal. The abnormal data detection model is only used to detect the first data when the attribute information of the first data does not match the first rule, reducing the number of times the abnormal data detection model is used to detect the data and reducing resource consumption.

[0134] See Figure 8 This application provides a data processing method 800, which can be applied to a network architecture including a first device and a second device. The first device includes a first rule, and the second device includes a first abnormal data detection module. The first rule is used to define attribute information of normal data and / or attribute information of abnormal data. This network architecture can be... Figure 2 or Figure 3 The network architecture is shown. If the attribute information of the first data to be detected matches the first rule, the first device determines that the first data is normal data or abnormal data; if the attribute information of the first data to be detected does not match the first rule, the first device sends a detection command to the second device, and the second device, after receiving the detection command, detects whether the first data is abnormal data based on the first abnormal data detection model. The method 800 includes the following process.

[0135] Step 801: The first device acquires the attribute information of the first data to be detected.

[0136] In some embodiments, the second device is a terminal device, and the first device can be a device other than a terminal device, such as a server (e.g., a server located on the cloud side) or an edge device of a WLAN. Figure 9 As shown, the second device receives the first data sent by the communication device corresponding to the communication counterpart, sends the first data to the first device, and the first device receives the first data.

[0137] In some embodiments, the first device may be a user's terminal device, which may receive first data sent by the communication device corresponding to the other party in the communication, while the second device may be a device other than the terminal device, such as a server (e.g., a server located on the cloud side) or an edge device of a WLAN.

[0138] In some embodiments, the attribute information of the first data includes one or more of the following: source information of the first data, identification information of the first data, or the location of the object to which the first data is sent, etc.

[0139] Optionally, see Figure 6 The first data is either audio data from a voice call between the user and the other party, or video data from a video call between the user and the other party. The attribute information of the first data may include source information and / or the location of the recipient of the first data. The source information includes the telephone number of the other party, the recipient of the first data can be the telephone number of the other party, and the location of the recipient of the first data is the location of that telephone number.

[0140] Optionally, the first data is data used by the user to communicate with the other party using a chat application. The first data can be chat information and includes one or more of the following: text data, images, video data, or audio data, etc. The attribute information of the first data may include the source information of the first data and / or the location of the recipient of the first data. The source information of the first data includes the chat account of the other party, the recipient of the first data is the other party, and the location of the recipient of the first data is the location of the address used by the recipient, which can be an IP address or a MAC address, etc.

[0141] Optionally, the first data is data from communication between the user and the other party using email. The first data can be emails, including email content, etc. The attribute information of the first data may include source information, such as the email address of the other party.

[0142] Optionally, the first data may be a webpage, etc., and the attribute information of the first data may include the identification information of the first data, which may be the address of the webpage.

[0143] Step 802: The first device matches the attribute information of the first data with the first rule included in the first device. If the attribute information of the first data matches the first rule, then proceed to step 803. If the attribute information of the first data does not match the first rule, then proceed to step 804.

[0144] The detailed implementation process of step 802 and Figure 5 The detailed implementation process of step 502 shown is the same, and will not be described in detail here.

[0145] Step 803: The first device determines whether the first data is normal or abnormal data.

[0146] If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data. If the attribute information of the first data matches the second sub-rule, the first data is determined to be abnormal data.

[0147] Step 804: The first device sends a detection command to the second device. The detection command is used to instruct the second device to detect whether the first data is abnormal data.

[0148] In some embodiments, the first device sends a detection instruction to the second device, the detection instruction including first data. Alternatively, the first device may preprocess the first data to obtain second data, the second data including a portion of the first data, and then send a detection instruction to the second device, the detection instruction including the second data.

[0149] For details on the preprocessing of the first data by the first device, please refer to [link / reference needed]. Figure 5 The details of step 504 in the method shown will not be explained in detail here.

[0150] In some embodiments, the first device further includes a second rule for defining at least one abnormal region. The attribute information of the first data also includes the location of the sending object corresponding to the first data.

[0151] If the attribute information of the first data does not match the first rule, the first device further determines whether the attribute information of the first data matches the second rule. If the attribute information of the first data does not match the second rule, a detection instruction is sent to the second device. The detection instruction includes either the first data or the second data. If the attribute information of the first data matches the second rule, a detection instruction is sent to the second device. In addition to including either the first data or the second data, the detection instruction also includes an adjustment instruction, which is used to instruct the first abnormal data detection model to improve its sensitivity in detecting abnormal data.

[0152] The detailed implementation process of the first device determining whether the attribute information of the first data matches the second rule can be found in [link to relevant documentation]. Figure 5 The details of step 504 in the method shown will not be explained in detail here.

[0153] Step 805: The second device receives the detection instruction and detects whether the first data is abnormal based on the first abnormal data detection model.

[0154] The first abnormal data detection model includes a first threshold.

[0155] In step 805, the detection instruction includes first data. The second device inputs the first data into a first anomaly detection model. The first anomaly detection model infers the anomaly confidence level of the first data. If the anomaly confidence level of the first data is greater than or equal to a first threshold, the first data is determined to be anomaly data; if the anomaly confidence level of the first data is less than the first threshold, the first data is determined to be normal data. Then, the second device obtains the detection result output by the first anomaly detection module, which indicates that the first data is normal data or the first data is anomaly data. Alternatively,

[0156] The detection instruction includes second data. The second device detects whether the first data is abnormal based on the second data and a first abnormal data detection model. Optionally, in implementation: the second device inputs the second data into the first abnormal data detection model. The first abnormal data detection model infers the anomaly confidence level of the second data. If the anomaly confidence level of the second data is greater than or equal to a first threshold, the second data is determined to be abnormal, i.e., the first data is determined to be abnormal. If the anomaly confidence level of the second data is less than the first threshold, the second data is determined to be normal, i.e., the second data is determined to be normal. Then, the second device obtains the detection result output by the first abnormal data detection module, which indicates whether the first data is normal or abnormal.

[0157] In some embodiments, if the detection instruction includes first data, the second device may preprocess the first data to obtain second data, and then detect whether the first data is abnormal based on the second data and the first abnormal data detection model.

[0158] For details on the preprocessing of the first data by the second device, please refer to [link / reference needed]. Figure 5 The details of step 504 in the method shown will not be explained in detail here.

[0159] In some embodiments, if the detection instruction includes an adjustment instruction in addition to the first data or the second data, the second device improves the sensitivity of the first abnormal data detection model to detect abnormal data based on the adjustment instruction to obtain a second abnormal data detection model, and detects whether the first data is abnormal based on the second abnormal data detection model.

[0160] In some embodiments, the second device can adjust the first threshold of the first abnormal data detection model to a second threshold to obtain a second abnormal data detection model. The second threshold is less than the first threshold, thereby obtaining a second abnormal data detection model with higher sensitivity in detecting abnormal data.

[0161] In step 805, the second device inputs the first data into the second anomaly detection model. The second anomaly detection model infers the anomaly confidence level of the first data. If the anomaly confidence level of the first data is greater than or equal to a second threshold, the first data is determined to be anomaly data; if the anomaly confidence level of the first data is less than the second threshold, the first data is determined to be normal data. Then, the second device obtains the detection result output by the second anomaly detection module, which indicates whether the first data is normal or anomaly data. Alternatively,

[0162] The second device inputs the second data into the second anomaly detection model. The second anomaly detection model infers the anomaly confidence level of the second data. If the anomaly confidence level of the second data is greater than or equal to a second threshold, the second data is determined to be anomaly data, i.e., the first data is determined to be anomaly data. If the anomaly confidence level of the second data is less than the second threshold, the second data is determined to be normal data, i.e., the first data is determined to be normal data. Then, the second device obtains the detection result output by the second anomaly detection module, which indicates whether the first data is normal data or anomaly data.

[0163] In some embodiments, when the first data is abnormal, the second device is the user's terminal device, and the second device can issue an alert to the user. Alternatively, the second device can be a device other than a terminal device, the first device is the user's terminal device, the second device notifies the first device, and the first device can issue an alert to the user. The alerting method can be one or more of the following: notification method, pop-up method, or blocking method, etc.

[0164] In some embodiments, the content of the aforementioned abnormal data may involve fraud or hacking attacks.

[0165] In this embodiment, the first device includes a first rule, and the second device includes a first abnormal data detection model. The first device may be a terminal device, and the second device may be a server (such as a server located in the cloud), or the first device may be a server (such as a server located in the cloud), and the second device may be a terminal device. The first device acquires the attribute information of the first data and matches the attribute information of the first data with the first rule. If the attribute information of the first data matches the first rule, the first device determines that the first data is normal data or abnormal data. This not only improves the accuracy of detecting whether the first data is abnormal, but also reduces the latency required for detection, improves detection efficiency, and reduces the overhead required for detecting the first data. If the attribute information of the first data does not match the first rule, a detection command is sent to the second device. The second device detects whether the first data is normal data or abnormal data based on the first abnormal data detection model. This improves the accuracy of detecting whether the first data is abnormal. Since the second device has more computing resources, the second device can detect the first data faster based on the first abnormal data detection model, thus improving detection efficiency. Alternatively, the first device can be a server and the second device a terminal device. This way, when the first rule changes, the first rule on the first device can be directly updated, ensuring that the first rule always maintains high timeliness. The anomaly detection model is only used to detect the first data if its attribute information does not match the first rule, reducing the number of times the anomaly detection model is used and thus reducing resource consumption.

[0166] See Figure 10 This application provides a data processing apparatus 1000, which includes a first rule for defining attribute information of normal data and / or attribute information of abnormal data. The apparatus 1000 can be deployed in... Figure 1 , Figure 2 or Figure 3 On the first device shown in the embodiment, or, it can be deployed on Figure 5 or Figure 8 On the first device in the method embodiment shown. The device 1000 further includes:

[0167] Processing unit 1001 is used to acquire attribute information of the first data to be detected;

[0168] The processing unit 1001 is also used to determine whether the first data is normal data or abnormal data when the attribute information of the first data matches the first rule;

[0169] The processing unit 1001 is also used to detect whether the first data is abnormal data based on the first abnormal data detection model when the attribute information of the first data does not match the first rule.

[0170] Optionally, the detailed implementation process of the processing unit 1001 acquiring the attribute information of the first data to be detected can be found in [reference needed]. Figure 5 Step 501 of method 500 shown Figure 8 The relevant content in step 801 of method 800 shown will not be described in detail here.

[0171] Optionally, the detailed implementation process of the processing unit 1001 determining whether the first data is normal or abnormal data when the attribute information of the first data matches the first rule can be found in [reference needed]. Figure 5 Step 503 of method 500 shown Figure 8 The relevant content in step 803 of method 800 shown will not be described in detail here.

[0172] Optionally, if the attribute information of the first data does not match the first rule, the processing unit 1001 can refer to the detailed implementation process of detecting whether the first data is abnormal data based on the first abnormal data detection model. Figure 5 Step 504 of method 500 shown Figure 8 The relevant content in step 805 of method 800 shown will not be described in detail here.

[0173] Optionally, the first rule includes a first sub-rule and / or a second sub-rule, wherein the first sub-rule is used to define the attribute information of normal data and the second sub-rule is used to define the attribute information of abnormal data;

[0174] Processing unit 1001 is used for:

[0175] If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data;

[0176] If the attribute information of the first data matches the second sub-rule, the first data is determined to be abnormal data.

[0177] Optionally, the detailed implementation process of the processing unit 1001 determining that the first data is normal data when the attribute information of the first data matches the first sub-rule can be found in [reference needed]. Figure 5 Step 503 of method 500 shown Figure 8 The relevant content in step 803 of method 800 shown will not be described in detail here.

[0178] Optionally, the detailed implementation process of the processing unit 1001 determining that the first data is abnormal data when the attribute information of the first data matches the second sub-rule can be found in [reference needed]. Figure 5 Step 503 of method 500 shown Figure 8 The relevant content in step 803 of method 800 shown will not be described in detail here.

[0179] Optionally, the attribute information of the first data includes the source information of the first data, the first sub-rule includes the source information of at least one normal data, and the second sub-rule includes the source information of at least one abnormal data; or,

[0180] The attribute information of the first data includes the identification information of the first data, the first sub-rule includes the identification information of at least one normal data, and the second sub-rule includes the identification information of at least one abnormal data.

[0181] Optionally, the device 1000 further includes a second rule, which is used to define an abnormal region. The attribute information of the first data includes the location of the sending object of the first data. The processing unit 1001 is used for:

[0182] When the location belongs to an abnormal area, the sensitivity of the first abnormal data detection model to detect abnormal data is improved to obtain a second abnormal data detection model.

[0183] The first data is used to detect whether it is abnormal based on the second abnormal data detection model.

[0184] Optionally, the processing unit 1001 improves the sensitivity of the first anomaly detection model to detect anomaly data to obtain a second anomaly detection model. For a detailed implementation process, please refer to [link to relevant documentation]. Figure 5 Step 504 of method 500 shown Figure 8The relevant content in step 805 of method 800 shown will not be described in detail here.

[0185] Optionally, the detailed implementation process of the processing unit 1001 detecting whether the first data is abnormal data based on the second abnormal data detection model can be found in [reference needed]. Figure 5 Step 504 of method 500 shown Figure 8 The relevant content in step 805 of method 800 shown will not be described in detail here.

[0186] Optionally, the first anomaly detection model includes a first threshold. The first anomaly detection model is used to obtain the anomaly confidence of the first data and determine that the first data is anomaly data when the anomaly confidence is greater than or equal to the first threshold.

[0187] Processing unit 1001 is used to adjust the first threshold of the first abnormal data detection model to a second threshold to obtain a second abnormal data detection model, wherein the second threshold is less than the first threshold.

[0188] Optionally, the processing unit 1001 adjusts the first threshold of the first anomaly detection model to a second threshold to obtain the detailed implementation process of the second anomaly detection model, which can be found in [reference needed]. Figure 5 Step 504 of method 500 shown Figure 8 The relevant content in step 805 of method 800 shown will not be described in detail here.

[0189] Optionally, the processing unit 1001 is used for:

[0190] The first data is preprocessed to obtain the second data, which includes a portion of the first data.

[0191] The first abnormal data is detected based on the first abnormal data detection model and the second data to determine whether the first data is abnormal.

[0192] Optionally, the detailed implementation process of the processing unit 1001 preprocessing the first data to obtain the second data can be found in [reference needed]. Figure 5 Step 504 of method 500 shown Figure 8 The relevant content in step 805 of method 800 shown will not be described in detail here.

[0193] Optionally, the detailed implementation process of the processing unit 1001 in detecting whether the first data is abnormal based on the first abnormal data detection model and the second data can be found in [reference needed]. Figure 5 Step 504 of method 500 shown Figure 8 The relevant content in step 805 of method 800 shown will not be described in detail here.

[0194] Optionally, the second data includes the portion of the first data where the quality meets the quality conditions.

[0195] Optionally, the first data is audio data, and the second data is the portion of the first data whose noise intensity is below a third threshold; or,

[0196] The first data is video data, and the second data is the portion of the first data where the contrast is below the fourth threshold, the illumination intensity is below the fifth threshold, and / or includes the unoccluded portion of the target image; or...

[0197] The first data is the image, and the second data includes the target image in the image or data obtained by expanding the edges of the target image.

[0198] Optionally, the first data is text data containing multiple characters, and the second data includes the first M characters in the first data, where M is an integer greater than 1.

[0199] Optionally, the device 1000 further includes:

[0200] The sending unit 1002 is used to send a detection command to the second device, the second device including a first data anomaly detection model, and the detection command is used to instruct the second device to detect whether the first data is abnormal based on the first abnormal data detection model.

[0201] Optionally, the detailed implementation process of the sending unit 1002 sending the detection command to the second device can be found in [reference needed]. Figure 8 The relevant content in step 804 of method 800 shown will not be described in detail here.

[0202] Optionally, the device 1000 is a device located on the cloud side, and the second device is a terminal device; or, the device 1000 is a terminal device, and the second device is a device located on the cloud side.

[0203] Optionally, the abnormal data may involve fraud.

[0204] Both the processing unit 1001 and the sending unit 1002 can be implemented in software or in hardware. For example, the implementation of the processing unit 1001 will be described below. Similarly, the implementation of the sending unit 1002 can refer to the implementation of the processing unit 1001.

[0205] As an example of a software functional unit, processing unit 1001 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the aforementioned computing instance may be one or more. For example, processing unit 1001 may include code running on multiple hosts / virtual machines / containers. The multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0206] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0207] As an example of a hardware functional unit, the processing unit 1001 may include at least one computing device, such as a server. Alternatively, the processing unit 1001 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0208] The processing unit 1001 includes multiple computing devices that can be distributed in the same region or in different regions. Similarly, the processing unit 1001 includes multiple computing devices that can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the processing unit 1001 includes multiple computing devices that can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0209] It should be noted that, in other embodiments, the processing unit 1001 can be used to execute any step in the data processing method, and the sending unit 1002 can be used to execute any step in the data processing method. The steps implemented by the sending unit 1002 and the processing unit 1001 can be specified as needed. By implementing different steps in the data processing method through the sending unit 1002 and the processing unit 1001 respectively, all functions of the data processing device 1000 can be realized.

[0210] In this embodiment, when the attribute information of the first data matches the first rule, the processing unit can accurately determine whether the first data is abnormal or normal. This ensures the accuracy of detecting whether the first data is abnormal without relying on a model, reducing computational overhead and latency. Conversely, when the attribute information of the first data does not match the first rule, it indicates that the first rule cannot accurately determine whether the first data is normal or abnormal. In this case, the processing unit uses the first abnormal data detection model to detect the first data, avoiding false positives and false negatives, and improving the accuracy of detecting whether the first data is abnormal.

[0211] See Figure 11 This application provides a computing device 1100. For example, the computing device 1100 can be... Figure 1 , Figure 2 or Figure 3 The first device in the illustrated embodiment, or the computing device 1100, can be... Figure 5 or Figure 8 The first device in the method embodiment shown. The computing device 1100 may be a device located on the cloud side, for example, it may be a device in a cluster of computing devices located on the cloud side.

[0212] like Figure 11As shown, the computing device 1100 includes a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0213] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 11 The bus 1102 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (e.g., processor 1104, memory 1106, communication interface 1108).

[0214] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0215] The memory 1106 may include volatile memory, such as random access memory (RAM). The memory 1106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0216] See Figure 11 The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the following respectively. Figure 10The processing unit 1001 and the sending unit 1002 in the illustrated device 1000 perform their functions to implement the method provided in any of the above embodiments. That is, the memory 1106 stores instructions for executing the method provided in any of the above embodiments. Alternatively,

[0217] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0218] This application also provides a data processing cluster. The data processing cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0219] like Figure 12 As shown, the data processing cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the data processing cluster may store the same instructions for executing the data processing method provided in any of the above embodiments. In some possible implementations, the memory 1106 of one or more computing devices 1100 in the data processing cluster may also each store partial instructions for executing the above data processing method. In other words, a combination of one or more computing devices 1100 can jointly execute the instructions for executing the method provided in any of the above embodiments.

[0220] The memory 1106 in different computing devices 1100 within a computing device cluster can store different instructions, each used to execute, such as... Figure 10 This describes some of the functions of the data processing device 1000 shown. That is, the instructions stored in the memory 1106 of different computing devices 1100 can implement the functions of one or more units in the processing unit 1001 and the sending unit 1002.

[0221] In some possible implementations, one or more computing devices in the data processing cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 One possible implementation is shown. For example... Figure 13 As shown, the two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device.

[0222] In this type of possible implementation, the memory 1106 in the computing device 1100A stores the execution of, for example Figure 10Instructions for the function of the processing unit 1001 in the illustrated embodiment. Meanwhile, the memory 1106 in the computing device 1100B stores instructions for executing such... Figure 10 Instructions for the function of the sending unit 1002 in the illustrated embodiment.

[0223] Figure 13 The connection method between the computing device clusters shown may take into account the data processing method provided in this application (e.g., large-scale data storage and / or data transmission and reception), and therefore consider delegating the functions implemented by the sending unit 1002 to the computing device 1100B.

[0224] It should be understood that Figure 13 The functions of computing device 1100A shown can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0225] This application also provides another data processing cluster. The connection relationships between the computing devices in this data processing cluster can be similarly referenced. Figure 13 The connection method of the data processing cluster. The difference is that the memory 1106 of one or more computing devices 1100 in the data processing cluster can store the same instructions for executing the data processing method provided in any of the above embodiments.

[0226] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the data processing cluster may also store partial instructions for executing the methods provided in any of the above embodiments. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for performing the methods provided in any of the above embodiments.

[0227] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the methods provided in any of the above embodiments.

[0228] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform the method provided in any of the above embodiments.

[0229] Those skilled in the art will recognize that the method steps and units described in the embodiments disclosed in this application can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0230] In the embodiments provided in this application, it should be understood that the disclosed system architecture, apparatus, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, or may be electrical, mechanical, or other forms of connection.

[0231] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0232] Furthermore, the modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or in software.

[0233] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0234] In this application, the terms "first" and "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first" and "second," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first" and "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first computational unit may be referred to as a second computational unit, and similarly, a second computational unit may be referred to as a first computational unit. Both the first and second computational units can be computational units, and in some cases, they can be separate and distinct computational units.

[0235] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0236] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for processing data, characterized in that, The method is applied to a first device, the first device including a first rule, the first rule being used to define attribute information of normal data and / or attribute information of abnormal data, the method including: Obtain the attribute information of the first data to be detected; If the attribute information of the first data matches the first rule, the first data is determined to be either normal data or abnormal data. If the attribute information of the first data does not match the first rule, the system detects whether the first data is the abnormal data based on the first abnormal data detection model.

2. The method as described in claim 1, characterized in that, The first rule includes a first sub-rule and / or a second sub-rule, wherein the first sub-rule is used to define the attribute information of the normal data, and the second sub-rule is used to define the attribute information of the abnormal data; The step of determining whether the first data is normal data or abnormal data when the attribute information of the first data matches the first rule includes: If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data; If the attribute information of the first data matches the second sub-rule, the first data is determined to be the abnormal data.

3. The method as described in claim 2, characterized in that, The attribute information of the first data includes the source information of the first data; the first sub-rule includes the source information of at least one normal data; and the second sub-rule includes the source information of at least one abnormal data; or... The attribute information of the first data includes the identification information of the first data, the first sub-rule includes the identification information of the at least one normal data, and the second sub-rule includes the identification information of the at least one abnormal data.

4. The method according to any one of claims 1-3, characterized in that, The first device further includes a second rule, which is used to define an abnormal region. The attribute information of the first data includes the location of the sending object of the first data. The step of detecting whether the first data is abnormal based on the first abnormal data detection model includes: When the location belongs to the abnormal region, a second abnormal data detection model is obtained by increasing the sensitivity of the first abnormal data detection model to detect abnormal data. The first data is detected as abnormal based on the second abnormal data detection model.

5. The method as described in claim 4, characterized in that, The first abnormal data detection model includes a first threshold. The first abnormal data detection model is used to obtain the abnormal confidence level of the first data and determine that the first data is abnormal data when the abnormal confidence level is greater than or equal to the first threshold. The step of improving the sensitivity of the first anomaly detection model to detect anomaly data to obtain a second anomaly detection model includes: The first threshold of the first abnormal data detection model is adjusted to the second threshold to obtain the second abnormal data detection model, wherein the second threshold is less than the first threshold.

6. The method according to any one of claims 1-3, characterized in that, The step of detecting whether the first data is abnormal data based on the first abnormal data detection model includes: The first data is preprocessed to obtain the second data, which includes a portion of the data in the first data. Based on the first abnormal data detection model and the second data, it is determined whether the first data is the abnormal data.

7. The method as described in claim 6, characterized in that, The second data includes the portion of the first data whose quality meets the quality conditions.

8. The method as described in claim 7, characterized in that, The first data is audio data, and the second data is the portion of the first data whose noise intensity is below a third threshold; or, The first data is video data, and the second data is the portion of the first data where the contrast is below a fourth threshold, the illumination intensity is below a fifth threshold, and / or the target image is not occluded; or, The first data is an image, and the second data includes the target image in the image or data obtained by expanding the edges of the target image.

9. The method as described in claim 6, characterized in that, The first data is text data containing multiple characters, and the second data includes the first M characters in the first data, where M is an integer greater than 1.

10. The method according to any one of claims 1-3, characterized in that, The step of detecting whether the first data is abnormal data based on the first abnormal data detection model includes: A detection instruction is sent to a second device, the second device including the first data anomaly detection model, the detection instruction being used to instruct the second device to detect whether the first data is the anomalous data based on the first anomalous data detection model.

11. The method as described in claim 10, characterized in that, The first device is a device located on the cloud side, and the second device is a terminal device; or, the first device is a terminal device, and the second device is a device located on the cloud side.

12. The method according to any one of claims 1-11, characterized in that, The content of the abnormal data involves fraud.

13. A data processing apparatus, characterized in that, The device includes a first rule, which defines attribute information for normal data and / or attribute information for abnormal data. The device further includes: The processing unit is used to acquire the attribute information of the first data to be detected; The processing unit is further configured to determine whether the first data is normal data or abnormal data when the attribute information of the first data matches the first rule; The processing unit is further configured to detect whether the first data is abnormal data based on the first abnormal data detection model when the attribute information of the first data does not match the first rule.

14. The apparatus as claimed in claim 13, characterized in that, The first rule includes a first sub-rule and / or a second sub-rule, wherein the first sub-rule is used to define the attribute information of the normal data, and the second sub-rule is used to define the attribute information of the abnormal data; The processing unit is used for: If the attribute information of the first data matches the first sub-rule, the first data is determined to be normal data; If the attribute information of the first data matches the second sub-rule, the first data is determined to be the abnormal data.

15. The apparatus as claimed in claim 14, characterized in that, The attribute information of the first data includes the source information of the first data; the first sub-rule includes the source information of at least one normal data; and the second sub-rule includes the source information of at least one abnormal data; or... The attribute information of the first data includes the identification information of the first data, the first sub-rule includes the identification information of the at least one normal data, and the second sub-rule includes the identification information of the at least one abnormal data.

16. The apparatus according to any one of claims 13-15, characterized in that, The device further includes a second rule for defining an abnormal region, and the attribute information of the first data includes the location of the sending object of the first data. The processing unit is used for: When the location belongs to the abnormal region, a second abnormal data detection model is obtained by increasing the sensitivity of the first abnormal data detection model to detect abnormal data. The first data is detected as abnormal based on the second abnormal data detection model.

17. The apparatus as claimed in claim 16, characterized in that, The first abnormal data detection model includes a first threshold. The first abnormal data detection model is used to obtain the abnormal confidence level of the first data and determine that the first data is abnormal data when the abnormal confidence level is greater than or equal to the first threshold. The processing unit is used to adjust the first threshold of the first abnormal data detection model to a second threshold to obtain the second abnormal data detection model, wherein the second threshold is less than the first threshold.

18. The apparatus according to any one of claims 13-15, characterized in that, The processing unit is used for: The first data is preprocessed to obtain the second data, which includes a portion of the data in the first data. Based on the first abnormal data detection model and the second data, it is determined whether the first data is the abnormal data.

19. The apparatus as claimed in claim 18, characterized in that, The second data includes the portion of the first data whose quality meets the quality conditions.

20. The apparatus as claimed in claim 19, characterized in that, The first data is audio data, and the second data is the portion of the first data whose noise intensity is below a third threshold; or, The first data is video data, and the second data is the portion of the first data where the contrast is below a fourth threshold, the illumination intensity is below a fifth threshold, and / or the target image is not occluded; or, The first data is an image, and the second data includes the target image in the image or data obtained by expanding the edges of the target image.

21. The apparatus as claimed in claim 18, characterized in that, The first data is text data containing multiple characters, and the second data includes the first M characters in the first data, where M is an integer greater than 1.

22. The apparatus according to any one of claims 13-15, characterized in that, The device further includes: A sending unit is configured to send a detection instruction to a second device, the second device including the first data anomaly detection model, the detection instruction being configured to instruct the second device to detect whether the first data is the anomalous data based on the first anomalous data detection model.

23. The apparatus as claimed in claim 22, characterized in that, The device is a cloud-based device, and the second device is a terminal device; or, the device is a terminal device, and the second device is a cloud-based device.

24. The apparatus according to any one of claims 13-23, characterized in that, The content of the abnormal data involves fraud.

25. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-12.

26. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1-12.

27. A computer program product having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-12.

Images