Data processing method, device, equipment, storage medium and program product
By building a network of objects to be tested and dividing target groups, and combining the target relationship network structure, the problem of low detection accuracy of the characteristics of objects to be tested is solved, and higher detection accuracy and efficiency are achieved.
Patent Information
- Application Number
- CN202111372329.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-18
- Publication Date
- 2025-08-12
- Estimated Expiration
- 2041-11-18
AI Technical Summary
In the prior art, when the target characteristic detection is performed based on the self-characteristics of the object to be tested, the accuracy of the detection results is low.
By obtaining the associated objects of the object to be tested, a network of objects to be tested is constructed, and the target group of objects to be tested is divided based on the association relationship of the connected objects in the network, the target degree relationship network is extracted, and its network structure and characteristic indicators are determined, thereby improving the accuracy of the detection results.
The accuracy of the target characteristic detection results is improved, and the detection efficiency is improved by considering the correlation between the target groups to be tested.
Smart Images

Figure CN116150691B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, electronic device, computer-readable storage medium, and computer program product. Background Art
[0002] With the rapid development of the internet and the widespread adoption of mobile communications, the scale of communication between various types of behavioral objects is increasing, leading to the emergence of many objects with abnormal behavior. Therefore, there is a need to detect these objects. Objects with abnormal behavior often exhibit certain characteristics. Related technologies for detecting target characteristics often utilize the object's inherent characteristics, such as using isolation forests to identify target characteristic points within a feature distribution. However, detecting target characteristics based solely on the object's inherent characteristics is rather one-sided and results in low accuracy.
[0003] As mentioned above, how to improve the accuracy of target characteristic detection results of the object to be tested has become an urgent problem to be solved.
[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention
[0005] The purpose of the present disclosure is to provide a method, apparatus, device, readable storage medium and program product for processing data of an object to be tested, which can improve the accuracy of target characteristic detection results at least to a certain extent.
[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.
[0007] An embodiment of the present disclosure provides a data processing method, comprising: obtaining associated objects corresponding to a plurality of objects to be tested, wherein target association relationships exist between the associated objects and the corresponding objects to be tested; obtaining a network of objects to be tested composed of the plurality of objects to be tested, wherein connected objects to be tested have common associated objects in the network of objects to be tested; dividing the network of objects to be tested into a plurality of target object to be tested groups based on the relationships between the associated objects corresponding to the connected objects to be tested in the network of objects to be tested and the common associated objects; extracting a target degree relationship network of target objects to be tested in the target object to be tested groups, wherein the target degree relationship network of target objects to be tested is composed of the target objects to be tested and objects to be tested that have target degree relationships with the target objects to be tested; determining whether the target degree relationship network of the target objects to be tested belongs to a network structure of a target category, and obtaining a target characteristic index of the target object to be tested, wherein the target characteristic index is used to indicate the degree to which the target object to be tested meets the target characteristic; and determining target characteristic objects in the plurality of target object to be tested groups based on the target characteristic index of the target objects to be tested.
[0008] The present disclosure provides a data processing device, comprising: an associated object acquisition module for obtaining associated objects corresponding to a plurality of objects to be tested, wherein a target association relationship exists between the associated objects and the corresponding objects to be tested; a test object network construction module for obtaining a test object network composed of the plurality of objects to be tested, wherein the connected objects to be tested in the test object network have a common associated object; a test object group division module for dividing the test object network into a plurality of target test object groups based on the relationship between the associated objects corresponding to the connected objects to be tested in the test object network and the common associated object; and a target degree relationship network extraction module for extracting the target degree relationship network. Extracting the target degree relationship network of the target objects to be tested in the target object group, wherein the target degree relationship network of the target objects to be tested is composed of the target objects to be tested and the objects to be tested that have a target degree relationship with the target objects to be tested; a target characteristic index obtaining module, used to determine whether the target degree relationship network of the target objects to be tested belongs to the network structure of the target category, and obtain the target characteristic index of the target objects to be tested, wherein the target characteristic index is used to indicate the degree to which the target objects to be tested meet the target characteristics; a target characteristic object determining module, used to determine the target characteristic objects in the target object group according to the target characteristic index of the target objects to be tested.
[0009] According to one embodiment of the present disclosure, the target characteristic index acquisition module includes: a target feature distribution acquisition module, which is used to obtain the distribution of target features of each object to be tested in the target degree relationship network of the target object to be tested; a network structure determination module, which is used to determine that the target degree relationship network of the target object to be tested belongs to the network structure of the target category if it is determined that the distribution of target features of each object to be tested in the target degree relationship network conforms to the target power law model; a target characteristic index calculation module, which is used to obtain the target characteristic index of the target object to be tested based on the distribution of target features of each object to be tested in the target degree relationship network and the target power law model.
[0010] According to one embodiment of the present disclosure, the target characteristic index calculation module includes: a power-law deviation index acquisition module, which is used to obtain the power-law deviation index of the target object to be measured based on the target characteristics of the target object to be measured and the target power-law model; a local density index acquisition module, which is used to obtain the local density index of the target object to be measured based on the distribution of the target characteristics of each object to be measured in the target degree relationship network; a target characteristic index normalization module, which is used to normalize the power-law deviation index and the local density target characteristic index of the target object to be measured respectively; the target characteristic index calculation module is also used to obtain the target characteristic index of the target object to be measured based on the normalized power-law deviation index and local density index.
[0011] According to one embodiment of the present disclosure, the distribution of target features of each object to be measured in the target degree relationship network includes the distribution of the relationship between the actual value of the first feature and the actual value of the second feature of each object to be measured in the target degree relationship network; the power-law deviation index acquisition module is also used to substitute the actual value of the second feature of the target object to be measured into the target power-law model to obtain the predicted value of the first feature of the target object to be measured; according to the actual value of the first feature of the target object to be measured and the predicted value of the first feature of the target object to be measured in the target power-law model, the power-law deviation index of the target object to be measured is obtained.
[0012] According to one embodiment of the present disclosure, the local density index acquisition module includes: a target feature point acquisition module, which is used to obtain the target point of the target feature corresponding to the target object to be measured and the surrounding neighborhood points of the target point according to the distribution of the target features of each object to be measured in the target degree relationship network; an average reachable distance acquisition module, which is used to obtain the average reachable distance from the surrounding neighborhood points to the target point; a target point local density acquisition module, which is used to obtain the local density of the target point according to the average reachable distance; a neighborhood point local density acquisition module, which is used to obtain the local density of the surrounding neighborhood points; a local density index calculation module, which is used to obtain the local density target characteristic index of the target object to be measured according to the local density of the surrounding neighborhood points and the local density of the target point.
[0013] According to one embodiment of the present disclosure, the target characteristic index normalization module includes: a power-law deviation index total amount acquisition module, used to obtain multiple power-law deviation indicators corresponding to multiple target objects in the target object group; a power-law deviation index normalization module, used to obtain the normalized power-law deviation index of the target object based on the power-law deviation index of the target object and the multiple power-law deviation indicators; a local density index total amount acquisition module, used to obtain multiple local density indicators corresponding to multiple target objects in the target object group; a local density index normalization module, used to obtain the normalized local density target characteristic index of the target object based on the local density index of the target object and the multiple local density indicators.
[0014] According to one embodiment of the present disclosure, the target power law model includes a first target power law model, a second target power law model and a third target power law model; the network structure of the target category includes a star or cluster structure, a total edge weight characteristic structure and a characteristic weight edge structure; the network structure determination module includes: a star or cluster structure determination module, which is used to determine that the target degree relationship network of the target object to be tested belongs to the star or cluster structure if the relationship between the total number of connection edges in the target degree relationship network of each object to be tested and the number of objects to be tested connected to each object to be tested conforms to the first target power law model; a total edge weight characteristic structure determination module, which is used to determine that the target degree relationship network of the target object to be tested belongs to the total edge weight characteristic structure if the relationship between the sum of the weights of each connection edge in the target degree relationship network of each object to be tested and the total number of connection edges conforms to the second target power law model; a characteristic weight edge structure determination module, which is used to determine that the target degree relationship network of the target object to be tested belongs to the characteristic weight edge structure if the relationship between the target characteristic value of the target connection edge of each object to be tested and the sum of the weights of each connection edge conforms to the third target power law model.
[0015] According to one embodiment of the present disclosure, the module for dividing the groups of objects to be tested includes: a connection edge weight acquisition module, which is used to obtain the weights of the connection edges between the objects to be tested connected in the object network according to the quantitative relationship between the associated objects corresponding to each of the connected objects to be tested and the common associated objects; the module for dividing the groups of objects to be tested is also used to divide the object network to be tested into multiple groups of objects to be tested based on the weights of the connection edges between the objects to be tested connected in the object network.
[0016] According to one embodiment of the present disclosure, the object group division module also includes: a current group division result acquisition module, which is used to divide each object to be tested in the object network to one of the multiple object groups to be tested, and obtain a first group division result as the current group division result; a first modularity acquisition module, which is used to obtain the first modularity of the object network according to the first group division result based on the weights of the connection edges between the objects to be tested connected in the object network to be tested; a second group division result acquisition module, which is used to move the current object to be tested to the object group to which the object to be tested connected in the object network to be tested is located, and obtain a second group division result; a second modularity acquisition module, which is used to obtain the second modularity of the object network according to the second group division result based on the weights of the connection edges between the objects to be tested connected in the object network to be tested; and a current group division result updating module, which is used to update the current group division result to the second group division result if the second modularity is greater than the first modularity.
[0017] According to one embodiment of the present disclosure, the target characteristic object determination module includes: a target characteristic index sorting module, which is used to sort the multiple target objects to be tested whose target degree relationship network belongs to the network structure of the target category in the multiple groups of objects to be tested according to the numerical values of the target characteristic indicators; the target characteristic object determination module is also used to determine the target characteristic objects among the multiple target objects to be tested based on the sorting results of the numerical values of the target characteristic indicators of the multiple target objects to be tested.
[0018] An embodiment of the present disclosure provides a device, including: a memory, a processor, and executable instructions stored in the memory and executable in the processor, wherein the processor implements any of the above methods when executing the executable instructions.
[0019] An embodiment of the present disclosure provides a computer-readable storage medium having computer-executable instructions stored thereon. When the executable instructions are executed by a processor, any of the above methods is implemented.
[0020] The present disclosure provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.
[0021] The data processing method provided by the embodiments of the present disclosure obtains the associated objects corresponding to each of the multiple objects to be tested, connects the objects to be tested that have common associated objects to form a network of objects to be tested, and then divides the network of objects to be tested into multiple target objects to be tested groups based on the relationship between the associated objects corresponding to each of the connected objects to be tested and the common associated objects, and then extracts the target degree relationship network of the target objects to be tested in the target object groups, determines that the target degree relationship network of the target objects to be tested belongs to the network structure of the target category, obtains the target characteristic index of the target objects to be tested, and determines the target characteristic objects in the multiple target objects to be tested groups based on the target characteristic index of the target objects to be tested. On the one hand, by dividing the objects to be tested into groups based on their common associated objects, in the process of target characteristic detection, not only the associated objects of each object to be tested are considered, but also the associations between different objects to be tested are taken into account, thereby improving the accuracy of the target characteristic detection results; on the other hand, target characteristic detection is performed on the target degree relationship network of each target object to be tested according to each target object to be tested group, without considering the connections between objects to be tested with low associations between different target object to be tested groups (not being divided into the same target object to be tested group indicates that the associations between objects to be tested in different target object to be tested groups are low), thereby further improving the efficiency of target characteristic detection.
[0022] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The above and other objects, features and advantages of the present disclosure will become more apparent by describing in detail example embodiments thereof with reference to the attached drawings.
[0024] Figure 1 A schematic diagram showing a system structure in an embodiment of the present disclosure.
[0025] Figure 2 A flow chart of a data processing method in an embodiment of the present disclosure is shown.
[0026] Figure 3 Shown Figure 2 FIG. 5 is a schematic diagram of the processing process of step S206 in one embodiment.
[0027] Figure 4 is based on Figure 3 FIG. 1 is a flow chart of another method for dividing groups of objects to be tested.
[0028] Figure 5 is based on Figure 3 and Figure 4 A schematic diagram of a process of dividing the groups of objects to be tested is shown.
[0029] Figure 6 is based on Figure 5 A schematic diagram of a first-degree relationship network is shown.
[0030] Figure 7 Shown Figure 2 FIG. 5 is a schematic diagram of the processing process of step S210 in one embodiment.
[0031] Figure 8 is based on Figure 7 A schematic diagram of a star-structured network is shown.
[0032] Figure 9 is based on Figure 7 A schematic diagram of a cluster structure network is shown.
[0033] Figure 10 is based on Figure 7 A schematic diagram of a total edge weight feature structure network is shown.
[0034] Figure 11 Shown Figure 7 FIG. 5 is a schematic diagram of the processing process of step S706 in one embodiment.
[0035] Figure 12 Shown Figure 11 FIG. 1 is a schematic diagram of the processing process of step S1104 in one embodiment.
[0036] Figure 13 is based on Figure 7-9 as well as Figure 11 An example of a power law fitting result is shown.
[0037] Figure 14 Shown Figure 11 FIG. 5 is a schematic diagram of the processing process of step S1106 in one embodiment.
[0038] Figure 15 Shown Figure 2 FIG. 5 is a schematic diagram of the processing process of step S212 in one embodiment.
[0039] Figure 16 A block diagram of a data processing device in an embodiment of the present disclosure is shown.
[0040] Figure 17 A block diagram showing another data processing device in an embodiment of the present disclosure.
[0041] Figure 18 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these examples are provided so that this disclosure will be more comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures indicate identical or similar parts, and thus repeated descriptions thereof will be omitted.
[0043] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, devices, steps, etc. may be adopted. In other cases, well-known structures, methods, devices, implementations or operations are not shown or described in detail to avoid obscuring various aspects of the present disclosure.
[0044] In addition, in the description of the present disclosure, unless otherwise clearly specified and limited, terms such as "connection" should be understood in a broad sense. For example, it can be an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium. The meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically limited. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances. The meaning of "target characteristic" is a specific characteristic of the object to be measured that serves as a detection target. For example, it can be the community density of the object to be measured, the abnormal situation of the object to be measured, etc. In the embodiment of the present disclosure, the target characteristic is an abnormal situation of the object to be measured.
[0045] As mentioned above, in the relevant anomaly detection technology, the characteristics of the object to be tested are usually used for anomaly detection. For example, when the object to be tested is a merchant user (also known as a "merchant"), based on the merchant's transaction characteristics, such as transaction amount, transaction frequency, transaction time, etc., methods such as isolation forest are used to detect outliers in the feature distribution to capture abnormal behavior in merchant transactions. For another example, when the object to be tested is a communication user (such as a landline user or a mobile phone user), anomaly detection is performed based on characteristics such as call duration, call frequency, etc.
[0046] However, many abnormal behaviors are associated with other objects to be tested. For example, when the object to be tested is a merchant, the merchant with abnormal transactions and the object with which the merchant conducts abnormal transactions (i.e., an associated object corresponding to the object to be tested) usually have abnormal transactions with multiple merchants; for another example, when an internet account is the object to be tested, the account with abnormal internet behavior and the target of abnormal internet behavior (i.e., an associated object corresponding to the object to be tested) are usually multiple; for another example, when the object to be tested is a communication user, the call object of the communication user (i.e., an associated object corresponding to the object to be tested) usually receives more than one abnormal call and may receive calls from multiple abnormal communication users. Therefore, performing anomaly detection based only on the characteristics of the object to be tested is rather one-sided, and the accuracy of the detection results is low.
[0047] Therefore, the present disclosure provides a data processing method, which divides the target objects to be tested into groups based on the common related objects between different objects to be tested, and takes into account the associations between different objects to be tested in the same target object group during the anomaly detection process, thereby improving the accuracy of the anomaly detection results; and performs anomaly detection on the target degree relationship network of the target objects to be tested according to the target object group, without considering the connections between the objects to be tested with low association between the target object groups (not divided into the same target object group, indicating low association), which can further improve the efficiency of anomaly detection.
[0048] Figure 1 An exemplary system architecture 10 is shown to which the data processing method and data processing apparatus of the present disclosure can be applied.
[0049] like Figure 1 As shown, system architecture 10 may include terminal device 102, network 104, and server 106. Terminal device 102 may be any electronic device with a display screen and support input and output, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, wearable devices, in-vehicle terminals, virtual reality devices, smart homes, etc. Network 104 is a medium used to provide a communication link between terminal device 102 and server 106. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables. Server 106 may be a server or server cluster that provides various services, such as a processing server, website server, database server, etc.
[0050] A user can use terminal device 102 to interact with server 106 via network 104 to receive or send data, etc. For example, a user can obtain target characteristic object information from server 106 via network 104 on terminal device 102. For another example, a user's behavior on terminal device 102 is recorded by terminal device 102 and then transmitted to server 106 via network 104 for storage, thereby enabling the user to obtain associated objects corresponding to each test object and common associated objects between different test objects. For another example, a user can operate on terminal device 102 to select associated object information for multiple test objects within a preset time period from a database server and send instructions to a processing server to perform target characteristic detection on this information.
[0051] The multiple servers 106 may also receive or send data to each other via the network 104. For example, a processing server may obtain data of a test object from a database server. Another example is that a website server may send recorded user behavior information to a database server for storage.
[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.
[0053] Figure 2 FIG. 1 is a flow chart showing a data processing method according to an exemplary embodiment. Figure 2 The method shown can be applied to a server of the above system, or to a terminal device of the above system, for example.
[0054] refer to Figure 2 , the method 20 provided in the embodiment of the present disclosure may include the following steps S202 to S212.
[0055] In step S202, associated objects corresponding to a plurality of objects to be detected are obtained, and a target association relationship exists between the associated objects and the corresponding objects to be detected.
[0056] In the disclosed embodiments, the object to be tested refers to the object to be tested for abnormal (target characteristic) behavior. An associated object refers to an object that has or has a target association relationship with the object to be tested. The object to be tested and the corresponding associated object are determined based on different application scenarios and corresponding target association relationships.
[0057] In some embodiments, for example, the subject to be tested may be a merchant on an internet platform, the target association relationship may be a transaction, and the associated objects may be transaction partners that have transacted with these merchants on the internet platform. Merchants that have transacted within a preset time period (e.g., the past two weeks, one month, or two months, etc.) and the transaction partners of these merchants within that month can be obtained from the transaction flow on the internet platform.
[0058] In other embodiments, for example, the object to be tested may also be an account owner of a certain Internet platform, the association relationship may be an interaction, and the associated object may be a fan user who interacts with these account owners on the Internet platform (such as likes, comments, rewards, etc.).
[0059] In step S204, a network of objects to be tested consisting of a plurality of objects to be tested is obtained. In the network of objects to be tested, connected objects to be tested have a common associated object.
[0060] In some embodiments, multiple objects to be tested that have common associated objects can be matched in pairs. For example, when the objects to be tested are merchants, after extracting transaction flows, the merchants can be matched in pairs to obtain merchant relationship pairs with common transaction objects.
[0061] In some embodiments, multiple test objects are used as multiple nodes in a test object network. Every two successfully matched test objects are connected nodes. The nodes of these two test objects are connected by a connecting edge. The connecting edge between the nodes of two test objects that are directly connected (without passing through other nodes in between) can be called a direct edge. If two test objects do not have a common associated object, then these two test objects do not have a direct edge in the test object network.
[0062] For example, transaction object Z has purchased product a from merchant A and product b from merchant B in the past month. In this network of the object to be tested, merchant A and merchant B are a merchant relationship pair with a common transaction object, and there is a direct edge connecting the two nodes of merchant A and merchant B.
[0063] In step S206 , the network of test objects is divided into a plurality of target test object groups based on the relationship between the associated objects corresponding to the connected test objects in the network of test objects and the common associated objects.
[0064] In some embodiments, the weights of the connection edges between the connected objects to be tested in the network of the object to be tested can be obtained based on the relationship between the number of associated objects corresponding to each of the connected objects to be tested in the network of the object to be tested and the number of common associated objects. Then, the network of the object to be tested can be divided according to the weights of the connection edges to divide the network of the object to be tested into multiple target groups of objects to be tested. For specific implementation methods, please refer to Figures 3 to 5 .
[0065] In other embodiments, the distance between the objects in the network can be obtained based on the relationship between the number of associated objects corresponding to each of the connected objects and the number of common associated objects. For example, the ratio of the number of common associated objects divided by the sum of the number of associated objects corresponding to each of the connected objects can be used as the distance between the two connected objects. A distance-based clustering method (such as K-means) can then be used to cluster the objects in the network, with similar objects being grouped as a target object group.
[0066] In step S208 , a target degree relationship network of target objects to be tested in the target object group is extracted. The target degree relationship network of target objects to be tested is composed of target objects to be tested and objects to be tested that have a target degree relationship with the target objects to be tested.
[0067] In the embodiment of the present disclosure, each target object group may include at least one object to be tested, the at least one object to be tested includes the target object to be tested, and the target object to be tested may be any one of the at least one object to be tested.
[0068] In some embodiments, the target degree relationship can be a first-degree relationship, a second-degree relationship, etc., and the target degree relationship network can be a first-degree relationship network, or a second-degree relationship network, etc. A first-degree relationship means that two objects to be tested are directly connected, that is, there is a direct edge between the two nodes; a second-degree relationship means that two objects to be tested are connected through another object to be tested, and so on. In the following embodiments, the target degree relationship network is used as an example for explanation as a first-degree relationship network, but the present disclosure is not limited to this. For example, each target object to be tested group of a plurality of target object to be tested groups can be selected in turn, and each object to be tested in the selected target object to be tested group can be used as a target object to be tested, to obtain a first-degree relationship network centered on the target object to be tested.
[0069] Figure 6 is based on Figure 5 A first-degree relationship network diagram is shown in Figure 1. Figure 5 The target object group 1" and the target object group 2' obtained by the second stage division are the final results. Figure 5The first-degree relationship network between node 7 and node 3 is as follows: Figure 6 As shown, the directly connected nodes in the target object group 2' where node 7 is located are nodes 8-13, and the directly connected nodes in the target object group 1" where node 3 is located are node 1, node 5, node 6, and node 16; and the directly connected node 12 of node 3 in the initial network (i.e., the object network) is not a node in the first-degree relationship network of node 3 extracted from the target object group 1" because node 12 is divided into the target object group 2'.
[0070] In step S210 , it is determined that the target degree relationship network of the target object to be tested belongs to the network structure of the target category, and the target characteristic index of the target object to be tested is obtained. The target characteristic index is used to indicate the degree to which the target object to be tested meets the target characteristic.
[0071] In some embodiments, for example, the network structure of the target category may include a star or cluster structure, a total edge weight feature structure, a feature weight edge structure, and the like. The network structure of different target categories can be determined by fitting the corresponding target feature distribution. If the fitting result satisfies the target power law model, the network structure belongs to the corresponding target category. For specific implementation methods, please refer to Figures 7 to 10 .
[0072] In some embodiments, for example, the target characteristic index of the target object to be measured can be obtained by comprehensively analyzing the deviation degree between the distribution of the target characteristics and the target power law model and the distribution of the target characteristics. For specific implementation methods, please refer to Figures 11 to 14 .
[0073] In step S212 , target characteristic objects in a group of multiple target objects to be tested are determined according to the target characteristic indicators of the target objects to be tested.
[0074] In some embodiments, for example, the target characteristic values of the target objects to be tested whose first-degree relationship networks belong to the network structure of a certain target category can be sorted from large to small according to different categories of network structures, and the top number of target objects to be tested can be used as target characteristic objects. For specific implementation methods, please refer to Figure 15 .
[0075] In other embodiments, for example, corresponding target characteristic indicator thresholds can also be set according to network structures of different target categories, and the target characteristic indicator values obtained by the target objects corresponding to the network structures belonging to the target categories and which are greater than the target characteristic indicator thresholds corresponding to the network structures of the target categories are classified as target characteristic objects.
[0076] According to the data processing method provided by the embodiment of the present disclosure, after obtaining the associated objects corresponding to each of the multiple objects to be tested, the objects to be tested that have common associated objects are connected to form a network of objects to be tested. Then, based on the relationship between the associated objects corresponding to the connected objects to be tested in the network of objects to be tested and the common associated objects, the network of objects to be tested is divided into multiple target groups of objects to be tested. Then, the target degree relationship network of the target objects to be tested in the target groups of objects to be tested is extracted, and the network structure of the target category of the target degree relationship network of the target objects to be tested is determined. The target characteristic index of the target object to be tested is obtained, and based on the target characteristic index of the target object to be tested, the target characteristic objects in the multiple target groups to be tested are determined.
[0077] Taking the merchant as an example, first, by using the merchant's transaction object as a relationship to associate with another merchant, and the more common transaction objects (i.e., common associated objects) between the two merchants, the greater the weight of the associated connection edge in the network, this method can connect two merchants that do not have common merchant information and seem to have no connection, thereby improving the accuracy of the anomaly detection results. Secondly, when identifying abnormal merchants, the relevant technical solutions mainly use the merchant's own transaction characteristics to identify the merchant corresponding to the abnormal point through data processing methods. The method provided by the present disclosure uses the target degree relationship network extracted after dividing the target object group to identify abnormal merchants (i.e., target characteristic objects) by judging the abnormal network structure of the target category to which the target degree relationship network belongs. The connection between merchants with low correlation between the target object groups is not considered, which can further improve the efficiency of anomaly detection.
[0078] Figure 3 Shown Figure 2 Schematic diagram of the processing process of step S206 in one embodiment. Figure 3 As shown, in the embodiment of the present disclosure, the above step S206 may further include the following steps S302 to S30410.
[0079] Step S302 : obtaining the weights of the connection edges between the connected objects under test in the network of the objects under test according to the quantity relationship between the associated objects corresponding to the connected objects under test and the common associated objects in the network of the objects under test.
[0080] In some embodiments, for two connected (commonly associated objects) objects to be tested, the ratio of the number of common associated objects to the sum of the number of associated objects corresponding to each of them can be calculated to obtain the weight of their connecting edge. For example, for any two different merchants, the transaction objects of the two merchants over a period of time are counted separately. Taking merchant A and merchant B as an example, merchant A has 2 transaction objects, merchant B has 3 transaction objects, and the two merchants have 2 common transaction objects. Then the ratio of common transaction objects of the two merchants can be (2+2) / (3+2)=4 / 5, that is, the ratio of common transaction objects = (2*common transaction objects) / (merchant A transaction objects + merchant B transaction objects). This ratio is the weight of the connecting edge between merchant A and merchant B in the merchant network. If the two merchants do not have common transaction objects, there is no direct connection edge in the merchant network.
[0081] In other embodiments, for two connected objects to be tested, the sum of the ratios of the number of common associated objects to the number of their respective corresponding associated objects can also be calculated to obtain the weight of their connecting edge. The specific calculation method can be adjusted according to the actual group division results, and this application does not impose any restrictions.
[0082] Step S304 : dividing the network of objects to be tested into a plurality of groups of objects to be tested based on the weights of the connection edges between the objects to be tested connected in the network of objects to be tested.
[0083] In some embodiments, step S304 may include the following steps S3042 to S30410.
[0084] Step S3042: Each of the objects under test in the network of objects under test is divided into one of the multiple groups of objects under test, and a first group division result is obtained as a current group division result.
[0085] Step S3044 : obtaining a first modularity of the network of the object to be tested according to the weights of the connection edges between the objects to be tested connected in the network of the object to be tested and the first community division result.
[0086] In some embodiments, modularity can be used as an important criterion for measuring the quality of community (group) division. The larger the modularity value of the divided network, the better the community division effect. The goal of community division is to make the connections within the divided community relatively close, while the connections between communities are relatively sparse. The quality of such division can be characterized by the size of the modularity value. The larger the modularity, the better the community division effect. For example, the calculation formula of modularity can be shown as follows:
[0087]
[0088] Wherein, i and j are both positive integers, i and j represent the i-th node and j-th node in the network to be tested respectively. It represents the sum of all weights in the network, A i,j represents the weight of the edge between node i and node j, k i =∑ j A i,j represents the sum of the weights of all edges connected to vertex i, c i represents the community to which the vertex is assigned, δ(c i ,c j ) is used to determine whether vertex i and vertex j are divided into the same community. If so, it returns 1, otherwise, it returns 0.
[0089] Step S3046: Move the current object to be tested to the object group to which the object to be tested connected in the object network to be tested belongs, to obtain a second group division result.
[0090] Step S3048 : obtaining a second modularity of the network of the objects to be tested according to the weights of the connection edges between the objects to be tested connected in the network of the objects to be tested and the second community division result.
[0091] In some embodiments, the second modularity is calculated in the same way as the first modularity, for example, the method of formula (1) can be used. Since the objects to be tested move between groups, δ(c i ,c j ) will change, so the final modularity Q will be different.
[0092] Step S30410: If the second modularity is greater than the first modularity, the current community division result is updated to the second community division result.
[0093] In some embodiments, the iteration can be performed according to step S3042 to step S30410. Initially, each object to be tested can be divided into an initial object group to be tested, and the initial modularity of the object network to be tested is calculated. Then, in a certain order (for example, for any specified first current object to be tested, starting from any of its connected edges, it moves in sequence to the object group to be tested where the connected object to be tested is located, and then calculates the current modularity of the object network to be tested after the movement, and then retains the current division when the current modularity is greater than the initial modularity, and continues to move the current object to be tested until the modularity of the object network to be tested no longer increases, and then specifies the next current object to be tested, and repeats the above process. The iterative process can refer to Figure 4 .
[0094] In some embodiments, the group division result after the above-mentioned object to be tested is used as the basis, and each group of objects to be tested is respectively regarded as an object to be tested, and the sum of the weights of the original connecting edges between the groups of objects to be tested is used as the weight to calculate the modularity. According to the above steps, the division is continued in the direction of increasing modularity. For the specific implementation method, please refer to Figure 5 .
[0095] Figure 4 is based on Figure 3 FIG. 1 is a flow chart of another method for dividing the groups of objects to be tested. Figure 4 As shown, the process starts, obtains the current node (object to be tested) and the current group division result, calculates the current network modularity Q according to formula (1) (S402); moves the current node to the object to be tested group where the current adjacent node (connected object to be tested) is located, and calculates the network modularity Q' after the move (S404); determines whether the network modularity Q' after the move is greater than the current network modularity Q (S406); if Q'>Q, then updates the current group division result to the group division result after the move (S408), and determines whether there is a next step. An adjacent node (i.e., whether it can move in other directions) (S410), if Q'≤Q, then go directly to step S410; if there is a next adjacent node, update the current adjacent node to the next adjacent node (S412), and return to step S404; if there is no next adjacent node, determine whether there is a next node (i.e., whether there are other nodes that have not moved) (S414); if there is a next node, update the current node to the next node (S416), and return to step S404; if there is no next node, end the process.
[0096] Figure 5 is based on Figure 3 and Figure 4 A schematic diagram of the process of dividing the groups of objects to be tested is shown in FIG. Figure 5 As shown, the nodes in the initial network diagram are numbered 1-20. In the first stage, first follow Figure 4 The process uses modularity optimization to perform group division (S502). Nodes with different fills in the diagram are assigned to different groups of test objects, for example, nodes 3 and 12 are assigned to different groups. Group aggregation is then performed (S504). In the second stage, the aggregated nodes 1', 2', and 3' are processed according to steps S502 and S504 to obtain aggregated nodes 1" and 2'. For a large number of nodes, multiple stages may be required until the network structure no longer changes, meaning there are no more node movements that can increase the network's modularity.
[0097] According to the group division method provided in the embodiment of the present disclosure, without any manual subjective labeling and with only the objective correlation between the objects to be tested as the required information, a large number of objects to be tested can be accurately divided into independent groups, weakly correlated objects to be tested can be preliminarily separated, and anomaly detection can be performed on each group separately, thereby improving the efficiency of anomaly detection.
[0098] Figure 7 Shown Figure 2 FIG. 1 is a schematic diagram of the processing process of step S210 in one embodiment. Figure 7 As shown, in the embodiment of the present disclosure, the above step S210 may further include the following steps S702 to S706.
[0099] Step S702: obtaining the distribution of target features of each target object in the target degree relationship network of the target object.
[0100] Step S704: If it is determined that the distribution of the target features of each target object in the target degree relationship network conforms to the target power law model, it is determined that the target degree relationship network of the target target object belongs to the network structure of the target category.
[0101] In some embodiments, for example, corresponding to the star-shaped or cluster-shaped structure, the total edge weight characteristic structure and the characteristic weight edge structure, the target power-law model may include a first target power-law model, a second target power-law model and a third target power-law model. The following steps S7042 to S7046 describe the judgment method of these three categories of network structures.
[0102] Step S7042: If it is determined that the relationship between the total number of connection edges in the target degree relationship network of each target object and the number of targets connected to each target object conforms to the first target power law model, it is determined that the target degree relationship network of the target target object belongs to a star or cluster structure.
[0103] In some embodiments, taking the extraction of a first-degree relationship network of an object to be tested as an example, the above-mentioned feature distribution conforming to the first target power law model can be expressed by the following formula:
[0104]
[0105] Among them, node i is any node in the first-degree relationship network of the target object to be tested, E i Represents the total number of connected edges in the first-degree relationship network centered on node i, N i represents the number of nodes directly connected to node i in the first-degree relationship network. Formula (2) expresses that E of any node in the first-degree relationship network of the target object to be tested is the αth power of its N.
[0106] Figure 8 and Figure 9is based on Figure 7 A schematic diagram of a star-shaped or cluster-shaped network is shown. Figure 8 As shown, the first-degree relationship network centered on node p belongs to a star structure, where E p is 12, N i is also 12. In this network, the E and N of the directly connected nodes of node p are both 1, and the first-degree relationship network centered on node p satisfies E i =N i .like Figure 9 As shown, the first-degree relationship network centered on node q belongs to a cluster structure. In this network, each node is almost connected to other nodes, that is, for each node in the network, E=N is nearly satisfied. 2 It can be seen that the star-shaped or ring-shaped structure graphs fall into the power-law distribution of E and N between 1≤α≤2.
[0107] Step S7044: If it is determined that the relationship between the sum of the weights of each connection edge in the target degree relationship network of each object to be tested and the total number of connection edges conforms to the second target power law model, then it is determined that the target degree relationship network of the target object to be tested belongs to the total edge weight feature structure.
[0108] In some embodiments, taking the extraction of a first-degree relationship network of an object to be tested as an example, the above-mentioned feature distribution conforming to the second target power law model can be expressed by the following formula:
[0109]
[0110] Wherein, node i is any node in the first-degree relationship network of the target object to be tested, W i It represents the sum of the weights of all connected edges in the first-degree relationship network centered on node i. Formula (3) expresses that the W of any node in the first-degree relationship network of the target object is the β-power of its E.
[0111] Figure 10 is based on Figure 7 A schematic diagram of a total edge weight characteristic structure network is shown in FIG. Figure 10 As shown, the first-degree relationship network centered on node r belongs to the total edge weight characteristic structure, where the thicker the edge, the greater the edge weight. Using formula (3), we can identify Figure 10 The abnormal objects shown have abnormally large edge weights, and the higher β is, the more abnormal it is.
[0112] Step S7046: If it is determined that the relationship between the target characteristic value of the target connection edge of each object to be tested and the sum of the weights of each connection edge conforms to the third target power law model, it is determined that the target degree relationship network of the target object to be tested belongs to the feature weight edge structure.
[0113] In some embodiments, taking the extraction of a first-degree relationship network of an object to be tested as an example, the above-mentioned feature distribution conforming to the third target power law model can be expressed by the following formula:
[0114]
[0115] Among them, node i is any node in the first-degree relationship network of the target object to be tested, λ ω,i The characteristic value of the edge ω of node i is represented by the characteristic value of the edge ω. The characteristic value of the edge ω is used as the main characteristic value of node i. The main characteristic value can be, for example, the value corresponding to the call duration between two communicating users, the value corresponding to the sum of the transaction amounts of the common transaction objects of two merchants, etc. Formula (4) is expressed as λ of any node in the first-degree relationship network of the target object to be tested ω is W raised to the power of γ.
[0116] In some embodiments, a first-degree relationship network of a target object under test may belong to one or more of the three aforementioned network structure categories, or may not belong to any of the three aforementioned network structure categories. If a first-degree relationship network does not belong to any of the three aforementioned network structure categories, the target characteristic index for the target object under test may not be calculated, and the target object under test may be deemed to lack the target characteristic, i.e., the target object under test is deemed to be non-abnormal.
[0117] Step S706 , obtaining target characteristic indicators of the target objects to be measured according to the distribution of target characteristics of each target object to be measured in the target degree relationship network and the target power law model.
[0118] In some embodiments, Figure 11 Shown Figure 7 Schematic diagram of the processing process of step S706 in one embodiment. Figure 11 As shown, in the embodiment of the present disclosure, the above step S706 may further include the following steps S1102 to S1108.
[0119] Step S1102 : obtaining a power-law deviation index of the target object to be measured according to the target characteristics and the target power-law model of the target object to be measured.
[0120] In some embodiments, the distribution of target features of each object to be measured in the target degree relationship network includes the distribution of the relationship between the actual value of the first feature and the actual value of the second feature of each object to be measured in the target degree relationship network.
[0121] Step S11022: Substitute the actual value of the second characteristic of the target object to be measured into the target power law model to obtain a predicted value of the first characteristic of the target object to be measured.
[0122] Step S11024 , obtaining a power-law deviation index of the target object to be measured according to the actual value of the first feature of the target object to be measured and the predicted value of the first feature of the target object to be measured in the target power-law model.
[0123] In some embodiments, the power law deviation index can be calculated according to the target power law model to be obeyed. For example, the power law deviation index OL1 of node i can be calculated using the following formula: i :
[0124]
[0125] Among them, y i is the corresponding vertical axis feature of node i (for example, the first target power law model is E i , the second target power law model is W i etc.), is the fitted value (predicted value) of the corresponding vertical axis feature of node i (for example, the first target power law model is The second target power law model is etc.), the difference between the two measures the degree of deviation; taking log is for smoothing; The term is the penalty coefficient, which is the larger of the actual value and the fitted value. with the smaller than.
[0126] Step S1104 : obtaining a local density index of the target object to be measured according to the distribution of the target features of each target object to be measured in the target degree relationship network.
[0127] In some embodiments, for example, referring to Figure 12 , the local density index of the target object to be measured can be calculated using steps S1202 to S12010.
[0128] Step S1106 , normalizing the power law deviation index and the local density index of the target object to be measured respectively.
[0129] In some embodiments, for example, referring to Figure 14 , the power law deviation index and the local density index can be normalized respectively using steps S1402 to S1408.
[0130] Step S1108 : obtaining a target characteristic index of the target object to be measured according to the normalized power law deviation index and the local density index.
[0131] In some embodiments, for example, the target characteristic index OS of node i can be calculated using the following formula: i :
[0132] OS i =OL1′i +OL2′ i (6)
[0133] Among them, OL1′ i is the normalized power law deviation index of node i, OL2′ i is the normalized local density index of node i, and the local density index of node i is OL2 i It can be calculated by formula (7).
[0134] Figure 12 Shown Figure 11 Schematic diagram of the processing process of step S1104 in one embodiment. Figure 12 As shown, in the embodiment of the present disclosure, the above step S1104 may further include the following steps S1202 to S12010.
[0135] Step S1202 : obtaining a target point of the target feature corresponding to the target object to be measured and surrounding neighborhood points of the target point according to the distribution of the target features of each target object to be measured in the target degree relationship network.
[0136] Step S1204: Obtain the average reachable distance from the surrounding neighborhood points to the target point.
[0137] Step S1206: Obtain the local density of the target point according to the average reachable distance.
[0138] Step S1208: Obtain the local density of surrounding neighborhood points.
[0139] Step S12010: Obtain a local density index of the target object according to the local density of the surrounding neighborhood points and the local density of the target point.
[0140] In some embodiments, the local density index may be calculated according to the distribution of target features that obey the target power law model. For example, the local density index of node i may be calculated using the following formula:
[0141]
[0142] Where k represents the point s of node i in the feature distribution (for example, the first target power law model is N i 、E i The kth distance neighborhood (k is a set positive integer) of the point s in the NE coordinate axis, the kth distance of point s represents the distance to the point k farthest from point s; N k,i represents the total number of points in the kth distance neighborhood of point s; u is a point in the kth distance neighborhood of node i in the feature distribution, lrd k,i represents the local reachability density of point s in its k-th distance neighborhood (i.e., the above local density), lrdk,u represents the local reachability density of point u in its k-th distance neighborhood. The local reachability density of a point in its k-th distance neighborhood is the reciprocal of the average reachable distance from the points in its k-th distance neighborhood to the point. The k-th reachable distance from point s to point u is the larger of s's k-th distance and the true distance between s and u. Then formula (7) represents the average ratio of the local reachability density of points in point s's k-th distance neighborhood to the local reachability density of point s. If this ratio is closer to 1, it means that the density of point s's neighborhood points is similar, and point s may belong to the same cluster as its neighborhood; if this ratio is less than 1, it means that the density of point s is higher than the density of its neighborhood points, and point s is a dense point; if this ratio is greater than 1, it means that the density of point s is lower than the density of its neighborhood points, and point s is more likely to be an outlier.
[0143] Figure 13 is based on Figure 7-9 as well as Figure 11 An example of a power law fitting result is shown in FIG. Figure 13 As shown, taking the above characteristic distribution of the first-degree relationship network of the object to be tested as an example, Figure 13 The dots, star points, and triangle points in the figure are NE feature points. The solid line in the middle is the fitted first target power law model. It can be seen that the star points are points that deviate from the first target power law model, and their corresponding power law deviation index values are large. The triangle points are points that are relatively outliers in the first target power law model, and their corresponding local density index values are large. The dotted line above the solid line in the figure is the fitted E=N 2 The first target power law model of , and the dotted line below the solid line is the fitted first target power law model of E=N.
[0144] Figure 14 Shown Figure 11 Schematic diagram of the processing process of step S1106 in one embodiment. Figure 14 As shown, in the embodiment of the present disclosure, the above step S1106 may further include the following steps S1402 to S1408.
[0145] Step S1402 : obtaining a plurality of power-law deviation indicators corresponding to a plurality of target objects in the target object group.
[0146] Step S1404 : obtaining a normalized power-law deviation index of the target object to be measured according to the power-law deviation index of the target object to be measured and the multiple power-law deviation indexes.
[0147] In some embodiments, for example, according to different types of network structures, the power-law deviation indexes of corresponding target objects in the target object group can be added together to obtain a total, and then the power-law deviation indexes of the target objects can be divided by the total for normalization.
[0148] In other embodiments, for example, the power law deviation indexes of corresponding target objects to be measured in all groups of objects to be measured may be added together to obtain a total.
[0149] Step S1406 , obtaining a plurality of local density indicators corresponding to a plurality of target objects in the target object group.
[0150] Step S1408 : obtaining a normalized local density index of the target object to be measured according to the local density index of the target object to be measured and the multiple local density indices.
[0151] In some embodiments, for example, according to different categories of network structures, the local density indexes of the corresponding target objects in the target object group can be added to obtain a total, and then the local density index of the target object can be divided by the total for normalization.
[0152] In other embodiments, for example, the local density indexes of the corresponding target objects to be measured in all the groups of objects to be measured may be summed up.
[0153] According to the data processing method provided in the embodiment of the present disclosure, through group division and graph-based structural type identification, a group of high-risk merchants and the discovery of some unknown risks can be identified without any manual subjective labels. It can also discover abnormal merchants that are not covered by the models and strategies in related technologies, further improving the risk management level in payment risk control.
[0154] Figure 15 Shown Figure 2 FIG. 1 is a schematic diagram of the processing process of step S212 in one embodiment. Figure 15 As shown, in the embodiment of the present disclosure, the above step S212 may further include the following steps S1502 to S1504.
[0155] Step S1502 , sorting the target objects to be tested whose target degree relationship networks belong to the target category in the target object groups according to the values of the target characteristic indicators.
[0156] Step S1504 , determining a target characteristic object among the multiple target objects to be tested according to the result of sorting the target characteristic index values of the multiple target objects to be tested.
[0157] In some embodiments, for example, in the case of the target characteristic index calculated according to formula (7), the larger the value, the greater the degree of abnormality of the object to be tested. Therefore, according to the network structure of different target categories, the target objects to be tested can be sorted from large to small according to the target characteristic index value, and the preset number of target objects to be tested in the front are taken as abnormal objects (target characteristic objects).
[0158] Figure 16 FIG. 1 is a block diagram of a data processing device according to an exemplary embodiment. Figure 16 The device shown can be applied to the server side of the above system, for example, and can also be applied to the terminal device of the above system.
[0159] refer to Figure 16 The device 160 provided in the embodiment of the present disclosure may include an associated object acquisition module 1602, a network construction module 1604 of an object to be measured, a group division module 1606 of an object to be measured, a target degree relationship network extraction module 1608, a target characteristic index acquisition module 1610 and a target characteristic object determination module 1612.
[0160] The associated object obtaining module 1602 may be configured to obtain associated objects corresponding to a plurality of objects to be detected, where a target association relationship exists between the associated objects and the corresponding objects to be detected.
[0161] The object network construction module 1604 may be used to obtain an object network composed of a plurality of objects under test. In the object network, connected objects under test have a common associated object.
[0162] The test object group division module 1606 may be configured to divide the test object network into a plurality of target test object groups based on the relationship between the associated objects corresponding to the connected test objects in the test object network and the common associated objects.
[0163] The target degree relationship network extraction module 1608 can be used to extract the target degree relationship network of the target objects in the target object group. The target degree relationship network of the target objects consists of the target objects and the objects that have a target degree relationship with the target objects.
[0164] The target characteristic index obtaining module 1610 can be used to determine whether the target degree relationship network of the target object to be tested belongs to the network structure of the target category, and obtain the target characteristic index of the target object to be tested, which is used to indicate the degree to which the target object to be tested meets the target characteristics.
[0165] The target characteristic object determination module 1612 may be configured to determine target characteristic objects in a group of multiple target objects to be measured according to target characteristic indicators of the target objects to be measured.
[0166] Figure 17 FIG. 1 is a block diagram of a data processing device according to an exemplary embodiment. Figure 17 The device shown can be applied to the server side of the above system, for example, and can also be applied to the terminal device of the above system.
[0167] refer to Figure 17The device 170 provided by the embodiment of the present disclosure may include an associated object acquisition module 1702, a network construction module 1704 of an object to be tested, a group division module 1706 of an object to be tested, a target degree relationship network extraction module 1708, a target characteristic index acquisition module 1710 and a target characteristic object determination module 1712, wherein the group division module 1706 of the object to be tested may include: a connection edge weight acquisition module 17062, a current group division result acquisition module 17064, a first modularity acquisition module 17066, a second group division result acquisition module 17068, a second modularity acquisition module 170610 and a current group division result update module 170612, the target characteristic index acquisition module 1710 may include: a target feature distribution acquisition module 17102, a network structure determination module 17104 and a target characteristic index calculation module 17106, the network structure determination module 17104 may include: a star or cluster structure determination module 171042, a total edge weight feature structure determination module 171044 and feature weight edge structure determination module 171046, target characteristic index calculation module 17106 may include: power law deviation index acquisition module 171062, local density index acquisition module 171064 and target characteristic index normalization module 171066, local density index acquisition module 171064 may include: target feature point acquisition module 1710642, average reachable distance acquisition module 1710644, target point local density acquisition module 1710646, The neighborhood point local density acquisition module 1710648 and the local density index calculation module 17106410, the target characteristic index normalization module 171066 may include: a power law deviation index total amount acquisition module 1710662, a power law deviation index normalization module 1710664, a local density index total amount acquisition module 1710666 and a local density index normalization module 1710668, the target characteristic object determination module 1712 may include: a target characteristic index sorting module 17122.
[0168] The associated object obtaining module 1702 may be configured to obtain associated objects corresponding to a plurality of objects to be detected, where a target association relationship exists between the associated objects and the corresponding objects to be detected.
[0169] The object network construction module 1704 may be used to obtain an object network composed of a plurality of objects under test. In the object network, connected objects under test have a common associated object.
[0170] The test object group division module 1706 may be configured to divide the test object network into a plurality of target test object groups based on the relationship between the associated objects corresponding to the connected test objects in the test object network and the common associated objects.
[0171] The object group division module 1706 may include: a connection edge weight acquisition module 17062, which is used to obtain the weight of the connection edges between the objects to be tested connected in the object network according to the quantitative relationship between the associated objects corresponding to each of the objects to be tested connected in the object network and the common associated objects.
[0172] The object group division module 1706 may also be configured to divide the object network into a plurality of object groups based on the weights of the connection edges between the connected objects in the object network.
[0173] The current group division result obtaining module 17064 may be used to divide each object to be tested in the network of objects to be tested into one of the multiple object to be tested groups, and obtain a first group division result as the current group division result.
[0174] The first modularity obtaining module 17066 may be configured to obtain the first modularity of the network of the objects to be tested according to the weights of the connection edges between the objects to be tested connected in the network of the objects to be tested and according to the first community division result.
[0175] The second community division result obtaining module 17068 may be used to move the current object to be tested to the object to be tested group where the object to be tested connected to it in the object to be tested network is located, to obtain a second community division result.
[0176] The second modularity obtaining module 170610 may be configured to obtain the second modularity of the network of objects to be tested according to the weights of the connection edges between the objects to be tested connected in the network of objects to be tested and according to the second community division result.
[0177] The current community division result updating module 170612 may be configured to update the current community division result to the second community division result if the second modularity is greater than the first modularity.
[0178] The target degree relationship network extraction module 1708 can be used to extract the target degree relationship network of the target objects in the target object group. The target degree relationship network of the target objects consists of the target objects and the objects that have a target degree relationship with the target objects.
[0179] The target characteristic index obtaining module 1710 can be used to determine the target degree relationship network of the target object to be tested belongs to the target category structure, and obtain the target characteristic index of the target object to be tested, which is used to indicate the degree to which the target object to be tested meets the target characteristics.
[0180] The target feature distribution obtaining module 17102 may be used to obtain the distribution of target features of each target object in the target degree relationship network of the target object.
[0181] The network structure determination module 17104 may be used to determine that the target degree relationship network of the target object to be tested belongs to the network structure of the target category if it is determined that the distribution of the target features of each target object to be tested conforms to the target power law model.
[0182] The target power law model includes the first target power law model, the second target power law model and the third target power law model; the network structure of the target category includes a star or cluster structure, a total edge weight feature structure and a feature weight edge structure.
[0183] The star-shaped or cluster-shaped structure determination module 171042 can be used to determine that the target degree relationship network of the target object to be tested belongs to a star-shaped or cluster-shaped structure if the relationship between the total number of connection edges in the target degree relationship network of each target object to be tested and the number of targets to be tested connected to each target object to be tested conforms to the first target power law model.
[0184] The total edge weight characteristic structure determination module 171044 can be used to determine that the target degree relationship network of the target object to be tested belongs to the total edge weight characteristic structure if the relationship between the sum of the weights of each connecting edge in the target degree relationship network of each target object to be tested and the total number of connecting edges conforms to the second target power law model.
[0185] The feature weight edge structure determination module 171046 can be used to determine whether the relationship between the target feature value of the target connection edge of each object to be tested and the sum of the weights of each connection edge conforms to the third target power law model, and determine whether the target degree relationship network of the target object to be tested belongs to the feature weight edge structure.
[0186] The target characteristic index calculation module 17106 can be used to obtain the target characteristic index of the target object to be measured based on the distribution of the target characteristics of each target object to be measured in the target degree relationship network and the target power law model.
[0187] The distribution of target features of each object to be measured in the target degree relationship network includes the distribution of the relationship between the actual value of the first feature and the actual value of the second feature of each object to be measured in the target degree relationship network.
[0188] The power-law deviation index obtaining module 171062 may be used to obtain a power-law deviation index of the target object to be measured according to the target characteristics of the target object to be measured and a target power-law model.
[0189] The power-law deviation index acquisition module 171062 can also be used to substitute the actual value of the second feature of the target object to be measured into the target power-law model to obtain the predicted value of the first feature of the target object to be measured; and obtain the power-law deviation index of the target object to be measured based on the actual value of the first feature of the target object to be measured and the predicted value of the first feature of the target object to be measured in the target power-law model.
[0190] The local density index obtaining module 171064 may be used to obtain the local density index of the target object to be measured according to the distribution of the target features of each target object to be measured in the target degree relationship network.
[0191] The target feature point acquisition module 1710642 can be used to obtain the target point of the target feature corresponding to the target object to be measured and the surrounding neighborhood points of the target point according to the distribution of the target features of each target object to be measured in the target degree relationship network.
[0192] The average reachable distance obtaining module 1710644 may be used to obtain the average reachable distance from the surrounding neighborhood points to the target point.
[0193] The target point local density obtaining module 1710646 may be used to obtain the local density of the target point according to the average reachable distance.
[0194] The neighborhood point local density obtaining module 1710648 may be used to obtain the local density of surrounding neighborhood points.
[0195] The local density index calculation module 17106410 may be used to obtain a local density index of the target object to be measured based on the local density of the surrounding neighborhood points and the local density of the target point.
[0196] The target characteristic index normalization module 171066 may be used to normalize the power law deviation index and the local density index of the target object to be measured respectively.
[0197] The target characteristic index calculation module 17106 can also be used to obtain the target characteristic index of the target object to be measured based on the normalized power law deviation index and the local density index.
[0198] The power-law deviation index total amount obtaining module 1710662 may be used to obtain a plurality of power-law deviation indicators corresponding to a plurality of target objects in a group of target objects.
[0199] The power-law deviation index normalization module 1710664 may be configured to obtain a normalized power-law deviation index of the target object to be measured based on the power-law deviation index of the target object to be measured and multiple power-law deviation indexes.
[0200] The local density index total amount obtaining module 1710666 may be used to obtain a plurality of local density indices corresponding to a plurality of target objects in a group of target objects.
[0201] The local density index normalization module 1710668 may be configured to obtain a normalized local density index of the target object to be measured based on the local density index of the target object to be measured and multiple local density indices.
[0202] The target characteristic object determination module 1712 may be configured to determine target characteristic objects in a group of multiple target objects to be measured according to target characteristic indicators of the target objects to be measured.
[0203] The target characteristic index sorting module 17122 may be used to sort multiple target objects in a group of target objects whose target degree relationship networks belong to the target category according to the numerical values of the target characteristic indexes.
[0204] The target characteristic object determining module 1712 may also be configured to determine the target characteristic object among the multiple target objects to be tested based on the numerical order of the target characteristic indicators of the multiple target objects to be tested.
[0205] The specific implementation of each module in the device provided by the embodiment of the present disclosure can refer to the content of the above method and will not be repeated here.
[0206] Figure 18 FIG. 1 shows a schematic diagram of the structure of an electronic device in an embodiment of the present disclosure. It should be noted that: Figure 18 The device shown is only an example of a computer system and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0207] like Figure 18 As shown, device 1800 includes a central processing unit (CPU) 1801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1802 or a program loaded from a storage portion 1808 into a random access memory (RAM) 1803. Various programs and data required for the operation of device 1800 are also stored in RAM 1803. CPU 1801, ROM 1802, and RAM 1803 are connected to each other via a bus 1804. An input / output (I / O) interface 1805 is also connected to bus 1804.
[0208] The following components are connected to the I / O interface 1805: an input section 1806 including a keyboard, a mouse, and the like; an output section 1807 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 1808 including a hard disk; and a communication section 1809 including a network interface card such as a LAN card or a modem. The communication section 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the I / O interface 1805 as needed. Removable media 1811, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1810 as needed, so that computer programs read therefrom can be installed into the storage section 1808 as needed.
[0209] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1809, and / or installed from a removable medium 1811. When the computer program is executed by the central processing unit (CPU) 1801, the above-mentioned functions defined in the system of the present disclosure are performed.
[0210] It should be noted that the computer-readable medium described in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.
[0211] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0212] The modules involved in the embodiments described in the present disclosure may be implemented in software or in hardware. The modules described may also be provided in a processor. For example, they may be described as follows: a processor including an associated object acquisition module, a network construction module for objects to be tested, a group division module for objects to be tested, a target degree relationship network extraction module, a target characteristic index acquisition module, and a target characteristic object determination module. The names of these modules do not, in certain cases, constitute a limitation on the modules themselves. For example, the associated object acquisition module may also be described as a "module for obtaining associated objects corresponding to each object to be tested."
[0213] The present disclosure also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being incorporated into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device includes:
[0214] Obtain associated objects corresponding to a plurality of objects to be tested, wherein target association relationships exist between the associated objects and the corresponding objects to be tested; obtain an object network composed of a plurality of objects to be tested, wherein the connected objects to be tested have common associated objects in the object network; divide the object network into a plurality of target object groups based on the relationships between the associated objects corresponding to the connected objects to be tested in the object network and the common associated objects; extract a target degree relationship network of the target objects to be tested in the target object groups, wherein the target degree relationship network of the target objects to be tested consists of the target objects to be tested and the objects to be tested that have target degree relationships with the target objects to be tested; determine whether the target degree relationship network of the target objects to be tested belongs to a network structure of a target category, obtain a target characteristic index of the target objects to be tested, wherein the target characteristic index is used to indicate the degree to which the target objects to be tested meet the target characteristics; and determine the target characteristic objects in the plurality of target object groups based on the target characteristic index of the target objects to be tested.
[0215] The present disclosure provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.
[0216] While the exemplary embodiments of the present disclosure have been specifically illustrated and described above, it should be understood that the present disclosure is not limited to the detailed structures, configurations, or implementations described herein; rather, the present disclosure is intended to encompass various modifications and equivalent configurations within the spirit and scope of the appended claims.
Claims
1. A data processing method, characterized in that: include: Obtaining associated objects corresponding to each of the plurality of objects to be tested, wherein a target association relationship exists between the associated objects and the corresponding objects to be tested; Obtaining a network of objects to be tested consisting of the plurality of objects to be tested, wherein connected objects to be tested in the network have a common associated object; Dividing the network of objects to be tested into a plurality of target groups of objects to be tested based on the relationship between the associated objects corresponding to the respective associated objects and the common associated objects connected to the network of objects to be tested; Extracting a target degree relationship network of target objects to be tested in the target object group, wherein the target degree relationship network of target objects to be tested is composed of the target objects to be tested and objects to be tested that have a target degree relationship with the target objects to be tested; Obtaining the distribution of target features of each target object in the target degree relationship network of the target object to be measured; If it is determined that the distribution of the target features of each target object in the target degree relationship network conforms to the target power law model, then it is determined that the target degree relationship network of the target target object belongs to the network structure of the target category; Obtaining a target characteristic index of the target object to be measured based on the distribution of target characteristics of each target object to be measured in the target degree relationship network and the target power law model, wherein the target characteristic index is used to indicate the degree to which the target object to be measured meets the target characteristic; According to the target characteristic indicators of the target objects to be measured, target characteristic objects in the group of multiple target objects to be measured are determined.
2. The method according to claim 1, characterized in that Obtaining target characteristic indicators of the target object to be measured according to the distribution of target characteristics of each target object to be measured in the target degree relationship network and the target power law model, including: Obtaining a power-law deviation index of the target object to be measured according to the target feature of the target object to be measured and the target power-law model; Obtaining a local density index of the target object to be measured according to the distribution of target features of each target object to be measured in the target degree relationship network; Normalizing the power law deviation index and the local density index of the target object to be measured respectively; According to the normalized power law deviation index and local density index, a target characteristic index of the target object to be measured is obtained.
3. The method according to claim 2, characterized in that The distribution of target features of each object to be measured in the target degree relationship network includes the distribution of the relationship between the actual value of the first feature and the actual value of the second feature of each object to be measured in the target degree relationship network; Obtaining a power-law deviation index of the target object to be measured according to the target feature of the target object to be measured and the target power-law model includes: Substituting the actual value of the second characteristic of the target object to be measured into the target power law model to obtain a predicted value of the first characteristic of the target object to be measured; A power-law deviation index of the target object to be measured is obtained according to an actual value of the first feature of the target object to be measured and a predicted value of the first feature of the target object to be measured in the target power-law model.
4. The method according to claim 2, characterized in that Obtaining a local density index of the target object to be measured according to the distribution of the target features of each target object in the target degree relationship network includes: According to the distribution of target features of each target object in the target degree relationship network, a target point of the target feature corresponding to the target object and surrounding neighborhood points of the target point are obtained; Obtaining the average reachable distance from the surrounding neighborhood points to the target point; Obtaining the local density of the target point according to the average reachable distance; Obtaining the local density of the surrounding neighborhood points; A local density index of the target object to be measured is obtained according to the local density of the surrounding neighborhood points and the local density of the target point.
5. The method according to claim 2, characterized in that Normalizing the power law deviation index and the local density index of the target object to be measured respectively includes: Obtaining a plurality of power-law deviation indicators respectively corresponding to a plurality of target objects to be measured in the group of target objects to be measured; Obtaining a normalized power-law deviation index of the target object to be measured according to the power-law deviation index of the target object to be measured and the multiple power-law deviation indexes; Obtaining a plurality of local density indicators respectively corresponding to a plurality of target objects in the group of target objects to be measured; A normalized local density index of the target object to be measured is obtained according to the local density index of the target object to be measured and the multiple local density indexes.
6. The method according to claim 1, characterized in that The target power law model includes a first target power law model, a second target power law model and a third target power law model; The network structure of the target category includes a star or cluster structure, a total edge weight feature structure, and a feature weight edge structure; If it is determined that the distribution of the target features of the respective objects to be tested conforms to the target power law model, then determining that the target degree relationship network of the target objects to be tested belongs to the network structure of the target category includes: If it is determined that the relationship between the total number of connected edges in the target degree relationship network of each target object and the number of targets connected to each target object conforms to the first target power law model, then it is determined that the target degree relationship network of the target target object belongs to the star-shaped or cluster-shaped structure; If it is determined that the relationship between the sum of the weights of the connecting edges in the target degree relationship network of each target object to be tested and the total number of connecting edges conforms to the second target power law model, then it is determined that the target degree relationship network of the target target object to be tested belongs to the total edge weight characteristic structure; If it is determined that the relationship between the target characteristic values of the target connection edges of each object to be tested and the sum of the weights of each connection edge conforms to the third target power law model, then it is determined that the target degree relationship network of the target object to be tested belongs to the feature weight edge structure.
7. The method according to claim 1, characterized in that Based on the relationship between the associated objects corresponding to the connected objects under test in the network of objects under test and the common associated objects, the network of objects under test is divided into a plurality of target groups of objects under test, including: Obtaining weights of connection edges between connected objects to be tested in the object network according to a quantitative relationship between associated objects corresponding to respective connected objects to be tested and common associated objects in the object network; The network of objects to be tested is divided into a plurality of groups of objects to be tested based on the weights of the connection edges between the objects to be tested connected in the network of objects to be tested.
8. The method according to claim 7, characterized in that Dividing the network of objects to be tested into a plurality of groups of objects to be tested based on the weights of the connection edges between the objects to be tested connected in the network of objects to be tested, including: Dividing each object to be tested in the network of objects to be tested into one of the plurality of object to be tested groups, and obtaining a first group division result as a current group division result; Obtaining a first modularity of the network of objects to be tested according to the weights of the connection edges between the objects to be tested connected in the network of objects to be tested and according to the first community division result; Moving the current object to be tested to the object group to which the object to be tested connected in the object network to be tested belongs, to obtain a second group division result; Obtaining a second modularity of the network of objects to be tested according to the weights of the connection edges between the objects to be tested connected in the network of objects to be tested and according to the second community division result; If the second modularity is greater than the first modularity, the current community division result is updated to the second community division result.
9. The method according to claim 1, characterized in that Determining target characteristic objects in a plurality of target object groups according to target characteristic indicators of the target objects to be measured includes: Sorting the target objects in the target object group whose target degree relationship network belongs to the target category according to the numerical values of the target characteristic indicators; The target characteristic object among the multiple target objects to be measured is determined according to the result of sorting the target characteristic index values of the multiple target objects to be measured.
10. A data processing device, characterized in that: include: An associated object obtaining module is used to obtain associated objects corresponding to each of the plurality of objects to be tested, wherein a target association relationship exists between the associated objects and the corresponding objects to be tested; A test object network construction module is used to obtain a test object network composed of the multiple test objects, in which the connected test objects have a common associated object; a test object group division module, configured to divide the test object network into a plurality of target test object groups based on the relationship between the associated objects corresponding to the connected test objects in the test object network and the common associated objects; a target degree relationship network extraction module, configured to extract a target degree relationship network of target objects in the target object group, wherein the target degree relationship network of the target objects is composed of the target objects and objects having a target degree relationship with the target objects; A target feature distribution acquisition module is used to obtain the distribution of target features of each target object in the target degree relationship network of the target object to be tested; a network structure determination module, configured to determine that the target degree relationship network of the target object to be measured belongs to the network structure of the target category if it is determined that the distribution of the target characteristics of each target object to be measured in the target degree relationship network conforms to the target power law model; a target characteristic index calculation module, configured to obtain a target characteristic index of the target object to be measured based on the distribution of target characteristics of each target object to be measured in the target degree relationship network and the target power law model, wherein the target characteristic index is used to indicate the degree to which the target object to be measured meets the target characteristic; The target characteristic object determination module is used to determine the target characteristic objects in the plurality of target characteristic objects group according to the target characteristic indicators of the target objects to be measured.
11. An electronic device comprising: A memory, a processor, and executable instructions stored in the memory and executable in the processor, wherein the processor implements the method according to any one of claims 1 to 9 when executing the executable instructions.
12. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.
13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 9 is implemented.
Citation Information
Patent Citations
Object estimation method and device, equipment and storage medium
CN112801377A
Monitoring method for network performance data, and related device
WO2021208877A1