Artificial intelligence-based anomaly detection method, device, computer equipment, and medium

Abstract

The present application belongs to the field of artificial intelligence, and relates to an anomaly detection method based on artificial intelligence and related equipment. The method includes obtaining the current state of the enterprise to be detected and the set of optional actions corresponding to the current state, and inputting the current state into a preset reinforcement learning model for risk detection, determining all the profit forecast values ​​corresponding to each optional action in the optional action set, and then selecting the maximum profit forecast value from all the profit forecast values. Finally, the maximum profit forecast value is compared with the actual profit value corresponding to the current state, and according to the comparison result, the enterprise to be detected is determined to be a risk enterprise. The present application also involves blockchain technology, and enterprise information and reinforcement learning models can be stored in the blockchain. The present application can obtain the maximum value of all expected prediction values ​​corresponding to the current state through the reinforcement learning model to accurately detect whether the enterprise is a risk enterprise.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to anomaly detection methods, devices, computer equipment, and storage media based on artificial intelligence. Background Art

[0002] Corporate clients are crucial to insurance companies as large clients, and different types of businesses have a wide range of customized and personalized products. Therefore, it's difficult to identify corporate claims risk using fixed labels, and risk screening often relies on the company's industry and region. This makes hidden corporate risks more difficult to uncover than in auto and non-auto insurance. Because corporate feature labels are difficult to pin down and customization is excessive, building a risk detection model using supervised learning binary classification algorithms on labeled data is unwise. This model suffers from numerous undefined features and overly broad labels, making it ineffective for detecting corporate risks. Furthermore, this approach is not suitable for corporate enterprise scenarios. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to propose an artificial intelligence-based anomaly detection method, device, computer equipment and storage medium to solve the technical problem that the risk detection model of the binary classification algorithm in existing supervised learning cannot effectively detect enterprise risks.

[0004] In order to solve the above technical problems, the present invention provides an artificial intelligence-based anomaly detection method, which includes the following steps:

[0005] Obtain the current status of the enterprise to be detected and a set of optional actions corresponding to the current status;

[0006] Inputting the current state into a preset reinforcement learning model for risk detection to determine all predicted benefits corresponding to each optional action in the optional action set;

[0007] Selecting the maximum profit forecast value from all the profit forecast values;

[0008] Comparing the maximum profit prediction value with the actual profit value corresponding to the current state to obtain a comparison result;

[0009] According to the comparison result, the enterprise to be detected is determined to be a risky enterprise.

[0010] Furthermore, before the step of inputting the current state into a preset reinforcement learning model for risk detection, the method further includes:

[0011] Acquire enterprise training samples, wherein the enterprise training samples include state data of each sample enterprise from an initial state to a final state and interaction actions corresponding to each state data;

[0012] Determining the expected value of the benefit of the interactive action corresponding to each of the state data;

[0013] Based on each of the state data and the expected value of the benefit of the interactive action corresponding to each of the state data, the reinforcement learning model is trained to obtain a trained reinforcement learning model, and the trained reinforcement learning model is used as the preset reinforcement learning model.

[0014] Furthermore, the step of determining the expected value of the benefit of the interaction action corresponding to each of the state data includes:

[0015] Each state data is input into a neural network, and the expected benefit value of the interactive action corresponding to each state data is calculated by the neural network, wherein each state data corresponds to at least one interactive action, and each state data has at least one expected benefit value.

[0016] Furthermore, when each state data corresponds to a plurality of interactive actions, the step of inputting each state data into a neural network and calculating the expected value of the benefit of each interactive action corresponding to the state data through the neural network includes:

[0017] Input the current state into a pre-trained intention recognition model to obtain the next state intention of the enterprise;

[0018] Based on the next state intention of the enterprise, determining actions that match the next state intention of the enterprise from the multiple interaction actions, classifying the actions into an intended action set, and classifying the remaining interaction actions into a prohibited action set;

[0019] Each of the state data is input into a neural network, and the expected value of the benefit of each intended action in the intended action set corresponding to each of the state data is calculated by the neural network.

[0020] Furthermore, before the step of obtaining enterprise training samples, the method further includes:

[0021] Match all enterprise sample data with a blacklist, which includes enterprises in shady industries, shady areas, and enterprises with historical fraud risks;

[0022] When the blacklist is matched successfully, the enterprise sample data that is matched successfully is eliminated, and the remaining enterprise sample data is used as enterprise training samples.

[0023] Furthermore, the step of training the reinforcement learning model based on each state data and the expected value of the benefit of the interaction action corresponding to each state data includes:

[0024] Obtaining the maximum expected value of the next state data corresponding to each state data;

[0025] According to the maximum expected benefit value, the preset decay value, the reward decay coefficient, and the return value corresponding to each state data, the expected benefit value of the interactive action corresponding to each state data is updated to obtain the final expected benefit value of the interactive action corresponding to each state data;

[0026] The reinforcement learning model is trained based on each state data and the expected value of the final benefit of the interaction action corresponding to each state data.

[0027] Furthermore, the step of determining that the enterprise to be detected is a risk enterprise based on the comparison result includes:

[0028] If the maximum profit prediction value is less than the actual profit value corresponding to the current state, it is determined that the enterprise to be detected has a fraud risk;

[0029] If the maximum profit prediction value is greater than the actual profit value corresponding to the current state, the process returns to the step of obtaining the current state of the enterprise to be detected and the optional action set corresponding to the current state.

[0030] In order to solve the above technical problems, the present application also provides an artificial intelligence-based anomaly detection device, including:

[0031] An acquisition module, configured to acquire the current status of the enterprise to be detected and a set of optional actions corresponding to the current status;

[0032] An input module, configured to input the current state into a preset reinforcement learning model for risk detection, and determine all predicted benefits corresponding to each optional action in the optional action set;

[0033] A selection module, configured to select a maximum profit prediction value from all the profit prediction values;

[0034] A comparison module, configured to compare the maximum profit prediction value with the actual profit value corresponding to the current state to obtain a comparison result;

[0035] The determination module is used to determine that the enterprise to be detected is a risk enterprise based on the comparison results, and adopts the following technical solution:

[0036] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of any of the above-described artificial intelligence-based anomaly detection methods.

[0037] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor, the steps of any of the above-described artificial intelligence-based anomaly detection methods are implemented.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0039] The technical solution provided by this application obtains the current state of the enterprise to be detected and the set of optional actions corresponding to the current state, and inputs the current state into a preset reinforcement learning model for risk detection, determines all the profit prediction values ​​corresponding to each optional action in the optional action set, and then selects the maximum profit prediction value from all the profit prediction values. Finally, the maximum profit prediction value is compared with the actual profit value corresponding to the current state. After obtaining the comparison result, the enterprise to be detected is determined to be a risky enterprise based on the comparison result. In this way, the expected prediction value corresponding to the current state of the enterprise to be detected can be accurately obtained through the reinforcement learning model, so that the enterprise can be accurately detected as a risky enterprise based on the maximum value of the expected prediction value. In addition, the above-mentioned detection method is universal, does not rely on environmental characteristics, can greatly improve detection accuracy, and is suitable for most group scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0042] Figure 2 A flowchart of an embodiment of an artificial intelligence-based anomaly detection method according to the present application;

[0043] Figure 3 is a flowchart of another embodiment of the artificial intelligence-based anomaly detection method of the present application;

[0044] Figure 4 is a structural diagram of an embodiment of an artificial intelligence-based anomaly detection device according to the present application;

[0045] Figure 5 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0047] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0050] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0051] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0052] Electronic devices generally include PC-based electronic devices and mobile electronic devices. PC-based electronic devices, such as all-in-one computers, may have operating systems including, but not limited to, Linux, Unix, Windows systems (such as Windows XP and Windows 7), and Mac OS X (Apple's operating system). Mobile electronic devices, such as smartphones, may have operating systems including, but not limited to, Android, iOS (Apple's operating system), and Windows.

[0053] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0054] It should be noted that the artificial intelligence-based anomaly detection method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based anomaly detection device is generally set in the server / terminal device.

[0055] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0056] Specifically, this AI-based anomaly detection method can be applied to electronic devices or servers deployed by financial institutions such as banks, securities companies, and insurance companies. Currently, applications in the financial sector are a hot topic in reinforcement learning research, primarily used for analysis, research, and decision-making in financial markets.

[0057] Reinforcement learning is a machine learning algorithm that maximizes rewards by allowing an agent to move relative to its environment. Therefore, reinforcement learning algorithms have several basic elements: the agent, the environment, the state, the action, and the reward. To facilitate understanding, the following section introduces these basic concepts.

[0058] (1) Intelligent agents, also known as “agents,” “agents,” or “intelligent subjects.” Intelligent agents can automatically adjust their behavior and state based on changes in the external environment, rather than simply passively accepting external stimuli. They have the ability to self-manage and self-regulate. In addition, intelligent agents can accumulate or learn experience and knowledge and modify their behavior to adapt to new environments.

[0059] (2) The environment is the part of the system other than the agent. It can provide feedback on status and rewards to the agent and can also change according to certain rules. In the financial field, the environment can be the financial market.

[0060] (3) State refers to the objective conditions of the system in each time period. For a certain underlying asset in the financial market, it can be in three states in a certain time period: rising, falling, and consolidating.

[0061] (4) Action, also known as decision-making. After the time and state are determined, the agent will make different choices based on the state of the environment, so that the current state can be transferred to the next state with certainty or with a certain probability. This process is called an action. (5) Return, also known as reward, can be defined as the subsequent benefits brought about by taking a certain action. Returns can be positive or negative.

[0062] Reinforcement learning algorithms can be broadly categorized into value-based algorithms and policy-based algorithms. A typical value-based algorithm is the Q-Learning algorithm, while a typical policy-based algorithm is the policy gradient (PG) algorithm.

[0063] In the technical field of this application, the Q-Learning algorithm is preferably used. The Q-Learning algorithm primarily defines market states. It then selects trading actions based on a greedy (ε-greedy) strategy, interacting with the environment to obtain rewards. The key idea of ​​this algorithm is to construct a Q-value table that stores Q values ​​based on states and actions, and then update the Q-value table based on rewards, thereby optimizing trading actions.

[0064] Continue to refer Figure 2, shows a flow chart of an embodiment of an artificial intelligence-based anomaly detection method according to the present application. The artificial intelligence-based anomaly detection method includes the following steps:

[0065] Step S201: Acquire the current status of the enterprise to be detected and a set of optional actions corresponding to the current status.

[0066] In this step, the current status of the enterprise to be tested is obtained. It should be understood that these enterprises to be tested may include business platforms and enterprises using interactive services provided by these platforms, such as automotive and real estate companies. Accordingly, the status of the enterprise to be tested may be information describing the enterprise to be tested. The specific information obtained can be determined based on the actual prediction task. The current status mentioned above may include information such as inquiry status, quotation status, insurance status, price confirmation status, review status, and case closure status.

[0067] The optional action set corresponding to the current state is the action that executes the next round of interaction when the current state is the current state. For example, if the current state is the inquiry state, the optional action set includes the action that moves to the quotation state and the action that moves to the insurance state in the next round. Alternatively, if the current state is the review state, the optional action set includes the action that moves to the price confirmation state and the action that moves to the case closing state in the next round. In other words, the optional action set is the set of actions that move from the current state to the next round of state.

[0068] Step S202: Input the current state into a preset reinforcement learning model for risk detection to determine all predicted benefits corresponding to each optional action in the optional action set.

[0069] In this embodiment, the current state is input into a preset reinforcement learning model for risk detection, and all profit prediction values ​​corresponding to each optional action in the optional action set corresponding to the above current state are calculated by the preset reinforcement learning model.

[0070] The preset reinforcement learning model calculates multiple predicted benefits corresponding to each optional action in the optional action set under the current state. It should be noted that when calculating the predicted benefits based on the preset reinforcement learning model, a greedy strategy can be directly used to directly calculate the multiple predicted benefits corresponding to each optional action in the optional action set.

[0071] Specifically, each optional action corresponds to an expected return value, and each expected return value corresponds to a state in the next round. That is, after the optional action is applied to the current state of the enterprise under test, the current state of the enterprise under test changes, and the changed state of the enterprise under test, or the next state, can be obtained accordingly. The current state is the state of this round.

[0072] It should be noted that there may be only one action in the optional action set, that is, when the action is applied to the enterprise to be detected, the enterprise to be detected changes from the current state to the next state, or the enterprise to be detected maintains the current state (in this case, the current state is the middle state of the enterprise to be detected.

[0073] Step S203, selecting the maximum profit prediction value from all the profit prediction values;

[0074] Step S204, comparing the maximum profit prediction value with the actual profit value corresponding to the current state to obtain a comparison result;

[0075] Step S205: Determine, based on the comparison result, that the enterprise to be detected is a risky enterprise.

[0076] In this embodiment, the maximum predicted revenue value is selected from all the predicted revenue values ​​corresponding to the optional actions in the set of optional actions corresponding to the current state. For example, when the current state is the Review state, Optional Action A is the action that leads to the Confirmation state in the next round, and its predicted revenue value is A1. Optional Action B is the action that leads to the Closure state in the next round, and its predicted revenue value is B1. Where A1 < B1, the next state of the current state is determined to be Optional Action B with the largest predicted revenue value, which is the Closure state.

[0077] Furthermore, after determining the maximum benefit prediction value, the actual benefit value corresponding to the current state is calculated. In this embodiment, a neural network can be used to calculate the actual benefit value corresponding to the current state. It should be understood that the neural network is also based on calculating the expected benefit corresponding to the state (state)-action (action) pair. The benefit value is also called the Q value. The neural network used to calculate the Q value can also be called a Q-Network or Q network. In one embodiment, the neural network can be implemented as a DNN network or a CNN network, etc.

[0078] The actual profit value is calculated, and the maximum profit prediction value is compared with the actual profit value corresponding to the current state to obtain a comparison result. Based on the comparison result, the enterprise to be detected is determined to be a risky enterprise.

[0079] Specifically, if the maximum profit prediction value is less than the actual profit value corresponding to the current state, the enterprise to be tested is determined to be a risky enterprise. Here, a risky enterprise refers to an enterprise that engages in fraudulent activities, such as providing false information to defraud claims, and is not limited to this. If the maximum profit prediction value is greater than the actual profit value corresponding to the current state, the process returns to the step of obtaining the current state of the enterprise to be tested and the set of optional actions corresponding to the current state, i.e., returns to step S202, and retests.

[0080] The technical solution provided by this application obtains the current state of the enterprise to be detected and the set of optional actions corresponding to the current state, and inputs the current state into a preset reinforcement learning model for risk detection, determines all the profit prediction values ​​corresponding to each optional action in the optional action set, and then selects the maximum profit prediction value from all the profit prediction values. Finally, the maximum profit prediction value is compared with the actual profit value corresponding to the current state. After obtaining the comparison result, the enterprise to be detected is determined to be a risky enterprise based on the comparison result. In this way, the expected prediction value corresponding to the current state of the enterprise to be detected can be accurately obtained through the reinforcement learning model, so that the enterprise can be accurately detected as a risky enterprise based on the maximum value of the expected prediction value. In addition, the above-mentioned detection method is universal, does not rely on environmental characteristics, can greatly improve detection accuracy, and is suitable for most group scenarios.

[0081] In some optional implementations of this embodiment, such as Figure 3 As shown, before inputting the current state into the preset reinforcement learning model for risk detection, the electronic device may further perform the following steps:

[0082] Step S2011: Acquire enterprise training samples, wherein the enterprise training samples include each state data from the initial state to the final state of each sample enterprise and the interaction actions corresponding to each state data.

[0083] In this step, enterprise training samples are obtained, wherein the enterprise training samples include the various state data of each sample enterprise from the initial state to the final state and the interactive actions corresponding to each state data. Here, the enterprise training samples can be closed enterprise case data stored in an open database or cloud. The so-called "closed" refers to enterprise cases that have completed all processes. For example, closed enterprise case data includes relevant data for all states from inquiry to closing. Inquiry, quotation, etc., are each considered a separate state, and the information corresponding to each separate state is the state data. That is, the enterprise training samples obtained above include the various state data of each sample enterprise from the initial state (such as the inquiry state) to the final state (such as the closed state) and the interactive actions corresponding to each state data. There is at least one interactive action corresponding to each state data. Exceptionally, the interactive action for the final state is one, that is, from the final state to the final state.

[0084] Furthermore, to ensure the reliability of enterprise training samples, this embodiment performs a blacklist match against all enterprise sample data before obtaining them. This blacklist includes enterprises in shady industries, those in shady areas, and those with historical fraud risks. If a blacklist match is successful, the matched enterprise sample data is removed, and the remaining enterprise sample data is used as the enterprise training sample. By removing enterprises in shady industries, those in shady areas, and those with historical fraud risks, the remaining enterprise training samples do not affect the weights of subsequent reinforcement learning model training, further increasing the prediction accuracy of the reinforcement learning model.

[0085] Step S2012: determining the expected value of the benefit of each interactive action corresponding to the state data.

[0086] After obtaining each state data of each sample enterprise from the initial state to the final state and the interaction actions corresponding to each state data, the expected value of the benefit of each interaction action corresponding to the state data is determined.

[0087] Specifically, each of the state data is input into a neural network, and the expected value of the benefit of the interactive action corresponding to each of the state data is calculated by the neural network, wherein each of the state data corresponds to at least one interactive action, and each of the state data has at least one expected value of benefit. In this embodiment, a neural network can be used to calculate the predicted benefit value corresponding to the interactive action of each of the state data. It should be understood that the neural network is used to calculate the expected benefit corresponding to the state (state)-action (action) pair, and the benefit value is also called the Q value. The neural network used to calculate the Q value can also be called Q-Network or Q network. In one embodiment, the neural network can be implemented as a DNN network or a CNN network, etc.

[0088] When there are multiple interactive actions corresponding to each of the state data, the current state is input into a pre-trained intention recognition model to obtain the next state intention of the enterprise, and then based on the next state intention of the enterprise, the actions that match the next state intention of the enterprise are determined from the multiple interactive actions, classified as the intention action set, and the remaining interactive actions are classified as the prohibited action set. Among them, the intention recognition model is a pre-trained machine learning model. In a specific embodiment, the intention recognition model can be implemented based on a deep neural network (DNN) or a Bert model. It should be understood that the intention recognition model can be obtained by supervised learning, and its training process is similar to the use process. The main difference is that the training data used in the training process carries the intent category label, and the model parameters are adjusted and changed during the training process, while the use process uses the trained intention recognition model, and its model parameters will not change during the use process. Therefore, the training of the intention recognition model can refer to its use, and the training process will no longer be described separately.

[0089] In a specific embodiment, based on the pre-established association between the enterprise's next-state intention and intended actions, several interactive actions associated with the enterprise's next-state intention can be determined and included in the intended action set. In another specific embodiment, the similarity between the enterprise's next-state intention and each interactive action can be calculated separately, and then several interactive actions with similarities above a preset threshold can be included in the intended action set. In this way, it is possible to divide the intended action set that conforms to the enterprise's next-state intention and the prohibited action set that deviates from the user's intention based on the determined explicit next-state intention of the enterprise.

[0090] In another embodiment, this step may include calculating the similarity between the interaction action and each intended action, then classifying the interaction actions with similarity above a preset threshold into the intended action set, and classifying the remaining interaction actions into the prohibited action set. In this way, the intended action set and the prohibited action set can be divided according to the implicit user intent.

[0091] Each state data is input into a neural network, and the neural network calculates the expected value of the benefit of each intended action in the intended action set corresponding to each state data, wherein there is at least one intended action in the intended action set.

[0092] Step S2013: Based on each of the state data and the expected value of the benefit of the interactive action corresponding to each of the state data, the reinforcement learning model is trained to obtain a trained reinforcement learning model, and the trained reinforcement learning model is used as the preset reinforcement learning model.

[0093] Furthermore, based on each of the state data and the expected benefit value of the interactive action corresponding to each of the state data (here, if there is an intended action, the expected benefit value of the interactive action is the expected benefit value of the intended action), the reinforcement learning model is trained to obtain a trained reinforcement learning model, and the trained reinforcement learning model is used as the preset reinforcement learning model.

[0094] Specifically, the maximum expected benefit value Q(s', a') of the next state data corresponding to each state data is obtained, and the expected benefit value Q(s', a') of the interactive action corresponding to each state data is updated according to the maximum expected benefit value Q(s', a'), the preset decay value γ, the reward decay coefficient α, and the return value r corresponding to each state data, to obtain the final expected benefit value of the interactive action corresponding to each state data, that is, the final expected benefit value is:

[0095] Q(s, a)←Q(s, a)+α[γ*maxa'*Q(s', a')-Q(s, a)];

[0096] Among them, the reward decay coefficient α is the learning rate and γ is the reward decay coefficient.

[0097] After obtaining the expected value of the final benefit, the reinforcement learning model is trained based on each of the state data and the expected value of the final benefit of the interactive action corresponding to each of the state data to obtain a trained reinforcement learning model, and the trained reinforcement learning model is used as the preset reinforcement learning model.

[0098] When taking the optimal action for the current state, there is always a deviation from the previous action. This embodiment uses a temporal difference method to update the Q value, as described above. This reduces the deviation of the reinforcement learning model and increases the accuracy of the reinforcement learning model's detection. Furthermore, this reinforcement learning model training method reduces manual feature extraction, saving resources and reducing costs.

[0099] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned enterprise information and reinforcement learning model, the above-mentioned enterprise information and reinforcement learning model can also be stored in a node of a blockchain.

[0100] The blockchain referred to in this application is a new application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0101] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0102] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0103] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0104] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0105] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of an abnormality detection device based on artificial intelligence, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0106] like Figure 4As shown, the artificial intelligence-based anomaly detection device 400 described in this embodiment includes: an acquisition module 401, an input module 402, a selection module 403, a comparison module 404 and a confirmation module 405.

[0107] in:

[0108] An acquisition module 401 is configured to acquire the current status of the enterprise to be detected and a set of optional actions corresponding to the current status;

[0109] An input module 402 is configured to input the current state into a preset reinforcement learning model for risk detection, and determine all predicted benefits corresponding to each optional action in the optional action set;

[0110] A selection module 403 is configured to select a maximum revenue prediction value from all the revenue prediction values;

[0111] A comparison module 404 is configured to compare the maximum revenue prediction value with the actual revenue value corresponding to the current state to obtain a comparison result;

[0112] The determination module 405 is configured to determine, based on the comparison result, whether the enterprise to be detected is a risky enterprise.

[0113] The technical solution provided by this application is that the acquisition module 401 acquires the current state of the enterprise to be detected and the set of optional actions corresponding to the current state, and inputs the current state into the preset reinforcement learning model through the input module 402 for risk detection, determines all the profit prediction values ​​corresponding to each optional action in the optional action set, and then selects the maximum profit prediction value from all the profit prediction values ​​through the selection module 403, and finally compares the maximum profit prediction value with the actual profit value corresponding to the current state through the comparison module 404. After obtaining the comparison result, the determination module 405 determines that the enterprise to be detected is a risk enterprise based on the comparison result. In this way, the expected prediction value corresponding to the current state of the enterprise to be detected can be accurately obtained through the reinforcement learning model, so that the enterprise can be accurately detected as a risk enterprise based on the maximum value of the expected prediction value. In addition, the above-mentioned detection method is universal, does not rely on environmental characteristics, can greatly improve the detection accuracy, and is suitable for most group scenarios.

[0114] To solve the above technical problems, the present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.

[0115] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 6 with components 61-63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0116] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0117] The memory 61 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as the hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions for an artificial intelligence-based anomaly detection method. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0118] In some embodiments, the processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to execute computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for executing the artificial intelligence-based anomaly detection method.

[0119] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0120] In this application, the current state of the enterprise to be detected and the set of optional actions corresponding to the current state are obtained, and the current state is input into a preset reinforcement learning model for risk detection, all the profit prediction values ​​corresponding to the optional actions in the optional action set are determined, and then the maximum profit prediction value is selected from all the profit prediction values. Finally, the maximum profit prediction value is compared with the actual profit value corresponding to the current state. After obtaining the comparison result, the enterprise to be detected is determined to be a risky enterprise based on the comparison result. In this way, the expected prediction value corresponding to the current state of the enterprise to be detected can be accurately obtained through the reinforcement learning model, so that the enterprise can be accurately detected as a risky enterprise based on the maximum value of the expected prediction value. In addition, the above-mentioned detection method is universal, does not rely on environmental characteristics, can greatly improve the detection accuracy, and is suitable for most group scenarios.

[0121] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned artificial intelligence-based anomaly detection method.

[0122] The technical solution provided by this application obtains the current state of the enterprise to be detected and the set of optional actions corresponding to the current state, and inputs the current state into a preset reinforcement learning model for risk detection, determines all the profit prediction values ​​corresponding to each optional action in the optional action set, and then selects the maximum profit prediction value from all the profit prediction values. Finally, the maximum profit prediction value is compared with the actual profit value corresponding to the current state. After obtaining the comparison result, the enterprise to be detected is determined to be a risky enterprise based on the comparison result. In this way, the expected prediction value corresponding to the current state of the enterprise to be detected can be accurately obtained through the reinforcement learning model, so that the enterprise can be accurately detected as a risky enterprise based on the maximum value of the expected prediction value. In addition, the above-mentioned detection method is universal, does not rely on environmental characteristics, can greatly improve detection accuracy, and is suitable for most group scenarios.

[0123] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0124] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. An artificial intelligence-based anomaly detection method, characterized in that: The steps include: Obtain the current status of the enterprise to be detected and a set of optional actions corresponding to the current status; Inputting the current state into a preset reinforcement learning model for risk detection to determine all predicted benefits corresponding to each optional action in the optional action set; Selecting the maximum profit forecast value from all the profit forecast values; Comparing the maximum profit prediction value with the actual profit value corresponding to the current state to obtain a comparison result; According to the comparison result, the enterprise to be detected is determined to be a risk enterprise, wherein the step of determining the enterprise to be detected to be a risk enterprise according to the comparison result includes: If the maximum profit prediction value is less than the actual profit value corresponding to the current state, it is determined that the enterprise to be detected has a fraud risk; If the maximum profit prediction value is greater than the actual profit value corresponding to the current state, the process returns to the step of obtaining the current state of the enterprise to be detected and the optional action set corresponding to the current state.

2. The artificial intelligence-based anomaly detection method according to claim 1, characterized in that: Before the step of inputting the current state into a preset reinforcement learning model for risk detection, the method further includes: Acquire enterprise training samples, wherein the enterprise training samples include state data of each sample enterprise from an initial state to a final state and interaction actions corresponding to each state data; Determining the expected value of the benefit of the interactive action corresponding to each of the state data; Based on each of the state data and the expected value of the benefit of the interactive action corresponding to each of the state data, the reinforcement learning model is trained to obtain a trained reinforcement learning model, and the trained reinforcement learning model is used as the preset reinforcement learning model.

3. The artificial intelligence-based anomaly detection method according to claim 2, characterized in that: The step of determining the expected value of the benefit of the interactive action corresponding to each state data includes: Each state data is input into a neural network, and the expected benefit value of the interactive action corresponding to each state data is calculated by the neural network, wherein each state data corresponds to at least one interactive action, and each state data has at least one expected benefit value.

4. The artificial intelligence-based anomaly detection method according to claim 3, characterized in that: When each state data corresponds to a plurality of interactive actions, the step of inputting each state data into a neural network and calculating the expected value of the benefit of each interactive action corresponding to the state data by the neural network includes: Input the current state into a pre-trained intention recognition model to obtain the next state intention of the enterprise; Based on the next state intention of the enterprise, determining actions that match the next state intention of the enterprise from the multiple interaction actions, classifying the actions into an intended action set, and classifying the remaining interaction actions into a prohibited action set; Each of the state data is input into a neural network, and the expected value of the benefit of each intended action in the intended action set corresponding to each of the state data is calculated by the neural network.

5. The artificial intelligence-based anomaly detection method according to claim 4, characterized in that: Before the step of obtaining enterprise training samples, the method further includes: Match all enterprise sample data with a blacklist, which includes enterprises in shady industries, shady areas, and enterprises with historical fraud risks; When the blacklist is matched successfully, the enterprise sample data that is matched successfully is eliminated, and the remaining enterprise sample data is used as enterprise training samples.

6. The artificial intelligence-based anomaly detection method according to claim 5, characterized in that: The step of training the reinforcement learning model based on each state data and the expected value of the benefit of the interaction action corresponding to each state data includes: Obtaining the maximum expected value of the next state data corresponding to each state data; According to the maximum expected benefit value, the preset decay value, the reward decay coefficient, and the return value corresponding to each state data, the expected benefit value of the interactive action corresponding to each state data is updated to obtain the final expected benefit value of the interactive action corresponding to each state data; The reinforcement learning model is trained based on each state data and the expected value of the final benefit of the interaction action corresponding to each state data.

7. An artificial intelligence-based anomaly detection device, characterized in that: The artificial intelligence-based anomaly detection device is used in the steps of the artificial intelligence-based anomaly detection method according to any one of claims 1 to 6, and the artificial intelligence-based anomaly detection device includes: An acquisition module, configured to acquire the current status of the enterprise to be detected and a set of optional actions corresponding to the current status; An input module, configured to input the current state into a preset reinforcement learning model for risk detection, and determine all predicted benefits corresponding to each optional action in the optional action set; A selection module, configured to select a maximum profit prediction value from all the profit prediction values; A comparison module, configured to compare the maximum profit prediction value with the actual profit value corresponding to the current state to obtain a comparison result; A determination module is configured to determine, based on the comparison result, that the enterprise to be detected is a risk enterprise, wherein the step of determining, based on the comparison result, that the enterprise to be detected is a risk enterprise comprises: If the maximum profit prediction value is less than the actual profit value corresponding to the current state, it is determined that the enterprise to be detected has a fraud risk; If the maximum profit prediction value is greater than the actual profit value corresponding to the current state, the process returns to the step of obtaining the current state of the enterprise to be detected and the optional action set corresponding to the current state.

8. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the artificial intelligence-based anomaly detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based anomaly detection method according to any one of claims 1 to 6.

Images