Estimation program, estimation method, and information processing device

Abstract

To provide an information processing device capable of efficiently preventing criminal damage caused by special fraud.SOLUTION: The information processing device acquires feature data representing the characteristics of people who use electronic devices. The information processing device estimates the emotions of a person by inputting the acquired feature data to a first machine learning model that is machine learned based on the feature data and emotional information about the emotions of the people. The information processing device estimates the risk of criminal activity concerning the person using an electronic device based on the estimated pattern of changes.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an estimation program, an estimation method, and an information processing device. [Background technology]

[0002] Since there have been cases of special frauds, such as "it's me" frauds and refund frauds, particularly targeting the elderly, local governments and police have been holding seminars to raise awareness and strengthen crackdowns. In recent years, a technology has become known that converts past telephone conversations involving special frauds into text and uses the conversations and keywords during the call to detect the occurrence of special frauds. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] NTT West Japan, [online], retrieved March 10, 2022, "NTT West Japan's Special Fraud Countermeasures," "URL: https: / / www.ntt-west.co.jp / info / support / special-fraud-support_service.html" Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above technology can only detect special frauds that have occurred in the past and cannot respond to new special frauds, so it is difficult to say that it can fully prevent the damage caused by special frauds.

[0005] In one aspect, an object of the present invention is to provide an estimation program, an estimation method, and an information processing device that can efficiently prevent crime damage caused by special fraud. [Means for solving the problem]

[0006] In the first proposal, the estimation program is characterized by causing a computer to execute the following process: acquire feature data indicating the characteristics of a person using an electronic device; estimate the emotions of the person by inputting the acquired feature data into a first machine learning model that has been machine-trained based on the feature data and emotional information regarding the person's emotions; and estimate the risk of a criminal act occurring against the person using the electronic device based on the pattern of changes in the estimated emotions of the person. [Effects of the Invention]

[0007] According to one embodiment, crime damage caused by special fraud can be efficiently prevented. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a system configuration according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the crime detection model. [Figure 3] FIG. 3 is a functional block diagram of the information processing apparatus according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating the first training data DB. [Figure 5] FIG. 5 is a diagram illustrating the second training data DB. [Figure 6] FIG. 6 is a diagram showing examples of the emotions of victims. [Figure 7] FIG. 7 shows emotions at the time of crime occurrence. [Figure 8] FIG. 8 is a diagram showing the relationship between refund fraud and emotion patterns. [Figure 9] FIG. 9 is a diagram showing the relationship between "it's me" fraud and emotion patterns. [Figure 10] FIG. 10 is a diagram illustrating machine learning of the emotion estimation model. [Figure 11] FIG. 11 is a diagram illustrating machine learning of a crime risk estimation model. [Figure 12] FIG. 12 is a diagram illustrating the estimation of crime risk. [Figure 13] FIG. 13 is a diagram illustrating an example of identifying an emotion pattern. [Figure 14] FIG. 14 is a flowchart showing the flow of the machine learning process for each model. [Figure 15] FIG. 15 is a flowchart showing the flow of the crime risk estimation process. [Figure 16] FIG. 16 is a diagram illustrating another example of machine learning of an emotion estimation model. [Figure 17] FIG. 17 is a diagram illustrating an example of exception determination based on keyword extraction. [Figure 18] FIG. 18 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an estimation program, an estimation method, and an information processing device disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. Furthermore, each embodiment can be appropriately combined within a consistent range. [Example]

[0010] (Overall composition) Figure 1 is a diagram illustrating the system configuration according to Example 1. As shown in Figure 1, this system is a public-private collaboration system that includes a special fraud prevention solution 1, a special fraud group 2, a user's home 3, and a local government 4. This system is an example of a special fraud prevention system that uses a crime detection model trained using criminal psychology and the like to detect special frauds in real time, prevent the occurrence of special frauds, and trains the evolving features of special frauds to provide effective guidance to local governments and the like.

[0011] Specialized fraud prevention solution 1 is a service provided by companies and other organizations that generates and provides a crime detection model that combines criminal psychology and machine learning. This crime detection model is trained on the emotional patterns of users when specialized fraud occurs, as identified by criminal psychology. For example, the crime detection model includes a first machine learning model that estimates a user's emotions from sensing data, which is an example of feature data that indicates the user's characteristics, and a second machine learning model that estimates the possibility of specialized fraud from the emotional patterns, and estimates the possibility of specialized fraud from the user's real-time emotions during a phone call.

[0012] Special fraud group 2 is a criminal that makes phone calls to users' homes 3 and commits crimes such as special fraud. For example, special fraud group 2 commits known special frauds including "it's me" fraud, deposit and savings fraud, cash card fraud, fictitious fee fraud, refund fraud, financial product fraud, gambling fraud, dating arrangement fraud, etc., as well as unknown special frauds including combinations of these frauds and new frauds.

[0013] The user's home 3 is the home of a user, such as an elderly person, and is the home of a user who is the target of fraud committed by the special fraud group 2. The user's home 3 is equipped with an information processing device 10 and non-contact sensing terminals including a camera for capturing video and facial images, a microphone for collecting audio, a millimeter-wave sensor for measuring heart rate and respiratory rate, and a wearable device worn on the arm or finger for measuring pulse waves, etc. The information processing device 10 also has a crime detection model implemented by the special fraud prevention solution 1, and is connected to the user's electronic device (e.g., a telephone) and each sensing terminal so as to be able to communicate with each other.

[0014] Local government 4 will implement various measures to prevent the victim from falling victim to special frauds by providing guidance and holding seminars for the elderly. The seminars given as examples here are not limited to seminars held by local public bodies such as local government 4, but also include services provided by private companies, seminars held by neighborhood associations, and seminars held at schools.

[0015] In such a system, the information processing device 10 acquires sensing data of a person using a telephone. The information processing device 10 estimates the person's emotions by inputting the acquired sensing data into a first machine learning model trained based on the sensing data. The information processing device 10 estimates the risk of a crime being committed against the person using the telephone based on the pattern of changes in the estimated person's emotions.

[0016] Here, we will explain the crime detection model used in Fig. 1. Fig. 2 is a diagram for explaining the crime detection model. As shown in Fig. 2, information processing device 10 executes a learning phase and an operation phase.

[0017] In the learning phase, the information processing device 10 generates a first machine learning model by machine learning using training data including the explanatory variable "sensing data" and the objective variable "person's emotion." The information processing device 10 also generates a second machine learning model by machine learning using training data including the explanatory variable "emotion pattern" and the objective variable "specialized fraud."

[0018] In the operation phase, the information processing device 10 estimates the risk of crimes, such as special frauds, each time it acquires sensing data for a predetermined period of time. Specifically, when the information processing device 10 acquires sensing data t0, it inputs the data into a first machine learning model and estimates "emotion A." When the information processing device 10 subsequently acquires sensing data t1, it inputs the data into the first machine learning model and estimates "emotion B." When the information processing device 10 subsequently acquires sensing data t2, it inputs the data into the first machine learning model and estimates "emotion B." When the information processing device 10 generates an emotion pattern "emotion A, emotion B, emotion A," it inputs this emotion pattern into a second machine learning model and estimates the risk of crimes, including various special frauds.

[0019] Returning to FIG. 1, when the information processing device 10 detects a special fraud in which the risk is equal to or greater than a threshold as a result of estimation, it notifies the user in real time of a message or the like that the current call is likely to be a special fraud.

[0020] In the special fraud prevention solution 1, the sensor data transferred from the user's home 3 is input into a crime detection model to estimate the risk of crime, including various special frauds. Here, when the special fraud prevention solution 1 detects a special fraud with a risk above a threshold, it notifies relevant parties linked to the user's home in real time that a crime may have occurred. As a result, the relevant parties can contact the police, etc., and the police can crack down on the special fraud group.

[0021] Furthermore, when the special fraud prevention solution 1 and the information processing device 10 accumulate sensing data and the estimation results of the crime detection model, they use them to analyze new criminal psychology, etc., and detect new frauds and new methods for dealing with known frauds. For example, in order to keep up with new frauds and new methods for dealing with known frauds, the special fraud prevention solution 1 re-learns the crime detection model using the new criminal psychology, etc., and delivers this information to the information processing device 10, while also providing this information to the local government 4, allowing for effective guidance.

[0022] As described above, the information processing device 10 can acquire sensing data of a person on the phone, input the data into an emotion estimation model to estimate the emotion during the phone call, and estimate the risk of special fraud from the pattern of changes in emotion during the phone call, thereby efficiently preventing crime damage from special fraud.

[0023] (Functional configuration) 3 is a functional block diagram illustrating a functional configuration of the information processing device 10 according to the first embodiment. As illustrated in FIG.

[0024] The communication unit 11 is a processing unit that controls communication with other devices, and is realized by, for example, a communication interface, etc. For example, the communication unit 11 receives sensing data from a sensing terminal and transmits alarms, messages, etc. to a user.

[0025] The storage unit 12 is an example of a processing unit that stores various data and programs executed by the control unit 20, and is realized by, for example, a memory or a hard disk. The storage unit 12 stores a first training data DB 13, a second training data DB 14, an emotion estimation model 15, a crime risk estimation model 16, and a sensing data DB 17.

[0026] The first training data DB13 is a database that stores training data used for machine learning of the emotion estimation model 15. FIG. 4 is a diagram illustrating the first training data DB13. As shown in FIG. 4, the training data stored in the first training data DB13 is data in which "sensing data," which serves as an explanatory variable during machine learning, is associated with "emotion," which serves as a target variable during machine learning. The "sensing data" stored here indicates sensing data that can be measured by a sensing terminal that can be installed in an ordinary home or a sensing terminal that can be worn by an ordinary user. The "emotion" indicates the emotion of a person at the time the sensing data was measured.

[0027] In the example of FIG. 4, the emotion measured using "image data S, voice data S1, and millimeter-wave data S2" is shown to be "emotion A." The image data S is data captured by a camera during a phone call, the voice data S1 is data collected by a microphone during a phone call, and the millimeter-wave data S2 is data acquired by a millimeter-wave sensor during a phone call. These pieces of sensed data may be the sensed data itself, or may be more detailed information. For example, tone of voice, facial pulse, etc. may also be used.

[0028] "Emotion A" refers to the facial expression of the user during the call, and can be a general indicator such as happy or sad, or one of the seven scales defined in the Profile of Mood States Second Edition (POM2): "anger-hostility," "confusion-bewilderment," "depression-disappointment," "fatigue-lethargy," "tension-anxiety," "liveliness-energy," and "friendship."

[0029] The second training data DB14 is a database that stores training data used in machine learning of the crime risk estimation model 16. FIG. 5 is a diagram illustrating the second training data DB14. As shown in FIG. 5, the training data stored in the second training data DB14 is data that associates "emotion patterns," which serve as explanatory variables during machine learning, with "specialized frauds," which serve as target variables during machine learning. The "emotion patterns" stored here indicate patterns of changes in a user's emotions during a phone call, and are patterns of changes in a person's emotions within a preset time period after a predetermined operation, such as pressing the answer button on the telephone, is detected. The "specialized frauds" are information that identifies the specialized frauds associated with the emotion pattern, and are the criminal acts that correspond to the emotion pattern among multiple criminal acts.

[0030] (Explanation of emotions) Here, the "emotions" used in Example 1 will be explained from the perspective of both the user (victim) who receives a call from the fraud group and the fraud group (perpetrator) who makes the call to the user. First, emotions will be explained in detail. Figure 6 is a diagram showing examples of the victim's emotions. Here, "friendship," "liveliness / vigor," and "confusion / perplexity" will be explained as examples. As shown in Figure 6, each emotion is identified based on the user's state, the user's tone and mood, and the content of the user's emotion.

[0031] For example, the emotional state of "friendship" is a state in which the user pays attention to the other person in conversation, such as introduction and trust. In this "friendship" state, the user's tone and atmosphere become "polite, cheerful, and sympathetic," and the user's emotional content is characterized by "a sense of trust and interest" in the other person in conversation.

[0032] The "confused-perplexed" emotional state is when the user becomes agitated and heart rate fluctuations occur in the conversation partner. In this "confused-perplexed" state, the user's tone and mood become "polite and calm," and the user's emotional content is characterized by "impatience and arousal of desire" in the conversation partner.

[0033] The emotional state of "lively-vital" is a state in which the user's conversation partner experiences an increased level of arousal, such as encouraging action. In this "lively-vital" state, the user's tone and mood become "polite and calm," and the user's emotional content is characterized by "taking action."

[0034] Next, we will explain the perpetrator's manipulation and the victim's emotions when a crime occurs. Figure 7 shows emotions at the time of a crime. As shown in Figure 7, in order to make the victim feel the above-mentioned "friendly" emotion, the perpetrator uses a tone and atmosphere that is "polite, cheerful, and sympathetic," and uses conversations that include keywords such as "This is the government office. Your money will be refunded. Have you received the documents?" with the aim of "gaining trust and making the victim interested." As a result, the victim develops unconditional trust in public institutions and hopes for a refund, which leads to feelings of "friendly."

[0035] In addition, in order to make the victim feel the above-mentioned "confusion and bewilderment," the perpetrator uses a "polite, flat" tone and atmosphere, and uses conversations that include keywords such as "the refund deadline has passed, special, just for you, bankbook and cash card, refund at the ATM" with the aim of "making the victim feel impatient and stimulating their desire." As a result, the victim becomes impatient, develops doubts, and is persuaded, resulting in feelings of "confusion and bewilderment."

[0036] In addition, the perpetrator will use a "polite, nonchalant" tone and atmosphere to make the victim feel the above-mentioned "liveliness and vitality," and will have conversations that include keywords such as "follow the instructions on the phone from the attendant" with the aim of getting the victim to "take their passbook and head to the ATM." As a result, the victim will feel uneasy about operating the ATM, will try to follow the instructions, and will experience feelings of "liveliness and vitality."

[0037] As mentioned above, perpetrators commit special frauds by skillfully manipulating the emotions of their victims for each type of special fraud using their rhetoric. In other words, the emotional changes of victims often differ depending on the type of special fraud. Therefore, the machine learning for the crime risk estimation model16 uses training data that associates "emotion patterns" with "special frauds" based on past history, analysis of criminal psychology, and surveys of perpetrators and victims.

[0038] (Explanation of emotional patterns) Here, as an example, we will explain the combination of special frauds and emotional patterns. Figure 8 is a diagram showing the relationship between refund frauds and emotional patterns. As shown in Figure 8, in refund frauds, the perpetrator often calls the user and starts by telling them something to pique their interest, such as "You'll get your money back," followed by something to make them feel anxious, such as "The refund deadline has passed," and finally ends with something to encourage them to take action, such as "As long as you pay the fee, everything will be fine." In this case, the user's emotions shift to a "friendly" state due to the interest-inducing story, to a "confused-perplexed" state due to the anxious story, and then to a "lively-energized" state due to the story to encourage them to take action. In other words, in "refund fraud," a change from positive to negative emotions occurs. As a result, the emotional patterns of "friendly, confused-perplexed, and lively-energized" are associated with "refund fraud."

[0039] Figure 9 illustrates the relationship between "it's me" scams and emotional patterns. As shown in Figure 9, in "it's me" scams, a first perpetrator, pretending to be a relative of the user, calls the user and begins by telling the user something that makes them feel anxious, such as "You lost company money." Then, a second perpetrator, acting as the user's boss, takes over and tells the user something like, "I'll cover the cost, so it's okay," to build trust. The first perpetrator then repeats the same anxious message, such as "Please transfer the money," and the second perpetrator often then begins encouraging the user to take action, such as, "I'll support you over the phone." In this case, the user's emotions shift from "confused to perplexed" due to the anxious message, to "friendly" due to the trustworthy message, to "confused to perplexed" again due to the anxious message, and then to "energetic to energized" due to the message that encourages action. In other words, "it's me" scams involve a shift from negative to positive emotions. As a result, the emotional patterns of "confused to perplexed, friendly, confused to perplexed, energized to energized" are associated with "it's me" scams.

[0040] Returning to FIG. 3, emotion estimation model 15 is an example of a first machine learning model included in the crime detection model that estimates a user's emotion. Specifically, emotion estimation model 15 is a machine learning model that outputs an emotion in response to input sensing data, and is a multi-valued decision model that outputs an estimated value (probability) corresponding to each emotion among multiple emotions. Note that various mathematical models such as neural networks can be used for emotion estimation model 15.

[0041] The crime risk estimation model 16 is an example of a second machine learning model included in the crime detection model that estimates the risk of crimes such as special fraud. Specifically, the crime risk estimation model 16 is a machine learning model that outputs a crime risk in response to an input of an emotion pattern that includes at least one emotion, and is a multi-valued judgment model that estimates the risk (probability) of each crime occurring among multiple crimes. Note that various mathematical models such as neural networks can be used for the crime risk estimation model 16.

[0042] The sensing data DB17 is a database that stores sensing data measured from users. Specifically, the sensing data DB17 stores sensing data for each user, for each date, or for each call. The data included in the sensing data can be arbitrarily set and changed by a sensing terminal installed in the user's home, and includes, for example, image data, voice data, millimeter wave data, etc.

[0043] The control unit 20 is a processing unit that controls the entire information processing device 10, and is realized by, for example, a processor. The control unit 20 has a machine learning unit 30, an operation unit 40, and a notification unit 50. The machine learning unit 30, the operation unit 40, and the notification unit 50 are realized by electronic circuits included in the processor, processes executed by the processor, etc.

[0044] The machine learning unit 30 has a first machine learning unit 31 and a second machine learning unit 32, and is a processing unit that generates an emotion estimation model 15 and a crime risk estimation model 16 prior to real-time detection of special fraud.

[0045] The first machine learning unit 31 is a processing unit that generates the emotion estimation model 15 by machine learning using training data stored in the first training data DB 13. FIG. 10 is a diagram illustrating machine learning of the emotion estimation model. As shown in FIG. 10, the first machine learning unit 31 inputs training data including "sensing data (image data S, audio data S1, millimeter wave data S2)" and "emotion A" to the emotion estimation model 15 and obtains an output result from the emotion estimation model 15. The first machine learning unit 31 then updates various parameters of the emotion estimation model 15 so as to reduce the error between the objective variable "emotion A" and the output result.

[0046] The second machine learning unit 32 is a processing unit that generates the crime risk estimation model 16 through machine learning using training data stored in the second training data DB 14. FIG. 11 is a diagram illustrating the machine learning of the crime risk estimation model. As shown in FIG. 11, the second machine learning unit 32 inputs training data including an "emotion pattern (emotion A → emotion B → emotion A)" and "crime AA" into the crime risk estimation model 16 and obtains an output result from the crime risk estimation model 16. The second machine learning unit 32 then updates various parameters of the crime risk estimation model 16 so as to reduce the error between the objective variable "crime AA" and the output result.

[0047] The operation unit 40 has an acquisition unit 41, a first estimation unit 42, and a second estimation unit 43, and is a processing unit that detects the crime risk of special fraud from the content of a user's phone call using the emotion estimation model 15 and the crime risk estimation model 16 generated by the machine learning unit 30. In other words, the operation unit 40 estimates the occurrence risk of known special frauds and unknown special frauds in real time.

[0048] The acquisition unit 41 is a processing unit that acquires sensing data of a user. Specifically, the acquisition unit 41 acquires sensing data sensed by various sensing terminals installed in the user's home 3 and stores the data in the sensing data DB 17. For example, when the acquisition unit 41 detects an operation on the telephone, such as a ring tone in the case of a landline telephone or an operation to start a call via wireless communication in the case of a mobile telephone, the acquisition unit 41 acquires video data captured by a camera attached to the telephone. Then, the acquisition unit 41 extracts facial image data of the person who operated the telephone as sensing data by applying known image analysis to the acquired video data.

[0049] The first estimation unit 42 is a processing unit that estimates the user's emotion by inputting the sensing data acquired by the acquisition unit 41 to the emotion estimation model 15. For example, when the first estimation unit 42 acquires sensing data including facial image data, it inputs the data to the emotion estimation model 15 to acquire an emotion estimation result. Then, the first estimation unit 42 identifies, from among the emotion estimation results, an emotion with an estimation probability equal to or higher than a threshold or an emotion with the highest estimation probability as the estimation result. The first estimation unit 42 stores the estimation results in the storage unit 12 or the like in chronological order.

[0050] The second estimation unit 43 is a processing unit that estimates the risk of special fraud occurring by using the emotions estimated by the first estimation unit 42 and the crime risk estimation model 16. For example, the second estimation unit 43 combines the estimated emotions in a time series to generate an emotion pattern, inputs the emotion pattern into the crime risk estimation model 16, and obtains an estimation result of the risk of special fraud occurring.

[0051] For example, the second estimation unit 43 estimates that a special fraud call is being made to a person when the person's emotions estimated by the first estimation unit 42 meet the condition that they transition from a negative state to a positive state within a specified period of time.

[0052] Furthermore, the second estimation unit 43 inputs the estimated emotion pattern into the crime risk estimation model 16 to obtain the magnitude of risk of occurrence of each of a plurality of criminal acts, and identifies a criminal act against a person from among the plurality of criminal acts based on the magnitude of risk of occurrence. For example, the second estimation unit 43 identifies, as the estimation result, a crime with an estimated probability equal to or greater than a threshold or a crime with the highest estimated probability in the estimation results of the risk of occurrence obtained from the crime risk estimation model 16. When a crime is detected, the second estimation unit 43 outputs information about the estimated crime to the notification unit 50.

[0053] Here, the estimation of crime risk will be explained. FIG. 12 is a diagram for explaining the estimation of crime risk. As shown in FIG. 12, the operation unit 40 acquires external and internal information of a user who is talking on the phone as sensing data through an appearance approach using a sensing terminal. The operation unit 40 inputs the acquired sensing data into the emotion estimation model 15 and estimates the person's emotions as needed.

[0054] The management unit 40 then generates emotion patterns using information about each emotion that is sequentially estimated from the start of the conversation. For example, the management unit 40 generates emotion patterns of "lively-vitality," "confused-bewildered," and "lively-vitality" in chronological order of the conversation time, inputs these into the crime risk estimation model 16, and obtains an estimation result such as "special fraud estimation probability: 0%."

[0055] The emotion pattern is generated by using emotion information estimated within a predetermined time period, such as a five-minute interval. FIG. 13 is a diagram illustrating an example of identifying an emotion pattern. As shown in FIG. 13, when a call starts at time T0, the operations unit 40 executes emotion estimation as needed to estimate emotion A at T1, emotion B at T2, emotion A at T3, emotion C at T4, and emotion B at T5.

[0056] In this case, the management unit 40 generates emotion pattern 1 including emotion A, emotion B, and emotion A using emotions B and C estimated within a specified time period starting from T1, when the first emotion (emotion A) was estimated, and performs a crime risk estimation. Next, the management unit 40 generates emotion pattern 2 including emotions B, emotion A, and emotion C using emotions A and C estimated within a specified time period starting from T2, when emotion (emotion B) was estimated after T1, and performs a crime risk estimation. Next, the management unit 40 generates emotion pattern 3 including emotions A, emotion C, and emotion B using emotions C and B estimated within a specified time period starting from T3, when emotion (emotion A) was estimated after T2, and performs a crime risk estimation.

[0057] In this way, the operations unit 40 generates emotion patterns using emotions within a predetermined interval while the telephone conversation is ongoing, and repeatedly estimates crime risk. Note that the emotion pattern generation interval shown in Figure 12 is an example, and can be set or changed as desired. For example, an emotion pattern can be generated using T, T2, and T3, and then an emotion pattern can be generated using T4, T5, and T6.

[0058] Returning to FIG. 3 , the notification unit 50 is a processing unit that notifies relevant parties when the operation unit 40 estimates a crime risk. For example, when the notification unit 50 estimates a "risk of crime occurrence" with an estimated probability equal to or greater than a threshold, it sends a message such as "The caller may be a fraud group" to a speaker installed in the user's home 3 or vibrates the user's mobile phone or wearable device to alert the user. The notification unit 50 can also send an emergency message such as "Mr. / Ms. X is on the phone and there is a possibility of a special fraud attempt" to a pre-registered emergency contact. The notification unit 50 can also notify the police or local government.

[0059] (Machine learning process flow) 14 is a flowchart showing the flow of machine learning processing for each model. Note that both the emotion estimation model 15 and the crime risk estimation model 16 can be generated using the same processing flow.

[0060] As shown in FIG. 14, when the machine learning unit 30 is instructed to start processing (S101: Yes), it acquires training data from each DB (S102) and inputs it into the corresponding machine learning model (S103).

[0061] Then, the machine learning unit 30 acquires the output result of the machine learning model (S104), calculates the error between the objective variable (correct answer information) of the training data and the output result (S105), and updates the parameters of the machine learning model to minimize the error (S106).

[0062] Here, if an end condition is reached, such as machine learning using a predetermined number of training data or completion of a predetermined number of epochs (S107: Yes), the machine learning unit 30 ends the machine learning. On the other hand, if the machine learning unit 30 decides to continue the machine learning (S107: No), it repeats S102 and subsequent steps.

[0063] (Flow of estimation process) Fig. 15 is a flowchart showing the flow of the crime risk estimation process. As shown in Fig. 15, when the operation unit 40 acquires sensing data (S201: Yes), it inputs the sensing data into the emotion estimation model 15 and estimates the user's emotion (S202). Here, the operation unit 40 repeats S201 and subsequent steps before a predetermined time has elapsed since the start of the call (S203: No).

[0064] On the other hand, when a predetermined time has elapsed since the start of the call (S203: Yes), the operation unit 40 generates an emotion pattern using the estimated emotion (S204). Subsequently, the operation unit 40 inputs the emotion pattern into the crime risk estimation model 16 to estimate the crime risk (S205).

[0065] Then, if the operation unit 40 estimates a crime risk (S206: Yes), the notification unit 50 notifies the relevant parties (S207). Thereafter, if the operation unit 40 continues processing (S208: No), S201 and subsequent steps are executed, and if the operation unit 40 ends processing (S208: Yes), the estimation of the crime risk is terminated. Note that if a crime risk of the notification target is not estimated in S206 (S206: No), S208 is executed without executing S207.

[0066] (effect) As described above, the information processing device 10 estimates a person's emotions by inputting sensing data into the emotion estimation model 15, and estimates the risk of a crime being committed against the person based on an emotion pattern, which is a change in the estimated person's emotions. Therefore, the information processing device 10 can automate and visualize the observation process of sensing data, etc., and generate a highly accurate detection model.

[0067] The information processing device 10 can infer crimes from a person's emotional patterns, thereby preventing crime damage caused by known and unknown specialized frauds. Furthermore, the information processing device 10 can generate a machine learning model that performs accurate inference using accurate training data, thereby realizing faster inference processing compared to when unnecessary training data is used.

[0068] The information processing device 10 estimates the risk of a crime being committed against a person using a crime risk estimation model 16 that has been trained on the relationship between emotion patterns and crimes. Therefore, even if a new crime occurs, the information processing device 10 can estimate the risk of the crime occurring from the emotion patterns of the person related to the crime, and therefore can effectively prevent unknown special fraud crimes.

[0069] The information processing device 10 generates emotion patterns obtained from past records and criminal psychology for each special crime, and trains the crime risk estimation model 16. Therefore, while training on past records, the information processing device 10 can also train emotion patterns at the time of unknown crimes that are inferred from past records. [Example]

[0070] In the first embodiment, the emotion estimation model 15 is described as being based on machine learning that uses sensing data from an appearance approach as explanatory variables, but the present invention is not limited to this. For example, the information processing device 10 can also improve the accuracy of the emotion estimation model 15 by machine learning that uses indices from an internal approach as explanatory variables.

[0071] Fig. 16 is a diagram illustrating another example of machine learning of the emotion estimation model 15. As shown in Fig. 16, the machine learning unit 30 of the information processing device 10 executes machine learning of the emotion estimation model 15 using training data in which, in addition to sensing data from an appearance approach, psychological scales generated from an internal approach are used as explanatory variables and emotions are used as objective variables.

[0072] This psychological scale is an index that quantifies abstract concepts such as human psychology, consciousness, and behavioral tendencies. For example, users in a special crime experiment or users who have actually received special crime phone calls are asked to select the appropriate score for multiple items such as "very angry" or "troubled," and the results are converted into a score. The machine learning unit 30 uses the scale, which calculates the total score, average score, variance, etc. for each special fraud, as explanatory variables.

[0073] As a result, the machine learning unit 30 performs machine learning on the "emotions" of users resulting from each special fraud using training data that associates sensing data from an external approach with psychological scales from an internal approach. For example, the machine learning unit 30 inputs training data including explanatory variables "sensing data (image data S, audio data S1, millimeter-wave data S2), psychological scale Z" and an objective variable "emotion A" into the emotion estimation model 15 and obtains an output result from the emotion estimation model 15. The machine learning unit 30 then updates various parameters of the emotion estimation model 15 so as to reduce the error between the objective variable "emotion A" and the output result. During operation, estimation is performed using sensing data from an external approach as input.

[0074] As a result, even if training based solely on the external approach is biased or sensing data based on the external approach is biased, the emotion estimation model 15 can be generated while correcting the bias using the internal approach, thereby improving the accuracy of emotion estimation.

[0075] In addition, in the first embodiment, an example using sensing data obtained by a non-contact sensing terminal has been described, but this has an aspect that it is effective as data that can be sensed at any time while a user is on the phone at the user's home 3, regardless of the location or time of the call. As one method for ensuring the validity of this non-contact sensing data, a contact sensor such as an electrodermal sensor or an electrocardiogram sensor can be used.

[0076] For example, the information processing device 10 can generate training data by converting the amount of sweat obtained from skin conductivity, an electrocardiogram obtained from electrocardiograms, etc. into non-contact sensing data using a known method. As a result, the information processing device 10 can improve the reliability of the training data.

[0077] The information processing device 10 can also use both non-contact sensing data and contact sensing data as training data. In this case, either the non-contact sensing data or the contact sensing data is used during estimation. As a result, more accurate crime risk estimation can be achieved in a user's home where non-contact sensing data can be obtained. Furthermore, crime risk estimation can be achieved without depending on the environment of the user's home. [Example]

[0078] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments.

[0079] (Exception handling) For example, the information processing device 10 can estimate the risk of crime by focusing on emotional patterns, which are changes in emotions. However, sad messages, such as condolences, may produce emotional patterns similar to those of specific crimes. In such cases, if condolences cannot be distinguished from specific crimes, unnecessary alarms will be generated, leading to a deterioration in the accuracy of the machine learning model.

[0080] Therefore, when the information processing device 10 detects a pre-registered keyword during a telephone conversation, it determines that the event is an "exceptional event" such as a "sympathy message" and suppresses the possibility of a specific crime. Fig. 17 is a diagram illustrating an example of exception determination by keyword extraction.

[0081] As shown in FIG. 17, when a call starts at time T0, the operations unit 40 performs emotion inference at any time, thereby inferring emotion A at T1, emotion B at T2, emotion A at T3, emotion C at T4, and emotion B at T5.

[0082] On the other hand, if the operations unit 40 detects a conversation between times T3 and T4 that includes the registered keyword "My condolences" in the keyword list, it determines that the content of the call is an "exceptional event." As a result, the operations unit 40 suppresses the inference of a special crime. Therefore, the information processing device 10 can distinguish between an exceptional event and a special fraud, thereby suppressing unnecessary alarms and improving the reliability of the service.

[0083] (Numbers, etc.) The example data sets, example training data, example sensors, number of data, time span, example and number of emotions, number and types of special frauds, emotion patterns, combinations of special frauds and emotion patterns, etc. used in the above examples are merely examples and can be changed as desired. Furthermore, the process flow described in each flowchart can also be changed as appropriate within a consistent range. Note that sensing data is an example of feature data.

[0084] (Examples of presumed special fraud) In the above embodiment, an example has been described in which the crime risk estimation model 16 is used to estimate the crime risk of special fraud from emotion patterns, but the present invention is not limited to this. For example, the information processing device 10 can hold a database that associates emotion patterns with special frauds, and use the database to estimate the special frauds that correspond to the identified emotion patterns.

[0085] (Example of electronic equipment) In the above embodiment, an example has been described in which a call is being made to the user's home 3, but the present invention is not limited to this. For example, the information processing device 10 can provide a similar service to a user who uses an ATM (Automatic Teller Machine) installed in a store such as a bank or convenience store. In this case, the information processing device 10 acquires sensing data from a camera or microphone installed around the ATM to estimate a crime risk, and if a crime risk is detected, makes an announcement in the store or notifies a responsible person.

[0086] (system) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0087] Furthermore, the components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. For example, the machine learning unit 30 and the operation unit 40 can be realized as separate computers (housings). In other words, they can be realized as an information processing device that performs the same functions as the machine learning unit 30 and an information processing device that performs the same functions as the operation unit 40.

[0088] Furthermore, all or any part of the processing functions performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0089] (Hardware) Fig. 18 is a diagram illustrating an example of a hardware configuration. As shown in Fig. 18, an information processing device 10 includes a communication device 10a, an HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. The components shown in Fig. 18 are connected to each other via a bus or the like.

[0090] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and DBs that operate the functions shown in FIG.

[0091] The processor 10d reads out from the HDD 10b or the like a program that executes the same processes as the respective processing units shown in Fig. 3 and loads it into the memory 10c, thereby operating a process that executes each function described in Fig. 3 or the like. For example, this process executes the same functions as the respective processing units of the information processing device 10. Specifically, the processor 10d reads out from the HDD 10b or the like a program that has the same functions as the machine learning unit 30, the operation unit 40, the notification unit 50, and the like. Then, the processor 10d executes a process that executes the same processes as the machine learning unit 30, the operation unit 40, the notification unit 50, and the like.

[0092] In this way, the information processing device 10 operates as an information processing device that executes an information processing method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the above-described embodiment may also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0093] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer. [Explanation of symbols]

[0094] 10. Information processing equipment 11 Communications Department 12 Storage section 13 First training data DB 14 Second training data DB 15 Emotion estimation model 16 Crime Risk Estimation Model 17 Sensing Data DB 20 Control Unit 30 Machine Learning Department 31 Machine Learning Department 1 32 Second Machine Learning Department 40 Operations Department 41 Acquisition Department 42 1st estimation part 43 Second estimation part 50 Information Department

Claims

1. Acquire feature data that indicates the features of the person using the electronic device; estimating the emotion of the person by inputting the acquired feature data into a first machine learning model that has been trained based on the feature data and emotion information related to the emotion of the person; estimating a risk of a criminal act occurring against the person who uses the electronic device based on the estimated pattern of changes in the person's emotions; Have the computer execute the process, The process of estimating the risk includes: obtaining a second machine learning model that has been machine-trained using correct answer data in which the pattern of change in the person's emotions is used as an explanatory variable and the criminal act corresponding to the pattern of change among a plurality of criminal acts is used as a target variable; acquiring a magnitude of risk of occurrence of each of the plurality of criminal acts by inputting the estimated pattern of change in the person's emotions into the acquired second machine learning model; An estimation program characterized by identifying a criminal act against the person who uses the electronic device from among the plurality of criminal acts based on the acquired magnitude of the risk of occurrence.

2. the pattern of change in the person's emotion is a pattern of change in the person's emotion within a preset time period after a predetermined operation on the electronic device is detected; 2. The estimation program according to claim 1, wherein:

3. Acquiring an electrodermal signal or an electrocardiogram signal of a person from a sensor attached to the person; generating characteristic data of the person from the electrodermal activity or the electrocardiogram; using correct answer data in which the feature data of the person is an explanatory variable and the emotional information of the person is a target variable, machine learning is performed on the first machine learning model so that an error between an output result when the correct answer data is input to the first machine learning model and the emotional information of the person is minimized; 2. The estimation program according to claim 1, wherein the program causes the computer to execute processing.

4. The process of performing machine learning includes: generating the first machine learning model so that an error between an output result when the correct answer data, which has as the explanatory variables the characteristic data of the person and a psychological scale that scores the mental state, consciousness, or behavioral tendency of a person at the time of the criminal act experiment or who has been subjected to the criminal act, and the emotional information of the person as the objective variable, is input into the first machine learning model, and the emotional information of the person is minimized; The estimating process includes: inputting the characteristic data and the psychological scale of the person using the electronic device into the first machine learning model to estimate the emotion of the person; The estimating process includes: Estimating the risk of the criminal act occurring based on the pattern of changes in the person's emotions; 4. The estimation program according to claim 3.

5. The criminal offense against the person is of multiple types, 5. The estimation program according to claim 4, wherein the index of the person's psychology is set for each of the plurality of types.

6. The acquiring process includes: Detects whether the phone is being operated or not, When the telephone is operated, facial image data of the person who operated the telephone is extracted from video data captured by a camera attached to the telephone; The estimating process includes: inputting the extracted facial image data of the person as the feature data into the first machine learning model to estimate the emotion of the person; The estimating process includes: When the estimated emotion of the person satisfies a condition that the emotion transitions from a negative state to a positive state within a predetermined period, it is estimated that a special fraud call is being made to the person.

2. The estimation program according to claim 1, wherein:

7. Acquire feature data that indicates the features of the person using the electronic device; estimating the emotion of the person by inputting the acquired feature data into a first machine learning model that has been trained based on the feature data and emotion information related to the emotion of the person; estimating a risk of a criminal act occurring against the person who uses the electronic device based on the estimated pattern of changes in the person's emotions; The computer executes the processing, The process of estimating the risk includes: obtaining a second machine learning model that has been machine-trained using correct answer data in which the pattern of change in the person's emotions is used as an explanatory variable and the criminal act corresponding to the pattern of change among a plurality of criminal acts is used as a target variable; acquiring a magnitude of risk of occurrence of each of the plurality of criminal acts by inputting the estimated pattern of change in the person's emotions into the acquired second machine learning model; An estimation method comprising: identifying a criminal act against the person who uses the electronic device from among the plurality of criminal acts based on the acquired magnitude of the risk of occurrence.

8. Acquire feature data that indicates the features of the person using the electronic device; estimating the emotion of the person by inputting the acquired feature data into a first machine learning model that has been trained based on the feature data and emotion information related to the emotion of the person; estimating a risk of a criminal act occurring against the person who uses the electronic device based on the estimated pattern of changes in the person's emotions; A control unit is provided. The control unit obtaining a second machine learning model that has been machine-trained using correct answer data in which the pattern of change in the person's emotions is used as an explanatory variable and the criminal act corresponding to the pattern of change among a plurality of criminal acts is used as a target variable; acquiring a magnitude of risk of occurrence of each of the plurality of criminal acts by inputting the estimated pattern of change in the person's emotions into the acquired second machine learning model; An information processing device characterized in that it identifies a criminal act against the person who uses the electronic device from among the plurality of criminal acts based on the acquired magnitude of the risk of occurrence.

Images