Information processing program, information processing method, and information processing device

Abstract

To highly accurately evaluate the occurrence state of criminal act on a target person.SOLUTION: An information processing apparatus 100 identifies, on the basis of speech data 101 indicating the conversation of a subject person in a subject period, a section in the subject period that matches each of one or more speech patterns. The information processing apparatus 100 acquires, for each of the identified sections, the biological data 102 of the subject person in the section. The information processing apparatus 100 implements, for each of the identified sections, a noise rejection process on the biological data 102 of the subject person in the acquired section, which corresponds to either of one or more speech patterns that matched the section. The information processing apparatus 100 evaluates the occurrence state of criminal act on the subject person, on the basis of the biological pattern 102 of the subject person after having implemented, in each of the identified sections, a noise rejection process that corresponds to either of the speech patterns having matched the section.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

Background Art

[0002] Conventionally, fraud may occur via a telephone. For example, there may be a fraud caused by a fraud act called an "oreore fraud," in which a fraudster deceives a relative of a targeted person via a telephone and tries to deceive money from the person. In contrast, a technique for analyzing voice data during a call of a target person and detecting the occurrence of a fraud act against the target person can be considered.

[0003] As a prior art, for example, based on biometric data of a caller, there is one that determines the degree of stress of the caller, which indicates the degree to which the mental state of the caller, such as the degree of tension of the caller, is different from the normal mental state. <s

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] <s However, the prior art has a problem that it is difficult to accurately detect the occurrence of a criminal act against a target person. For example, when a target person becomes a target of a fraud act via a telephone, the speaking time or the number of speaking times of the target person tends to decrease, and even if the voice data during the call of the target person is analyzed, it is difficult to accurately detect the occurrence of a fraud act against the target person.

[0006] In one aspect, an object of the present invention is to accurately evaluate the occurrence situation of a criminal act against a target person. [Means for solving the problem]

[0007] According to one embodiment, an information processing program, information processing method, and information processing device are proposed that acquire audio data showing the conversation of a target person over a certain period, identify sections within the period that match each of one or more set audio patterns based on the acquired audio data, perform noise reduction processing on the biometric data of the target person in each identified section, corresponding to one of the one or more audio patterns that matches the section, and evaluate the occurrence of criminal acts against the target person based on the biometric data of the target person after performing noise reduction processing corresponding to one of the audio patterns that matches the section in each identified section. [Effects of the Invention]

[0008] According to one embodiment, it becomes possible to accurately assess the circumstances under which criminal acts occur against a target individual. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. [Figure 2] Figure 2 is an explanatory diagram showing an example of the information processing system 200. [Figure 3] Figure 3 is an explanatory diagram showing the first application example of the information processing system 200. [Figure 4] Figure 4 is an explanatory diagram showing a second application example of the information processing system 200. [Figure 5] Figure 5 is a block diagram showing an example of the hardware configuration of the information processing device 100. [Figure 6] Figure 6 is an explanatory diagram showing an example of the contents stored in the filter information management table 600. [Figure 7] Figure 7 is an explanatory diagram showing an example of the contents of the emotion pattern information management table 700. [Figure 8] Figure 8 is an explanatory diagram showing an example of the contents stored in the bio-pattern information management table 800. [Figure 9] Figure 9 is a block diagram showing an example of the hardware configuration of the audio data acquisition device 201. [Figure 10] Figure 10 is a block diagram showing an example of the functional configuration of the information processing device 100. [Figure 11] Figure 11 is an explanatory diagram (part 1) showing an example of the operation of the information processing device 100. [Figure 12] Figure 12 is an explanatory diagram (part 2) showing an example of the operation of the information processing device 100. [Figure 13] Figure 13 is an explanatory diagram (part 3) showing an example of the operation of the information processing device 100. [Figure 14] Figure 14 is a flowchart showing an example of the overall processing procedure. [Modes for carrying out the invention]

[0010] Embodiments of the information processing program, information processing method, and information processing apparatus according to the present invention will be described in detail below with reference to the drawings.

[0011] (An embodiment of the information processing method according to the embodiment) Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. The information processing device 100 is a computer for evaluating the occurrence of criminal acts.

[0012] The information processing device 100 may be, for example, a server or a PC (Personal Computer). Alternatively, the information processing device 100 may be, for example, a smartphone or a smart speaker.

[0013] The criminal act is, for example, a fraud act called special fraud. Specifically, the criminal act is a fraud act through telephone. For example, there is a fraud act called ole ole fraud where a fraudster tries to deceive a target person's relative through telephone and swindle money from the target person.

[0014] Also, for example, there is a fraud act called refund fraud where a fraudster tries to deceive a staff member of a local government or a tax office, etc., suggests to the target person to receive an overpayment for medical expenses or insurance premiums, etc., and swindle money from the target person. Also, for example, there is a fraud act called theater fraud where multiple fraudsters try to deceive a target person's relative, a police officer, a lawyer, etc., respectively, and swindle money from the target person.

[0015] Therefore, a technology for preventing fraud damage caused by fraud acts is desired. For example, there is a first technology that tries to detect the occurrence of a fraud act against a target person and prevent fraud damage caused by the fraud act by analyzing voice data during a call of the target person and evaluating the occurrence situation of the fraud act. Specifically, it is conceivable to analyze the voice data during a call of the target person during the call period from the start of the call to the end of the call and evaluate the occurrence situation of the fraud act.

[0016] However, this first technology has a problem that it is difficult to accurately detect the occurrence of a fraud act against a target person. For example, when the target person becomes a target of a fraud act through telephone, the speaking time or the number of speaking times of the target person tends to decrease. Therefore, even if the voice data during a call of the target person is analyzed, it is difficult to accurately detect the occurrence of a fraud act against the target person. For example, in the voice data during a call of the target person, partial data regarding a section with a relatively short speaking time, such as a section where the target person makes a response, may be noise when evaluating the occurrence situation of a fraud act against the target person.

[0017] Another possible technique involves analyzing the biometric data of a target individual during a phone call to assess the likelihood of fraudulent activity and prevent fraudulent damage. Specifically, this could involve analyzing the biometric data of the target individual during the entire phone call, from start to finish, to assess the likelihood of fraudulent activity.

[0018] Even with this second technology, accurately detecting fraudulent activity against a target individual remains difficult. For example, because the biometric data from a target individual's phone calls may contain noise, analyzing this data may not accurately detect fraudulent activity against that individual.

[0019] Specifically, due to the influence of the respiratory cycle, the biometric data of a person during a phone call may contain noise of varying magnitude rather than a constant size. Furthermore, specifically, the noise in the biometric data of a person during a phone call tends to increase due to the person's body movements while they are speaking. Moreover, specifically, depending on the flow of the conversation, the person may perform various actions or the magnitude of their body movements may change, resulting in the inclusion of noise of varying magnitude in the biometric data of a person during a phone call. For this reason, even if the biometric data of a person during a phone call is analyzed, it is difficult to accurately detect the occurrence of fraudulent activity against that person. For example, even if one attempts to remove noise from the biometric data of a person during a phone call using a uniform standard, it may not be possible to remove the noise appropriately.

[0020] Therefore, this embodiment describes an information processing method that can accurately evaluate the occurrence of criminal acts against a target person.

[0021] In Figure 1, (1-1) the information processing device 100 acquires audio data 101 that shows the conversation of the target person during the target period. The target person is a person who is to be protected from criminal activity. Criminal activity is, for example, fraud. The audio data 101 shows a conversation between the target person and another person via telephone. The other person may be, for example, a criminal. A criminal is, for example, a con artist. The target period is, for example, the period during which the target person is having a conversation. Specifically, the target period is the duration of the call from the start to the end of the call.

[0022] The information processing device 100 may, for example, have a microphone and acquire audio data 101 by recording the conversation of a target person using the microphone. The information processing device 100 may also acquire the audio data 101 by receiving it from another computer. The other computer may be, for example, a smartphone or a smart speaker.

[0023] Specifically, the information processing device 100 continuously records the target person's conversation using a microphone while the target person is on a call, and acquires audio data 101 representing the target person's conversation during the period from the start to the end of the call. Specifically, the information processing device 100 may continuously record the target person's conversation using a microphone while the target person is on a call, and acquire audio data 101 representing the target person's conversation during the period from the current time to a certain time prior at predetermined timings during the target person's call. The predetermined timings are, for example, at regular intervals.

[0024] (1-2) The information processing device 100 stores one or more speech patterns. A speech pattern represents, for example, the characteristics of conversational speech indicated by speech data corresponding to a certain speech attribute. Speech attributes include, for example, "silence" indicating that the person in question was silent, and "acknowledgment" indicating that the person in question nodded in agreement. Speech attributes may also include, for example, "listening" indicating that the person in question heard what another person was saying. Speech attributes may also include, for example, "utterance" indicating that the person in question spoke.

[0025] A voice pattern represents a reference pattern that, by matching it with, for example, the speech or utterance content of a certain section, makes it possible to identify the speech attribute corresponding to that section. Specifically, a voice pattern represents a keyword corresponding to a certain speech attribute. Specifically, a voice pattern may represent the tone, volume, or wavelength of a voice corresponding to a certain speech attribute.

[0026] Specifically, the information processing device 100 stores voice patterns corresponding to "silence" and voice patterns corresponding to "acknowledgments." Specifically, the information processing device 100 may also store voice patterns corresponding to "speech." The voice patterns corresponding to "speech" may be characterized as not matching the voice patterns corresponding to "silence" and the voice patterns corresponding to "acknowledgments."

[0027] The information processing device 100 identifies sections within the target period that match one or more voice patterns based on the acquired voice data 101. For example, based on the voice data 101, the information processing device 100 classifies and identifies sections within the target period in which a voice pattern corresponding to "silence" appears as a silent section. For example, based on the voice data 101, the information processing device 100 classifies and identifies sections within the target period in which a voice pattern corresponding to "acknowledgment" appears as an acknowledgment section. Silent sections and acknowledgment sections are part of the listening section.

[0028] The information processing device 100, for example, classifies and identifies the sections within the target period in which speech patterns corresponding to "utterances" appear, based on the speech data 101, into utterance sections. Specifically, the information processing device 100 may classify and identify the remaining sections within the target period, excluding silence sections and interjection sections, into utterance sections, based on the speech data 101.

[0029] (1-3) For each of the identified intervals, the information processing device 100 acquires the biometric data 102 of the target person in that interval. For example, the information processing device 100 acquires the biometric data 102 of the target person for silence intervals, nodding intervals, and speaking intervals.

[0030] Specifically, the information processing device 100 has a biosensor and acquires biometric data 102 by measuring characteristic values ​​such as heart rate, pulse rate, body temperature, or sweating of a target person using the biosensor. Specifically, the information processing device 100 may acquire the biometric data 102 by receiving it from another computer. The other computer may be, for example, a smartphone or a wearable device.

[0031] More specifically, the information processing device 100 continuously measures the characteristic values ​​of the target person using a biosensor during the target person's phone call, and acquires overall biometric data showing the time change of the target person's characteristic values ​​from the start to the end of the phone call. More specifically, the information processing device 100 acquires the target person's biometric data 102 by extracting the target person's biometric data 102 from the acquired overall biometric data during silence, interjection, and speaking intervals.

[0032] More specifically, the information processing device 100 continuously measures the characteristic values ​​of the target person using a biosensor while the target person is on a call. More specifically, the information processing device 100 may acquire overall biometric data showing the time change of the target person's characteristic values ​​from the present to a certain time prior, at predetermined timings during the target person's call. The predetermined timings are, for example, at regular intervals. More specifically, the information processing device 100 acquires the target person's biometric data 102 by extracting the target person's biometric data 102 from the acquired overall biometric data during silence intervals, acknowledgment intervals, and speaking intervals.

[0033] (1-4) For each identified section, the information processing device 100 performs noise reduction processing on the acquired biometric data 102 of the target person in that section, corresponding to one or more voice patterns that match the section. For example, the information processing device 100 removes noise of magnitude and type corresponding to the speech tendency or behavioral tendency that corresponds to the voice pattern that matches the identified section from the biometric data 102 of the target person in that section.

[0034] Action tendencies indicate, for example, that a person's actions when they are silent are relatively small. Action tendencies indicate, for example, that a person's actions when they nod in agreement tend to be larger than when they are silent. Action tendencies indicate, for example, that a person's actions when they speak are relatively large, and that a person's actions when they speak tend to be larger than when they nod in agreement.

[0035] (1-5) The information processing device 100 evaluates the occurrence of criminal acts against the target person based on the biometric data 102 of the target person after noise reduction processing has been performed on the target person in each identified section, corresponding to any voice pattern that matches that section.

[0036] The information processing device 100 generates feature quantities related to a target person based on the biometric data 102 of the target person after performing noise reduction processing on each identified section, corresponding to any of the voice patterns that match that section.

[0037] Features are, for example, biological features. Specifically, features are one or more biological feature values, statistical values ​​of biological feature values, or changes in biological feature values. Biological feature values ​​are, for example, feature values ​​related to heart rate, pulse rate, body temperature, or sweating. Specifically, a biological feature value is heart rate. Statistical values ​​are, for example, mean, median, mode, maximum, or minimum. As a result, the information processing device 100 can obtain features that serve as indicator values ​​for evaluating the occurrence of criminal acts against the target person.

[0038] The information processing device 100 evaluates, for example, the occurrence of criminal acts against a target person based on the generated features. Specifically, for each specified interval, the information processing device 100 compares the generated features with a threshold if the generated features are changes in biological feature values. The threshold is set in advance by the user, for example. Specifically, the information processing device 100 evaluates that a criminal act against the target person has occurred if the generated features are greater than or equal to the threshold. Specifically, the information processing device 100 evaluates that a criminal act against the target person has not occurred if the generated features are less than the threshold.

[0039] Specifically, the information processing device 100 may evaluate the occurrence of criminal acts against the target person based on the pattern of change in the feature quantities within each specified interval, provided that the generated feature quantities are one or more biological feature values. Specifically, the information processing device 100 may evaluate the occurrence of criminal acts against the target person based on the pattern of change in the feature quantities spanning multiple intervals, provided that the generated feature quantities are statistical values ​​of biological feature values.

[0040] As a result, the information processing device 100 can accurately evaluate the occurrence of fraudulent activities against the target person. Therefore, the information processing device 100 can accurately detect the occurrence of fraudulent activities against the target person. The information processing device 100 can make it easier to prevent fraudulent damage caused by fraudulent activities against the target person.

[0041] The information processing device 100 can, for example, appropriately remove noise from the biometric data 102 of the target person in a specified section, according to the speech tendencies or behavioral tendencies corresponding to any voice pattern that matches the section. Therefore, the information processing device 100 can accurately evaluate the occurrence of fraudulent acts against the target person based on the biometric data 102 from which noise has been appropriately removed.

[0042] This explanation describes the case where the information processing device 100 operates independently, but it is not limited to this. For example, the information processing device 100 may collaborate with other computers. For example, multiple computers may implement the functions of the information processing device 100. Specifically, the functions of the information processing device 100 may be implemented on the cloud.

[0043] (An example of information processing system 200) Next, using Figure 2, we will describe an example of an information processing system 200 to which the information processing device 100 shown in Figure 1 is applied.

[0044] Figure 2 is an explanatory diagram showing an example of an information processing system 200. In Figure 2, the information processing system 200 includes an information processing device 100, a voice data acquisition device 201, a biometric data acquisition device 202, and an alert output device 203.

[0045] In the information processing system 200, the information processing device 100 and the voice data acquisition device 201 are connected via a wired or wireless network 210. The network 210 may be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet.

[0046] Furthermore, in the information processing system 200, the information processing device 100 and the biometric data acquisition device 202 are connected via a wired or wireless network 210. Also, in the information processing system 200, the information processing device 100 and the alert output device 203 are connected via a wired or wireless network 210.

[0047] The information processing device 100 is a computer for evaluating the occurrence of criminal acts against a target person. The target person is a person who is deemed to need protection from criminal acts. A criminal act is, for example, fraud. The occurrence status is, for example, whether or not the act occurred.

[0048] The information processing device 100 acquires voice data of the target person during the target period. The voice data represents a conversation between the target person and another person via telephone. The other person may be, for example, a criminal. The criminal may be, for example, a con artist. The information processing device 100 acquires the voice data of the target person during the target period by receiving it from the voice data acquisition device 201.

[0049] Specifically, the information processing device 100 acquires voice data of the target person for a certain period of time in the past, received from the voice data acquisition device 201 at regular intervals. Specifically, the information processing device 100 may acquire voice data of the target person for the period from the start to the end of a call, received from the voice data acquisition device 201, at the end of the call by the target person.

[0050] The information processing device 100 acquires biometric data of a target person during the target period. The biometric data shows the time changes in the biometric characteristics of the target person. These biometric characteristics include, for example, heart rate, pulse rate, body temperature, or sweating amount. The information processing device 100 acquires the biometric data of a target person during the target period, for example, by receiving it from the biometric data acquisition device 202.

[0051] Specifically, the information processing device 100 acquires biometric data of the target person for a certain period of time in the past, received from the biometric data acquisition device 202 at regular intervals. Specifically, the information processing device 100 may acquire biometric data of the target person for the period from the start to the end of a call, received from the biometric data acquisition device 202, at the end of the call with the target person.

[0052] The information processing device 100 evaluates the occurrence of fraudulent activities against the target person based on the acquired voice data and acquired biometric data. For example, based on the acquired voice data, the information processing device 100 classifies the target period into speaking periods and listening periods. Specifically, based on the acquired voice data, the information processing device 100 identifies multiple speakers during the target period. Specifically, based on the acquired voice data, the information processing device 100 identifies, for each speaker, the section in which the speaker spoke, the speaker's voice during that section, and the content of the speaker's speech during that section.

[0053] Here, the information processing device 100 specifically stores the voice pattern and the speech attribute in association. The voice pattern is a reference pattern that makes it possible to identify the speech attribute corresponding to a section by matching it with the voice or speech content of the target person. The speech attribute is, for example, "silence" indicating that the target person was silent, and "acknowledgment" indicating that the target person acknowledged the statement. The speech attribute may also be, for example, "utterance" indicating that the target person spoke. More specifically, the information processing device 100 stores the filter information management table 600, which will be described later in Figure 6.

[0054] Specifically, the information processing device 100 identifies a section of the target period that has speech attributes such as "silence" or "acknowledgment" by comparing the voice or speech content of the identified target person with a stored voice pattern, based on the identified speaker. Alternatively, the information processing device 100 may identify a section of the target period that has speech attributes such as "speech" by comparing the voice or speech content of the identified target person with a stored voice pattern, based on the identified speaker.

[0055] Specifically, the information processing device 100 classifies sections within the target period that have the speech attribute of "silence" and in which the speaker is a person other than the target person as listening sections. For example, the information processing device 100 classifies sections that have the speech attribute of "acknowledgment" as listening sections. Specifically, the information processing device 100 classifies sections within the target period that include at least the target person as the speaker and do not have the speech attribute of "acknowledgment" as speaking sections. More specifically, the information processing device 100 may classify sections within the target period that have the speech attribute of "speaking" as speaking sections. Specifically, the information processing device 100 may combine two or more consecutive listening sections. Specifically, the information processing device 100 may combine two or more consecutive speaking sections.

[0056] The information processing device 100 extracts and processes the biometric data of the target person from the biometric data of the target person during the listening period. Here, the information processing device 100 stores, for example, speech attributes in association with filters for processing biometric data. Specifically, the information processing device 100 stores a filter information management table 600, which will be described later in Figure 6.

[0057] The information processing device 100 removes noise from the biometric data of the target person in the section of the classified listening section that has the speech attribute of silence by applying a filter corresponding to the speech attribute of silence. The information processing device 100 removes noise from the biometric data of the target person in the section of the classified listening section that has the speech attribute of nodding by applying a filter corresponding to the speech attribute of nodding.

[0058] The information processing device 100 calculates a first feature for each listening interval based on the biometric data of the subject person after noise has been removed from that listening interval. The first feature is, for example, a statistical value of the heart rate during the listening interval. The first feature may also be the amount of change in heart rate during the listening interval.

[0059] The information processing device 100 calculates a second feature for each utterance interval based on the voice data of the target person in that utterance interval. The second feature is, for example, a feature value of the target person's emotion in the utterance interval. The emotion feature value indicates, for example, whether the target person's emotion is negative or positive.

[0060] The emotional feature value may indicate, for example, that the subject's emotion is joy, anger, sadness, disappointment, surprise, enthusiasm, or negativity. The emotional feature value may also be, for example, the level of stress. The second feature may be, for example, the amount of change in the subject's emotional feature value during the speech interval. Specifically, the second feature may be the amount of change in the subject's level of stress during the speech interval.

[0061] The information processing device 100 evaluates the occurrence of fraudulent acts against the target person based on the calculated first feature and the calculated second feature. Here, the information processing device 100 stores, for example, the change pattern of the first feature and the solution for the occurrence of fraudulent acts in association with each other. Specifically, the information processing device 100 stores the emotion pattern information management table 700, which will be described later in Figure 7. The information processing device 100 evaluates the occurrence of fraudulent acts against the target person by, for example, identifying the solution for the occurrence of fraudulent acts that corresponds to the change pattern of the calculated first feature.

[0062] The information processing device 100 may, for example, evaluate the occurrence of fraudulent activity against a target person by comparing the first calculated feature with a threshold, if the first calculated feature is the change in heart rate. The threshold is set in advance by the user, for example. Specifically, the information processing device 100 evaluates that fraudulent activity has occurred against the target person if the first feature is greater than or equal to the threshold. Specifically, the information processing device 100 evaluates that fraudulent activity has not occurred against the target person if the first feature is less than the threshold.

[0063] Furthermore, the information processing device 100 stores, for example, the change patterns of the second feature quantity and the solutions for the occurrence of fraudulent activity in association with each other. Specifically, the information processing device 100 stores the bio-pattern information management table 800, which will be described later in Figure 8. The information processing device 100 evaluates the occurrence of fraudulent activity against the target person by, for example, identifying the solutions for the occurrence of fraudulent activity that correspond to the calculated change patterns of the second feature quantity.

[0064] The information processing device 100 may, for example, evaluate the occurrence of fraudulent activity against the target person by comparing the second calculated feature with a threshold, if the second feature is the change in stress level. The threshold is set in advance by the user, for example. Specifically, the information processing device 100 evaluates that fraudulent activity has occurred against the target person if the second feature is greater than or equal to the threshold. Specifically, the information processing device 100 evaluates that fraudulent activity has not occurred against the target person if the second feature is less than the threshold.

[0065] The information processing device 100 evaluates the status of fraudulent activity against the target person, and if it determines that fraudulent activity has occurred against the target person, it sends a notification to the alert output device 203 indicating that fraudulent activity has occurred against the target person. The information processing device 100 is, for example, a server or a PC.

[0066] The voice data acquisition device 201 is a computer that acquires voice data of a target person during a specified period. For example, the voice data acquisition device 201 continuously records the target person's conversation during a phone call. For example, at regular intervals, the voice data acquisition device 201 generates voice data representing the recorded conversation of the target person for a specified period of time in the past and transmits it to the information processing device 100. The voice data acquisition device 201 can be, for example, a PC, tablet, smartphone, wearable device, or smart speaker.

[0067] The biometric data acquisition device 202 is a computer that acquires biometric data of a target person during a specified period. For example, the biometric data acquisition device 202 continuously measures the target person's biometric characteristics during a phone call. For example, at regular intervals, the biometric data acquisition device 202 generates biometric data showing the temporal changes in the measured biometric characteristics of the target person over a specified period in the past, and transmits it to the information processing device 100. The biometric data acquisition device 202 may be, for example, a PC, tablet terminal, smartphone, or wearable device.

[0068] The alert output device 203 is a computer that outputs an alert indicating that a fraudulent act has occurred against the target person. For example, the alert output device 203 receives a notification from the information processing device 100 indicating that a fraudulent act has occurred against the target person. For example, in response to receiving a notification indicating that a fraudulent act has occurred against the target person, the alert output device 203 outputs an alert indicating that a fraudulent act has occurred against the target person.

[0069] Specifically, the alert output device 203 outputs an alert to a specific person or organization indicating that fraudulent activity has occurred against a target person. The specific person may be, for example, the target person themselves or a close relative of the target person. The specific organization may be, for example, the police, a financial institution, a security company, or a nursing home. The alert output device 203 may be, for example, a PC, tablet, smartphone, wearable device, or smart speaker.

[0070] This explanation describes a case where the information processing device 100 and the voice data acquisition device 201 are different devices, but it is not limited to this case. For example, the information processing device 100 may also have the functionality of a voice data acquisition device 201 and may operate as a voice data acquisition device 201.

[0071] This explanation describes a case where the information processing device 100 and the biological data acquisition device 202 are different devices, but it is not limited to this case. For example, the information processing device 100 may also have the functionality of a biological data acquisition device 202 and may operate as a biological data acquisition device 202.

[0072] This explanation describes a case where the information processing device 100 and the alert output device 203 are different devices, but this is not limited to this case. For example, the information processing device 100 may also have the functionality of an alert output device 203 and may operate as an alert output device 203.

[0073] This explanation describes a case where the voice data acquisition device 201 and the biometric data acquisition device 202 are different devices, but it is not limited to this case. For example, the voice data acquisition device 201 may also have the functionality of a biometric data acquisition device 202 and may operate as a biometric data acquisition device 202.

[0074] This explanation describes a case where the voice data acquisition device 201 and the alert output device 203 are different devices, but this is not limited to this case. For example, the voice data acquisition device 201 may also have the function of an alert output device 203 and may operate as an alert output device 203.

[0075] This explanation describes a case where the biometric data acquisition device 202 and the alert output device 203 are different devices, but this is not limited to this case. For example, the biometric data acquisition device 202 may also have the function of an alert output device 203 and may operate as an alert output device 203.

[0076] (Examples of applications of Information Processing System 200) Next, we will explain application examples of the information processing system 200 using Figures 3 and 4.

[0077] Figure 3 is an explanatory diagram showing a first application example of the information processing system 200. In Figure 3, user 301, who is the person to be protected from fraudulent activity, has a smartphone 310. User 301 is, for example, at their home. User 301 may also be, for example, out. The information processing system 200 can be applied when user 301 is conversing with another person via the smartphone 310. The other person is, for example, a fraudster 302.

[0078] In this case, for example, the smartphone 310 would function as an information processing device 100, a voice data acquisition device 201, a biometric data acquisition device 202, and an alert output device 203.

[0079] The smartphone 310, for example, has a microphone and, during a call with user 301, uses the microphone to acquire voice data indicating user 301's conversation, thereby realizing the function of a voice data acquisition device 201. The smartphone 310, for example, has a biosensor 311 on its side and, during a call with user 301, uses the biosensor 311, which is in contact with user 301's hand, to acquire user 301's biometric data, thereby realizing the function of a biometric data acquisition device 202.

[0080] The smartphone 310 functions as an information processing device 100 by, for example, evaluating the occurrence of criminal activity against a target person based on voice data and biometric data, and determining whether or not fraudulent activity has occurred against the target person. If, for example, the smartphone 310 determines that fraudulent activity has occurred against the target person, it functions as an alert output device 203 by outputting an alert indicating that fraudulent activity has occurred against the target person. As a result, the information processing system 200 makes it easier for user 301 to prevent fraudulent activity when communicating with a fraudster via the smartphone 310. Next, we will move on to the explanation of Figure 4.

[0081] Figure 4 is an explanatory diagram showing a second application example of the information processing system 200. In Figure 4, user 401, who is the person to be protected from fraudulent activity, is wearing a wearable device 411. It is also assumed that a smart speaker 412 is present near user 401. User 401 is, for example, at their home.

[0082] Furthermore, it is assumed that there is a wearable device 411 and a server 413 capable of communicating with a smart speaker 412. Also, it is assumed that user 401's associate 403 has a smartphone 414. Associate 403 is, for example, a close relative of user 401. Associate 403 is, for example, at their own home. The information processing system 200 can be applied when user 401 converses with another person via a landline telephone 415. The other person is, for example, a fraudster 402.

[0083] In this case, for example, server 413 will function as information processing device 100. Server 413 is installed, for example, in a management company that manages the information processing system 200. Also, smart speaker 412 will function as voice data acquisition device 201. Also, wearable terminal 411 will function as biometric data acquisition device 202. Also, smartphone 414 will function as alert output device 203.

[0084] The smart speaker 412, for example, has a microphone and, during a call with user 401, uses the microphone to acquire voice data representing user 401's conversation, thereby realizing its function as a voice data acquisition device 201. The smart speaker 412 then transmits the acquired voice data to the server 413.

[0085] The wearable terminal 411, for example, has a biosensor and, when user 401 is making a call, uses the biosensor that is in contact with user 401's hand to acquire user 401's biometric data, thereby realizing the function of a biometric data acquisition device 202. The wearable terminal 411 then transmits the acquired biometric data to the server 413.

[0086] The server 413 functions as an information processing device 100 by, for example, evaluating the occurrence of criminal acts against the target person based on voice data and biometric data, and determining whether or not fraudulent acts have occurred against the target person. If the server 413 determines, for example, that fraudulent acts have occurred against the target person, it sends a notification to the smartphone 414 indicating that fraudulent acts have occurred against the target person.

[0087] The smartphone 414 functions as an alert output device 203 by, for example, receiving a notification that a fraudulent act has occurred against the target person, and outputting an alert indicating that a fraudulent act has occurred against the target person. This makes it easier for the information processing system 200 to prevent fraudulent activity when the user 401 converses with the fraudster 402 via the landline telephone 415.

[0088] For example, if the information processing system 200 determines that a fraudulent act has occurred against user 401, it can enable related parties 403 to become aware that a fraudulent act has occurred against user 401. Therefore, the information processing system 200 makes it easier for related parties 403 to quickly take measures to prevent fraudulent damage caused by the fraudulent act, such as contacting user 401 or contacting the police or bank.

[0089] This explanation describes a case where a smartphone 414 owned by a person related to user 401 (403) functions as an alert output device 203, but it is not limited to this. For example, a server or PC installed in a police station or bank may also function as an alert output device 203.

[0090] This explanation describes the application of the information processing system 200 to a situation in which fraudster 402 attempts to commit fraud via telephone, but it is not limited to this. For example, the information processing system 200 may also be applied to a situation in which fraudster 402 attempts to commit fraud in person with user 401.

[0091] (Example of hardware configuration of information processing device 100) Next, an example of the hardware configuration of the information processing device 100 will be described using Figure 5.

[0092] Figure 5 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Figure 5, the information processing device 100 includes a CPU (Central Processing Unit) 501, memory 502, network interface 503, recording medium interface 504, and recording medium 505. Each component is connected by a bus 500.

[0093] Here, the CPU 501 is responsible for the overall control of the information processing device 100. The memory 502 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as the work area for the CPU 501. Programs stored in memory 502 are loaded into the CPU 501, causing the CPU 501 to execute the coded processes. For example, memory 502 stores the filter information management table 600 (described later) in Figure 6, the emotion pattern information management table 700 (described later) in Figure 7, and the bio-pattern information management table 800 (described later) in Figure 8.

[0094] The network interface 503 is connected to network 210 via a communication line, and then connects to other computers via network 210. The network interface 503 manages the internal interface with network 210 and controls the input and output of data from other computers. The network interface 503 is, for example, a modem or a LAN adapter.

[0095] The recording medium interface (I / F) 504 controls the reading and writing of data to the recording medium (SSD) 505 according to the control of the CPU 501. The recording medium interface (I / F) 504 is, for example, a disk drive, an SSD (Solid State Drive), or a USB (Universal Serial Bus) port. The recording medium (SSD) 505 is a non-volatile memory that stores the data written under the control of the recording medium interface (I / F) 504. The recording medium (SSD) 505 is, for example, a disk, semiconductor memory, or USB memory. The recording medium (SSD) 505 may be detachable from the information processing device (SOS) 100.

[0096] In addition to the components described above, the information processing device 100 may also have, for example, a keyboard, mouse, printer, or scanner. Furthermore, the information processing device 100 may have multiple recording medium interfaces 504 and recording mediums 505. Alternatively, the information processing device 100 may not have recording medium interfaces 504 and recording mediums 505.

[0097] Furthermore, the information processing device 100 may have components corresponding to the voice sensor 906, which will be described later in Figure 9, in addition to the components described above. Furthermore, the information processing device 100 may have components corresponding to the biosensor 907, which will be described later in Figure 9, in addition to the components described above. Furthermore, the information processing device 100 may have components corresponding to the output device 908, which will be described later in Figure 9, in addition to the components described above.

[0098] (Contents stored in filter information management table 600) Next, an example of the contents of the filter information management table 600 will be explained using Figure 6. The filter information management table 600 is implemented, for example, by a storage area such as the memory 502 or recording medium 505 of the information processing device 100 shown in Figure 5.

[0099] Figure 6 is an explanatory diagram showing an example of the contents stored in the filter information management table 600. As shown in Figure 6, the filter information management table 600 has fields for voice pattern, type, and filter. Filter information is stored as record 600-a in the filter information management table 600 by setting information in each field for each voice pattern. a is an arbitrary integer.

[0100] The voice pattern field contains a voice pattern that indicates the conditions for classifying the period during which a person is speaking into multiple segments and assigning speech attributes to those segments. The voice pattern makes it possible to identify the speech attributes to be assigned to each segment. For example, the voice pattern indicates the characteristics of the conversational speech shown by the voice data corresponding to the speech attributes. Specifically, the voice pattern indicates keywords corresponding to the speech attributes. Specifically, the voice pattern may indicate the tone, volume, or wavelength of the voice corresponding to the speech attributes.

[0101] The voice pattern field may contain, specifically, voice patterns corresponding to "silence" or voice patterns corresponding to "acknowledgments." The voice pattern field may also contain, specifically, voice patterns corresponding to "speech." The voice patterns corresponding to "speech" may indicate that they do not match the voice patterns corresponding to "silence" or the voice patterns corresponding to "acknowledgments," rather than being specific keywords, specific voice tones, volume, or wavelengths.

[0102] The "Type" field is set to the type of speech attribute described above. For example, the "Type" field is set to the type of speech attribute to be assigned to the section corresponding to the voice pattern during the period in which the target person is speaking. The "Filter" field is set to the filter to be applied to the biometric data in the section corresponding to the voice pattern. The filter has a function, for example, to remove noise from the biometric data.

[0103] (Memory contents of Emotion Pattern Information Management Table 700) Next, an example of the contents of the emotion pattern information management table 700 will be explained using Figure 7. The emotion pattern information management table 700 is implemented, for example, by a storage area such as the memory 502 or recording medium 505 of the information processing device 100 shown in Figure 5.

[0104] Figure 7 is an explanatory diagram showing an example of the contents stored in the emotion pattern information management table 700. As shown in Figure 7, the emotion pattern information management table 700 has fields for emotion pattern and judgment result. The emotion pattern information is stored as record 700-b by setting information in each field for each emotion pattern in the emotion pattern information management table 700. b is an arbitrary integer.

[0105] The emotion pattern field is set with emotion patterns that indicate criteria for evaluating the occurrence of fraudulent activity. Emotion patterns are, for example, patterns of combinations of emotional characteristic values ​​of the target person in each of several intervals. The judgment result field is set with judgment results that represent the solution to the occurrence of fraudulent activity, corresponding to the case where the combination of emotional characteristic values ​​of the actually identified target person in each of the multiple intervals matches the above emotion pattern.

[0106] For example, record 700-b specifically outlines a rule that determines fraud has occurred if the emotional characteristic values ​​of the person in question in each of the multiple utterance segments are in the order of positive, negative, positive from beginning to end.

[0107] (Contents stored in the bio-pattern information management table 800) Next, an example of the contents of the biopattern information management table 800 will be explained using Figure 8. The biopattern information management table 800 is implemented, for example, by a storage area such as the memory 502 or recording medium 505 of the information processing device 100 shown in Figure 5.

[0108] Figure 8 is an explanatory diagram showing an example of the contents stored in the biopattern information management table 800. As shown in Figure 8, the biopattern information management table 800 has fields for biopattern and judgment result. The biopattern information management table 800 stores biopattern information as record 800-c by setting information in each field for each biopattern. c is an arbitrary integer.

[0109] The biopattern field is set with biopatterns that indicate criteria for evaluating the occurrence of fraudulent activity. A biopattern is, for example, a pattern of combinations of biometric characteristics of a target person in each of several intervals. A biopattern may also be, for example, a pattern of combinations of statistical values ​​of biometric characteristics of a target person in each of several intervals. A biopattern may also be, for example, a pattern of the temporal change of biometric characteristics of a target person in a given interval.

[0110] The judgment result field contains a judgment result that represents the solution to the occurrence of fraudulent activity, corresponding to cases where the combination pattern of the biological characteristic values ​​of the actually identified target person matches the above-mentioned biological pattern for each of the multiple intervals. The judgment result field contains a judgment result that represents the solution to the occurrence of fraudulent activity, corresponding to cases where the combination pattern of the statistical values ​​of the biological characteristic values ​​of the actually identified target person matches the above-mentioned biological pattern for each of the multiple intervals. The judgment result field may also contain a judgment result that represents the solution to the occurrence of fraudulent activity, corresponding to cases where the pattern of the temporal change of the biological characteristic values ​​of the actually identified target person matches the above-mentioned biological pattern for a given interval.

[0111] For example, record 800-c specifically outlines a rule that determines fraud has occurred if, over a certain interval, the pattern of change in the biological characteristics of the identified individual over time matches a biological pattern that shows monotonically increasing changes.

[0112] (Example of hardware configuration for audio data acquisition device 201) Next, an example of the hardware configuration of the audio data acquisition device 201 will be described using Figure 9.

[0113] Figure 9 is a block diagram showing an example of the hardware configuration of the voice data acquisition device 201. In Figure 9, the voice data acquisition device 201 includes a CPU 901, a memory 902, a network interface 903, a recording medium interface 904, a recording medium 905, a voice sensor 906, a biosensor 907, and an output device 908. Each component is connected by a bus 900.

[0114] Here, the CPU 901 is responsible for the overall control of the audio data acquisition device 201. The memory 902 includes, for example, ROM, RAM, and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as the work area for the CPU 901. Programs stored in memory 902 are loaded into the CPU 901, causing the CPU 901 to execute the coded processes.

[0115] The network interface 903 is connected to network 210 via a communication line, and then connects to other computers via network 210. The network interface 903 then manages the internal interface with network 210 and controls the input and output of data from other computers. The network interface 903 is, for example, a modem or a LAN adapter.

[0116] The recording medium interface 904 controls the reading and writing of data to the recording medium 905 according to the control of the CPU 901. The recording medium interface 904 is, for example, a disk drive, SSD, or USB port. The recording medium 905 is a non-volatile memory that stores the data written under the control of the recording medium interface 904. The recording medium 905 is, for example, a disk, semiconductor memory, or USB memory. The recording medium 905 may be detachable from the audio data acquisition device 201.

[0117] The voice sensor 906 has a microphone for detecting sound, and acquires audio data indicating the detected sound by detecting the sound of a target person using the microphone. The voice sensor 906 acquires audio data, for example, by generating audio data indicating a conversation between the target person and another person detected using the microphone.

[0118] The biosensor 907 acquires biometric data of a target person. The biosensor 907 acquires this data by, for example, measuring the target person's biometric characteristics such as heart rate, pulse rate, sweat rate, or body temperature, and generating biometric data that shows the time change of the measured biometric characteristics. The biosensor 907 may, for example, include an imaging device and measure the target person's biometric characteristics such as heart rate, pulse rate, sweat rate, or body temperature based on image information of the target person captured by the imaging device. The biosensor 907 may, for example, include a millimeter-wave sensor and measure the target person's biometric characteristics such as heart rate, pulse rate, sweat rate, or body temperature using the millimeter-wave sensor.

[0119] The output device 908 includes a speaker or a display. The output device 908 outputs data such as audio, documents, or images using the speaker or display. The output device 908 may, for example, output an alert indicating that fraudulent activity has occurred. The display may be, for example, a CRT (Cathode Ray Tube), a liquid crystal display, or an organic EL (Electroluminescence) display.

[0120] In addition to the components described above, the voice data acquisition device 201 may also have, for example, a keyboard, mouse, printer, or scanner. Furthermore, the voice data acquisition device 201 may have multiple recording medium interfaces 904 and recording mediums 905. Moreover, the voice data acquisition device 201 may not have recording medium interfaces 904 and recording mediums 905.

[0121] (Example hardware configuration of the biometric data acquisition device 202) The hardware configuration example for the biometric data acquisition device 202 is specifically the same as the hardware configuration example for the voice data acquisition device 201 shown in Figure 9, so a detailed explanation is omitted.

[0122] (Example hardware configuration of alert output device 203) The hardware configuration example for the alert output device 203 is specifically the same as the hardware configuration example for the voice data acquisition device 201 shown in Figure 9, so a detailed explanation is omitted.

[0123] (Example of the functional configuration of the information processing device 100) Next, an example of the functional configuration of the information processing device 100 will be described using Figure 10.

[0124] Figure 10 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 1000, an acquisition unit 1001, a classification unit 1002, a processing unit 1003, a calculation unit 1004, a determination unit 1005, and an output unit 1006.

[0125] The storage unit 1000 is implemented by a storage area such as the memory 502 or recording medium 505 shown in Figure 5. The following description will focus on the case where the storage unit 1000 is included in the information processing device 100, but is not limited to this case. For example, the storage unit 1000 may be included in a device different from the information processing device 100, and the contents of the storage unit 1000 may be accessible from the information processing device 100.

[0126] The acquisition unit 1001 to the output unit 1006 function as an example of a control unit. Specifically, the acquisition unit 1001 to the output unit 1006 realize their functions, for example, by having the CPU 501 execute a program stored in a storage area such as the memory 502 or recording medium 505 shown in Figure 5, or by using the network I / F 503. The processing results of each functional unit are stored in a storage area such as the memory 502 or recording medium 505 shown in Figure 5.

[0127] The memory unit 1000 stores various types of information that are referenced or updated during the processing of each functional unit. For example, the memory unit 1000 stores audio data. Specifically, the memory unit 1000 acquires audio data that shows the conversations of a target person during a target period. The target person is a person who is desired to be protected from criminal activity.

[0128] The criminal act is, for example, fraud. Specifically, the criminal act is fraud conducted via telephone. Specifically, the criminal act may also be fraud carried out in person with the target person. The audio data shows, for example, a conversation between the target person and another person via telephone. The other person may be, for example, a criminal. The criminal is, for example, a con artist.

[0129] The target period is, for example, the period during which the target person is having a conversation. The period during a conversation corresponds to, for example, the period during a phone call. Specifically, the target period is the period from the start of the target person's conversation to the end of the target person's conversation. The start of the conversation corresponds to, for example, the start of a phone call. The end of the conversation corresponds to, for example, the end of a phone call. Specifically, the target period may be a portion of the entire period from the start of the target person's conversation to the end of the target person's conversation. The audio data is acquired, for example, by the acquisition unit 1001.

[0130] The memory unit 1000 stores, for example, biometric data. Specifically, the memory unit 1000 acquires biometric data of a subject person over a specified period. The biometric data shows, for example, the temporal changes in the biometric characteristics of the subject person. These biometric characteristics include, for example, heart rate, pulse rate, body temperature, or sweating. Specifically, these biometric characteristics include heart rate, pulse rate, body temperature, or sweat volume. The biometric data is acquired, for example, by the acquisition unit 1001.

[0131] The memory unit 1000 stores, for example, speech patterns that allow the target period to be classified into multiple segments. These segments include, for example, utterance segments in which the target person spoke and listening segments in which the target person listened to the utterances of other people. Specifically, the utterance segments are segments in which the target person uttered content other than acknowledgments.

[0132] Multiple intervals may include, for example, a silence interval in which the subject is silent and an interjection interval in which the subject nods in agreement. The silence interval and the interjection interval are included, for example, in the listening interval. Specifically, the memory unit 1000 stores a speech pattern for each speech attribute. Speech attributes include, for example, "silence" indicating that the subject was silent, and "interjection" indicating that the subject nodded in agreement. Speech attributes may also include, for example, "utterance" indicating that the subject spoke. Speech attributes may also include, for example, "listening" indicating that the subject listened to the utterance of another person.

[0133] More specifically, the memory unit 1000 stores voice patterns corresponding to "silence" and voice patterns corresponding to "acknowledgments." More specifically, the memory unit 1000 may also store voice patterns corresponding to "speech." More specifically, the memory unit 1000 may also store voice patterns corresponding to "listening." Specifically, the voice patterns indicate keywords corresponding to speech attributes. More specifically, the voice patterns indicate keywords corresponding to "acknowledgments." Specifically, the voice patterns may indicate the tone, volume, or wavelength of the voice corresponding to speech attributes. More specifically, the voice patterns may indicate the upper limit threshold of the target person's voice volume in association with "silence." The memory unit 1000 may store as a voice pattern corresponding to "speech" that does not match the voice patterns corresponding to "silence" and the voice patterns corresponding to "acknowledgments." More specifically, the memory unit 1000 stores the filter information management table 600 shown in Figure 6.

[0134] The memory unit 1000 stores, for example, each voice pattern of one or more voice patterns in association with the content of noise reduction processing to remove noise from biometric data. The noise reduction processing is performed, for example, using a filter. Specifically, the memory unit 1000 stores, for example, each voice pattern of one or more voice patterns in association with a filter to remove noise from biometric data. The filter removes noise of a magnitude and type corresponding to the speech tendency or the movement tendency of the person corresponding to the voice pattern, depending on the speech tendency or the movement tendency of the person corresponding to the voice pattern. More specifically, the memory unit 1000 stores the filter information management table 600 shown in Figure 6.

[0135] The memory unit 1000 stores, for example, information that enables the evaluation of the circumstances under which a criminal act occurs. Specifically, the memory unit 1000 stores first correspondence information that associates a change pattern of feature quantities based on audio data with a solution for the circumstances under which a criminal act occurs that corresponds to the change pattern. The feature quantities based on audio data include, for example, one or more feature values ​​of a person's emotions, a statistical value of the feature values ​​of a person's emotions in a certain interval, or the amount of change in the feature values ​​of a person's emotions in a certain interval. The feature values ​​of emotions are, for example, feature values ​​that indicate a psychological state.

[0136] The emotional characteristic value indicates, for example, whether the subject person's emotions are negative or positive. The emotional characteristic value may also indicate, for example, whether the subject person's emotions are joy, anger, sadness, disappointment, surprise, enthusiasm, or negativity. The emotional characteristic value may also be, for example, the degree of stress. The change pattern corresponds to, for example, the emotional pattern shown in Figure 7. The solution is, for example, whether or not a criminal act occurred. More specifically, the memory unit 1000 stores the emotional pattern information management table 700 shown in Figure 7. The first correspondence information is acquired, for example, by the acquisition unit 1001. The first correspondence information may be stored in the memory unit 1000 in advance, for example.

[0137] Specifically, the memory unit 1000 may store a first model that outputs a solution for the occurrence of a criminal act corresponding to a change pattern in feature quantities based on audio data, in response to the input of such change pattern. The first model is acquired, for example, by the acquisition unit 1001. The first model may be stored in the memory unit 1000 in advance, for example. The first model may be trained, for example, based on training data.

[0138] Specifically, the memory unit 1000 stores second correspondence information that associates a change pattern of feature quantities based on biometric data with a solution for the occurrence of a criminal act corresponding to that change pattern. The feature quantities based on biometric data are, for example, one or more biometric feature values ​​of a person, a statistical value of a person's biometric feature values ​​in a certain interval, or the amount of change in a person's biometric feature values ​​in a certain interval. The change pattern corresponds to, for example, the biometric pattern shown in Figure 8. The solution is, for example, whether or not a criminal act occurred. More specifically, the memory unit 1000 stores the biometric pattern information management table 800 shown in Figure 8. The second correspondence information is acquired, for example, by the acquisition unit 1001. The second correspondence information may, for example, be stored in the memory unit 1000 in advance.

[0139] Specifically, the memory unit 1000 may store a second model that outputs a solution to the occurrence of criminal acts corresponding to a change pattern in feature quantities based on biometric data, in response to the input of such change pattern. The second model is acquired, for example, by the acquisition unit 1001. The second model may be stored in the memory unit 1000 in advance, for example. The second model may be trained, for example, based on training data.

[0140] Specifically, the memory unit 1000 may store third correspondence information that associates a combination of a feature change pattern based on voice data and a feature change pattern based on biometric data with a solution for the occurrence of a criminal act corresponding to that combination. The solution is, for example, whether or not a criminal act occurred. The third correspondence information is acquired, for example, by the acquisition unit 1001. The third correspondence information may, for example, be stored in the memory unit 1000 in advance.

[0141] Specifically, the memory unit 1000 may store a third model that outputs a solution to the occurrence of a criminal act corresponding to a combination of a pattern of change in feature quantities based on speech data and a pattern of change in feature quantities based on biometric data. The third model is acquired, for example, by the acquisition unit 1001. The third model may be stored in the memory unit 1000 in advance, for example. The third model may be learned, for example, based on training data.

[0142] The acquisition unit 1001 acquires various types of information used in the processing of each functional unit. The acquisition unit 1001 stores the acquired information in the storage unit 1000 or outputs it to each functional unit. The acquisition unit 1001 may also output the information stored in the storage unit 1000 to each functional unit. The acquisition unit 1001 acquires various types of information, for example, based on user input. The acquisition unit 1001 may also receive various types of information from a device other than the information processing device 100, for example.

[0143] The acquisition unit 1001 acquires, for example, audio data representing the conversation of the target person during the target period. Specifically, the acquisition unit 1001 acquires this data by continuously recording the conversation of the target person using a microphone while the target person is on a call, and generating audio data representing the conversation of the target person during the target period from the start to the end of the conversation.

[0144] Specifically, the acquisition unit 1001 may continuously record the target person's conversation using a microphone while the target person is on a call, and generate audio data at predetermined intervals during the target person's call, indicating the target person's conversation during the period from the current time to a certain time prior. The predetermined intervals are, for example, at regular intervals from the start of the conversation.

[0145] Specifically, the acquisition unit 1001 may acquire audio data representing the conversation of the target person during the target period by receiving it from another computer. More specifically, the acquisition unit 1001 acquires audio data representing the conversation of the target person during the target period from the start to the end of the conversation by receiving it from another computer at the end of the conversation of the target person. More specifically, the acquisition unit 1001 acquires audio data representing the conversation of the target person during the target period from the present to a certain time prior by receiving it from another computer at predetermined timings during the conversation of the target person. The predetermined timings are, for example, at regular intervals from the start of the conversation.

[0146] The acquisition unit 1001 acquires, for example, the biometric data of the target person during the target period. Specifically, the acquisition unit 1001 continuously measures the biometric characteristic values ​​of the target person using a biosensor while the target person is on a call. Then, specifically, the acquisition unit 1001 acquires the data by generating biometric data that shows the time change of the biometric characteristic values ​​of the target person during the target period, based on the biometric characteristic values ​​of the target person measured during the target period from the start to the end of the conversation.

[0147] Specifically, the acquisition unit 1001 continuously measures the biological characteristics of the target person using a biosensor while the target person is on a call. The acquisition unit 1001 may then acquire the data by generating biometric data that shows the temporal changes in the biological characteristics of the target person over a specified period at predetermined timings during the target person's call. The predetermined timings are, for example, at regular intervals from the start of the conversation. More specifically, the acquisition unit 1001 generates biometric data that shows the temporal changes in the biological characteristics of the target person over a specified period, based on the biological characteristics of the target person measured during the specified period from the present to a certain time prior, at predetermined timings.

[0148] Specifically, the acquisition unit 1001 may acquire biometric data showing the time-dependent changes in the biological characteristics of the target person during the target period by receiving it from another computer. More specifically, the acquisition unit 1001 acquires biometric data showing the time-dependent changes in the biological characteristics of the target person during the target period from the start to the end of the conversation by receiving it from another computer at the end of the conversation with the target person. More specifically, the acquisition unit 1001 acquires biometric data showing the time-dependent changes in the biological characteristics of the target person during the target period from the present to a certain time prior by receiving it from another computer at predetermined timings during the conversation with the target person. The predetermined timings are, for example, at regular intervals from the start of the conversation.

[0149] The acquisition unit 1001 acquires, for example, information that makes it possible to evaluate the occurrence of criminal acts. Specifically, the acquisition unit 1001 acquires first correspondence information that associates a change pattern of feature quantities based on voice data with a solution for the occurrence of criminal acts corresponding to the change pattern. More specifically, the acquisition unit 1001 acquires the first correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the first correspondence information by accepting input of the first correspondence information based on user operation input.

[0150] Specifically, the acquisition unit 1001 may acquire a first model that outputs a solution to the occurrence of a criminal act corresponding to a change pattern in feature quantities based on audio data, in response to the input of such change pattern. More specifically, the acquisition unit 1001 may acquire the first model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the first model by accepting input for the first model based on user operation input. More specifically, the acquisition unit 1001 may acquire the first model by training it based on training data that associates samples of change patterns in feature quantities based on audio data with the correct solution to the occurrence of a criminal act corresponding to the change pattern.

[0151] Specifically, the acquisition unit 1001 acquires second correspondence information, which associates a change pattern of a feature quantity based on biometric data with a solution for the occurrence of a criminal act corresponding to that change pattern. More specifically, the acquisition unit 1001 acquires the second correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the second correspondence information by accepting input of the second correspondence information based on user operation input.

[0152] Specifically, the acquisition unit 1001 may acquire a second model that outputs a solution for the occurrence of a criminal act corresponding to a change pattern in a feature based on biometric data, in response to the input of the change pattern in the feature based on the biometric data. More specifically, the acquisition unit 1001 may acquire the second model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the second model by accepting input for the second model based on user operation input. More specifically, the acquisition unit 1001 may acquire the second model by training it based on training data that associates samples of change patterns in feature based on biometric data with the correct solution for the occurrence of a criminal act corresponding to the change pattern.

[0153] Specifically, the acquisition unit 1001 acquires third correspondence information, which associates a combination of a feature change pattern based on voice data and a feature change pattern based on biometric data with a solution for the occurrence of a criminal act corresponding to that combination. More specifically, the acquisition unit 1001 acquires the third correspondence information by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the third correspondence information by accepting input of the third correspondence information based on user operation input.

[0154] Specifically, the acquisition unit 1001 may acquire a third model that outputs a solution to the occurrence of a criminal act corresponding to a combination of a feature change pattern based on speech data and a feature change pattern based on biometric data, in response to the input of such combination. More specifically, the acquisition unit 1001 may acquire the third model by receiving it from another computer. More specifically, the acquisition unit 1001 may acquire the third model by accepting input for the third model based on user operation input. More specifically, the acquisition unit 1001 may acquire the third model by training it based on training data. The training data is, for example, data that associates samples of combinations of feature change patterns based on speech data and feature change patterns based on biometric data with the correct solution to the occurrence of a criminal act corresponding to that combination.

[0155] The acquisition unit 1001 may receive a start trigger to initiate processing in any of the functional units. A start trigger may be, for example, a predetermined operation input by a user. A start trigger may also be, for example, the receipt of predetermined information from another computer. A start trigger may also be, for example, the output of predetermined information by any of the functional units.

[0156] The acquisition unit 1001 may, for example, receive the detection of the end of a conversation with a target person as a start trigger to initiate processing by the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005. The acquisition unit 1001 may, for example, receive the arrival of a predetermined timing as a start trigger to initiate processing by the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005. The acquisition unit 1001 may also receive the acquisition of voice data and biometric data as a start trigger to initiate processing by the classification unit 1002, the processing unit 1003, the calculation unit 1004, and the determination unit 1005.

[0157] The classification unit 1002 classifies the target period into multiple segments based on the acquired audio data. These segments may include, for example, speech segments and listening segments. Alternatively, the segments may include, for example, silence segments and response segments. The segments may also include speech segments, listening segments, silence segments, and response segments. Silence segments and response segments may be included within listening segments. The target period may include, for example, unclassified time periods. The classification unit 1002 may choose not to classify any of the time periods included in the target period into speech segments, listening segments, silence segments, or response segments.

[0158] The classification unit 1002, for example, classifies the target period into speaking intervals and listening intervals based on the acquired audio data. The classification unit 1002 may also classify the target period into silent intervals and response intervals based on the acquired audio data. The classification unit 1002 may also classify the target period into silent intervals and response intervals, as well as speaking intervals and listening intervals, based on the acquired audio data.

[0159] Specifically, the classification unit 1002 identifies multiple speakers during the target period by analyzing the audio data. Specifically, for each of the identified speakers, the classification unit 1002 identifies the section in which the speaker spoke, the speaker's voice during that section, and the content of the speaker's speech during that section. The speaker may be, for example, the target person or another person. The speaker's voice may indicate, for example, the speaker's tone, volume, or wavelength.

[0160] Specifically, the classification unit 1002 classifies sections in which the target person utters utterances other than acknowledgments, based on the identified speaker, section, voice, and utterance content, into utterance sections. More specifically, the classification unit 1002 classifies sections in which the target person utters utterances, based on the identified speaker, section, voice, and utterance content, into utterance sections, where the target person's voice or utterance content matches a voice pattern corresponding to "utterance". This allows the classification unit 1002 to obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process voice data or biometric data in order to evaluate the occurrence of criminal acts against the target person.

[0161] Specifically, the classification unit 1002 classifies sections in which the target person utters utterances corresponding to acknowledgments as acknowledgment sections, based on the identified speaker, section, voice, and utterance content. Specifically, the classification unit 1002 may also classify sections in which another person is speaking and the target person utters utterances corresponding to acknowledgments as acknowledgment sections. More specifically, the classification unit 1002 classifies sections in which the target person utters utterances, based on the identified speaker, section, voice, and utterance content, as acknowledgment sections, where the target person's voice or utterance content matches a voice pattern corresponding to an "acknowledgment." This allows the classification unit 1002 to obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process voice data or biometric data in order to evaluate the occurrence of criminal acts against the target person.

[0162] Specifically, the classification unit 1002 classifies sections in which the target person does not speak and remains silent as silent sections, based on the identified speaker, section, voice, and content of speech. Specifically, the classification unit 1002 may also classify sections in which another person is speaking and the target person does not speak and remains silent as silent sections. More specifically, the classification unit 1002 classifies sections in which the target person speaks and whose voice or content of speech matches a voice pattern corresponding to "silence" as silent sections, based on the identified speaker, section, voice, and content of speech. This allows the classification unit 1002 to obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process voice data or biometric data in order to evaluate the occurrence of criminal acts against the target person.

[0163] Specifically, the classification unit 1002 classifies the sections in which the target person listened to another person's speech as listening sections, based on the identified speaker, section, voice, and content of speech. More specifically, the classification unit 1002 classifies the sections in which the person gave verbal cues and the sections of silence as listening sections. More specifically, the classification unit 1002 may classify the sections in which the target person spoke as listening sections, based on the identified speaker, section, voice, and content of speech, if the target person's voice or content of speech matches a voice pattern corresponding to "listening". This allows the classification unit 1002 to obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process voice data or biometric data in order to evaluate the occurrence of criminal acts against the target person.

[0164] The classification unit 1002, for example, identifies sections within the target period that match one or more voice patterns based on the acquired voice data. Specifically, the classification unit 1002 identifies listening sections that match the voice pattern corresponding to "speech," as described above. Specifically, the classification unit 1002 identifies listening sections that match the voice pattern corresponding to "silence," as described above. Specifically, the classification unit 1002 identifies listening sections that match the voice pattern corresponding to "acknowledgment," as described above. Specifically, the classification unit 1002 identifies listening sections that match the voice pattern corresponding to "listening," as described above. This allows the classification unit 1002 to obtain guidelines for evaluating the occurrence of criminal acts against the target person. For example, the classification unit 1002 can obtain guidelines for determining how to process biometric data in order to evaluate the occurrence of criminal acts against the target person.

[0165] The processing unit 1003 performs noise reduction processing corresponding to the audio pattern on the biometric data of the target person in the section that matches the audio pattern. Specifically, the biometric data of the target person in the section is a portion of the biometric data of the target person during the target period.

[0166] The processing unit 1003, for example, refers to the storage unit 1000 and performs noise reduction processing on the biometric data of the target person in the section that matches each voice pattern, using a filter associated with that voice pattern. In this way, the processing unit 1003 can process the biometric data in order to improve the accuracy of evaluating the occurrence of criminal acts against the target person.

[0167] The calculation unit 1004 generates a first feature quantity relating to the target person based on biometric data. For example, the calculation unit 1004 generates the first feature quantity relating to the target person based on partial biometric data relating to the classified listening interval from the biometric data acquired for the target period. The partial biometric data may, for example, have undergone noise reduction processing in the processing unit 1003. The partial biometric data may, for example, not have undergone noise reduction processing in the processing unit 1003.

[0168] Specifically, the calculation unit 1004 generates a first feature quantity related to the target person based on partial biometric data related to the acquired biometric data for the target period for each classified listening interval. The first feature quantity is, for example, one or more biometric feature values ​​of the target person, a statistical value of the biometric feature values ​​of the target person in the listening interval, or the amount of change in the biometric feature values ​​of the target person in the listening interval. Specifically, the first feature quantity is a statistical value of the feature value related to the heart rate or pulse of the target person in the listening interval, or the amount of change in the feature value related to the heart rate or pulse of the target person in the listening interval. As a result, the calculation unit 1004 enables the determination unit 1005 to evaluate the occurrence of criminal acts against the target person.

[0169] The calculation unit 1004 can calculate a first feature quantity by excluding, for example, partial speech data from the biometric data that tends to have relatively large noise superpositions such as body movement noise, making it difficult to use in evaluating the occurrence of criminal acts. The calculation unit 1004 can selectively use, for example, partial biometric data from the biometric data that tends to have relatively small noise superpositions such as body movement noise, making it easy to use in evaluating the occurrence of criminal acts. The calculation unit 1004 can calculate a first feature quantity by selectively using partial biometric data from the biometric data that is easy to use in evaluating the occurrence of criminal acts. As a result, the calculation unit 1004 enables the determination unit 1005 to accurately evaluate the occurrence of criminal acts against the target person.

[0170] The calculation unit 1004 generates a second feature quantity relating to the target person based on the audio data. For example, the calculation unit 1004 generates a second feature quantity relating to the target person based on partial audio data relating to the classified speech interval from the audio data acquired for the target period.

[0171] Specifically, the calculation unit 1004 generates a second feature quantity related to the target person for each classified speech segment, based on the partial speech data related to that speech segment from the acquired speech data for the target period. The second feature quantity may be, for example, one or more feature values ​​of the target person's emotions, a statistical value of the feature value of the target person's emotions in the speech segment, or the amount of change in the feature value of the target person's emotions in the speech segment. Specifically, the second feature quantity may be a statistical value or the amount of change of the target person's stress level in the speech segment. This allows the calculation unit 1004 to enable the determination unit 1005 to evaluate the occurrence of criminal acts against the target person.

[0172] The calculation unit 1004 can calculate a second feature quantity by excluding, for example, partial audio data from the audio data relating to the interjection interval, which, although spoken by the target person, are difficult to read for their voice characteristics and are therefore difficult to use in evaluating the occurrence of criminal activity. The calculation unit 1004 can also calculate a second feature quantity by selectively using, for example, partial audio data from the audio data relating to the utterance interval, which are easy to read for their voice characteristics and are therefore easy to use in evaluating the occurrence of criminal activity. As a result, the calculation unit 1004 enables the determination unit 1005 to accurately evaluate the occurrence of criminal activity against the target person.

[0173] The determination unit 1005 evaluates the circumstances under which a criminal act has occurred against the target person. The criminal act in question is, for example, a criminal act that may be committed by another person who is conversing with the target person via telephone. Specifically, the criminal act is a fraudulent act. Examples of fraudulent acts include "ore-ore" fraud (impersonation fraud), refund fraud, or theatrical fraud.

[0174] The determination unit 1005 evaluates, for example, the occurrence of criminal acts against the target person based on the first feature quantity it generates. Specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the change pattern of the first feature quantity it generates.

[0175] More specifically, the determination unit 1005 calculates the change in the generated first feature. The change is, for example, the difference between the first feature generated for each of two different listening intervals. Specifically, the change is the difference between the first feature generated for the first listening interval and the first feature generated for the last listening interval during the target period. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred if the change in the generated first feature is greater than or equal to a threshold. The threshold is, for example, set in advance by the user. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has not occurred if the change in the generated first feature is less than the threshold.

[0176] More specifically, the determination unit 1005 may evaluate the occurrence of a criminal act against the target person by comparing the second correspondence information described above with the change pattern of the generated first feature quantity. More specifically, the determination unit 1005 evaluates that a criminal act has occurred against the target person if the change pattern of the generated first feature quantity matches a specific change pattern set in the second correspondence information, which has as the solution that a criminal act has occurred.

[0177] More specifically, the determination unit 1005 evaluates that no criminal act has occurred against the target person if the generated first feature change pattern does not match a specific change pattern set in the second corresponding information which has the solution that a criminal act has occurred. More specifically, the determination unit 1005 may evaluate that no criminal act has occurred against the target person if the generated first feature change pattern matches a specific change pattern set in the second corresponding information which has the solution that a criminal act has not occurred.

[0178] More specifically, the determination unit 1005 may use the second model described above to evaluate the occurrence of criminal acts against the target person by inputting the generated change pattern of the first feature quantity into the second model. More specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the solution of the occurrence of criminal acts output by the second model in response to the input.

[0179] Specifically, the determination unit 1005 may compare the generated first feature with a threshold if the first feature represents a change in a biological feature value. The threshold is set in advance by the user, for example. Specifically, the determination unit 1005 evaluates that a criminal act has occurred against the target person if the generated first feature is greater than or equal to the threshold. Specifically, the determination unit 1005 evaluates that a criminal act has not occurred against the target person if the generated first feature is less than the threshold.

[0180] As a result, the determination unit 1005 can accurately evaluate the circumstances under which a criminal act has occurred. For example, the determination unit 1005 can accurately determine whether or not a criminal act has occurred. Specifically, the determination unit 1005 utilizes a first feature quantity based on partial biometric data, which tends to have relatively small noise superposition, and can therefore accurately evaluate the circumstances under which a criminal act has occurred against the target person.

[0181] The determination unit 1005 evaluates, for example, the occurrence of criminal acts against the target person based on the generated second feature quantity. Specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the change pattern of the generated second feature quantity.

[0182] More specifically, the determination unit 1005 calculates the change in the generated second feature. The change is, for example, the difference between the second feature generated for each of two different speech segments. Specifically, the change is the difference between the second feature generated for the first speech segment and the second feature generated for the last speech segment during the target period. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has occurred if the change in the generated second feature is greater than or equal to a threshold. The threshold is, for example, set in advance by the user. More specifically, the determination unit 1005 evaluates that a criminal act against the target person has not occurred if the change in the generated second feature is less than the threshold.

[0183] More specifically, the determination unit 1005 may evaluate the occurrence of a criminal act against the target person by comparing the first correspondence information described above with the change pattern of the generated second feature quantity. More specifically, the determination unit 1005 evaluates that a criminal act has occurred against the target person if the change pattern of the generated second feature quantity matches a specific change pattern set in the first correspondence information, which has the solution that a criminal act has occurred.

[0184] More specifically, the determination unit 1005 evaluates that no criminal act has occurred against the target person if the generated change pattern of the second feature quantity does not match a specific change pattern set in the first corresponding information which has the solution that a criminal act has occurred. More specifically, the determination unit 1005 may evaluate that no criminal act has occurred against the target person if the generated change pattern of the second feature quantity matches a specific change pattern set in the first corresponding information which has the solution that a criminal act has not occurred.

[0185] More specifically, the determination unit 1005 may use the first model described above to evaluate the occurrence of criminal acts against the target person by inputting the generated change pattern of the second feature quantity into the first model. More specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the solution of the occurrence of criminal acts output by the first model in response to the input.

[0186] Specifically, the determination unit 1005 may compare the generated second feature with a threshold if the generated second feature is a change in the emotion feature value. The threshold is set in advance by the user, for example. Specifically, the determination unit 1005 evaluates that a criminal act has occurred against the target person if the generated second feature is greater than or equal to the threshold. Specifically, the determination unit 1005 evaluates that a criminal act has not occurred against the target person if the generated second feature is less than the threshold.

[0187] As a result, the determination unit 1005 can accurately evaluate the circumstances under which a criminal act has occurred. For example, the determination unit 1005 can accurately determine whether or not a criminal act has occurred. Specifically, the determination unit 1005 utilizes a second feature quantity based on partial audio data from the audio data, which is easy to read for its characteristics and is readily usable for evaluating the circumstances under which a criminal act has occurred. Therefore, it can accurately evaluate the circumstances under which a criminal act has occurred against the target person.

[0188] The determination unit 1005 may, for example, evaluate the occurrence of criminal acts against the target person based on the first and second features generated. Specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the change pattern of the first and second features generated.

[0189] The determination unit 1005 more specifically calculates the change in the generated first feature. The change is, for example, the difference between the first feature generated for each of two different speech segments. Specifically, the change is the difference between the first feature generated for the first speech segment and the first feature generated for the last speech segment during the target period.

[0190] The determination unit 1005 more specifically calculates the change in the generated second feature. The change is, for example, the difference between the second feature generated for each of two different speech segments. Specifically, the change is the difference between the second feature generated for the first speech segment and the second feature generated for the last speech segment during the target period.

[0191] More specifically, the determination unit 1005 determines that a criminal act has occurred against the target person if the change in the generated first feature is greater than or equal to a threshold, and the change in the generated second feature is greater than or equal to a threshold. More specifically, the determination unit 1005 determines that a criminal act has not occurred against the target person if the change in the generated first feature is less than a threshold, or the change in the generated second feature is less than a threshold.

[0192] More specifically, the determination unit 1005 may evaluate the occurrence of a criminal act against the target person by comparing the above-described third correspondence information with the combination of the generated first feature change pattern and the generated second feature change pattern. More specifically, the determination unit 1005 determines whether the combination of the generated first feature change pattern and the generated second feature change pattern matches a specific change pattern set in the third correspondence information, which has the solution "criminal act occurred".

[0193] More specifically, the determination unit 1005 determines that a criminal act has occurred against the target person if it determines that the combination matches a specific change pattern set in the third corresponding information which resolves to "the occurrence of a criminal act." More specifically, the determination unit 1005 determines that a criminal act has not occurred against the target person if it determines that the combination does not match a specific change pattern set in the third corresponding information which resolves to "the occurrence of a criminal act."

[0194] More specifically, the determination unit 1005 determines whether the combination of the generated first feature change pattern and the generated second feature change pattern matches a specific change pattern set in the third corresponding information, which has no criminal activity as the solution. More specifically, if the determination unit 1005 determines that the combination matches a specific change pattern set in the third corresponding information, which has no criminal activity as the solution, it may evaluate that no criminal activity has occurred against the person in question.

[0195] More specifically, the determination unit 1005 may evaluate the occurrence of criminal acts against the target person using the third model by inputting a combination of the generated change pattern of the first feature quantity and the generated change pattern of the second feature quantity into the third model described above. More specifically, the determination unit 1005 evaluates the occurrence of criminal acts against the target person based on the solution of the occurrence of criminal acts output by the third model in response to the input.

[0196] Specifically, the determination unit 1005 may compare the generated first feature quantity with a first threshold and compare the generated second feature quantity with a second threshold. The first threshold is set in advance by the user, for example. The second threshold is set in advance by the user, for example. Specifically, the determination unit 1005 evaluates that a criminal act has occurred against the target person if the first feature quantity is greater than or equal to the first threshold and the second feature quantity is greater than or equal to the second threshold. Specifically, the determination unit 1005 evaluates that a criminal act has not occurred against the target person if the first feature quantity is less than the threshold or the second feature quantity is less than the second threshold.

[0197] As a result, the determination unit 1005 can accurately evaluate the circumstances under which a criminal act has occurred. For example, the determination unit 1005 can accurately determine whether or not a criminal act has occurred. Specifically, the determination unit 1005 can utilize a first feature quantity based on partial biometric data from which noise superposition tends to be relatively small. Furthermore, the determination unit 1005 can utilize a second feature quantity based on partial audio data from which audio characteristics are easily read and which can be easily used to evaluate the circumstances under which a criminal act has occurred. Therefore, the determination unit 1005 can accurately evaluate the circumstances under which a criminal act has occurred against the target person.

[0198] The output unit 1006 outputs the processing result of at least one of the functional units. The output format can be, for example, display on a screen, print to a printer, transmit to an external device via the network interface 503, or store in a storage area such as the memory 502 or recording medium 505. This allows the output unit 1006 to notify the user of the processing result of at least one of the functional units, thereby improving the usability of the information processing device 100.

[0199] The output unit 1006 outputs, for example, the result of the determination unit 1005's evaluation of the occurrence of a criminal act. Specifically, the output unit 1006 outputs the result of the determination unit 1005's evaluation of the occurrence of a criminal act in a format that can be viewed by the person in question. Specifically, the output unit 1006 may output the result of the determination unit 1005's evaluation of the occurrence of a criminal act in a format that can be viewed by people related to the person in question. These people related to the person in question are, for example, close relatives. Specifically, the output unit 1006 may output the result of the determination unit 1005's evaluation of the occurrence of a criminal act in a format that can be viewed by a designated organization such as the police or a bank. More specifically, the output unit 1006 transmits the result of the determination unit 1005's evaluation of the occurrence of a criminal act to the alert output device 203. This makes it easier for the output unit 1006 to prevent criminal damage caused by criminal acts.

[0200] The output unit 1006 may, for example, output a notification or alert indicating that a criminal act has occurred when the determination unit 1005 evaluates that a criminal act has occurred. Specifically, the output unit 1006 outputs a notification or alert indicating that a criminal act has occurred so that the person concerned can refer to it. Specifically, the output unit 1006 may output a notification or alert indicating that a criminal act has occurred so that the person concerned can refer to it. The person concerned may, for example, be a close relative. Specifically, the output unit 1006 may output a notification or alert indicating that a criminal act has occurred so that it can refer to it by a designated organization such as the police or a bank. More specifically, the output unit 1006 transmits a notification or alert indicating that a criminal act has occurred to the alert output device 203. This makes it easier for the output unit 1006 to prevent criminal damage caused by criminal acts.

[0201] This explanation describes a case where the information processing device 100 evaluates the circumstances of a criminal act based on audio data, but it is not limited to this. For example, another computer may analyze the audio data to generate text data indicating the content of a conversation between the target person and another person, or analysis data indicating the voice characteristics of the target person and another person. In this case, the information processing device 100 receives, for example, text data indicating the content of a conversation between the target person and another person, or analysis data indicating the voice characteristics of the target person and another person, from the other computer. The information processing device 100 uses, for example, text data indicating the content of a conversation between the target person and another person, or analysis data indicating the voice characteristics of the target person and another person, in place of the audio data.

[0202] Here, we have described a case where the information processing device 100 includes an acquisition unit 1001, a classification unit 1002, a processing unit 1003, a calculation unit 1004, a determination unit 1005, and an output unit 1006, but it is not limited to this. For example, the information processing device 100 may not include any of the functional units. Specifically, the information processing device 100 may not include the processing unit 1003.

[0203] (Example of operation of the information processing device 100) Next, we will explain an example of the operation of the information processing device 100 using Figures 11 to 13.

[0204] Figures 11 to 13 are explanatory diagrams showing examples of the operation of the information processing device 100. In Figure 11, the information processing device 100 acquires audio data showing the conversation between the recipient and the caller via telephone during the call period from the start to the end of the call, when the recipient who may be a victim ends the call. The caller may be a fraudster. The information processing device 100 acquires biometric data showing the time change of the recipient's heart rate during the call period.

[0205] The information processing device 100 extracts the receiver's voice 1101 and the caller's voice 1102 based on the voice data. Based on the extracted receiver's voice 1101 and caller's voice 1102, the information processing device 100 classifies the call period into a listening section in which the receiver listened to the caller's speech and a speaking section in which the receiver spoke content other than acknowledgments.

[0206] The information processing device 100, for example, refers to the filter information management table 600 to obtain voice patterns corresponding to "silence" and voice patterns corresponding to "acknowledgments." Based on the receiver's voice 1101 and the caller's voice 1102, the information processing device 100 classifies the call period into silent periods and listening periods, based on the time when the caller is speaking and a voice pattern corresponding to "silence" appears in the receiver's voice 1101.

[0207] The information processing device 100, based on the receiver's voice 1101 and the caller's voice 1102, classifies the call period into "acknowledgment intervals" and "listening intervals" based on the intervals in which the caller is speaking and an acknowledgment-corresponding voice pattern appears in the receiver's voice 1101. The information processing device 100 then combines consecutive listening intervals. For example, based on the receiver's voice 1101, the information processing device 100 classifies the intervals other than the acknowledgment intervals in which the receiver speaks into "speech intervals." Next, we will move on to the explanation of Figure 12.

[0208] In Figure 12, the information processing device 100 generates an emotion value for each utterance segment based on the portion of the receiver's voice 1101 corresponding to that utterance segment. The emotion value is an emotional characteristic value. The emotion value can be, for example, negative or positive. The emotion value may also be, for example, "disappointed" or "enthusiastic". For example, the information processing device 100 generates an emotion value for each utterance segment based on the portion of the voice corresponding to that utterance segment, taking into account the tone of the receiver's voice, volume, keywords, etc.

[0209] In the example shown in Figure 12, the information processing device 100 specifically generates a negative "disappointed" emotion value for the receiver in the first utterance, taking into account the tone, volume, and keywords of the receiver's partial speech "Was it only until yesterday?" corresponding to the first utterance. The arrow indicates whether the tone at the end of the speech is rising or falling. Specifically, the information processing device 100 generates a positive "motivated" emotion value for the receiver in the second utterance, taking into account the tone, volume, and keywords of the receiver's speech "I understand. I'll be right there." corresponding to the second utterance.

[0210] The information processing device 100 generates a pattern of change in the recipient's emotional value across multiple speech segments. The information processing device 100 refers to the emotional pattern information management table 700 and determines, based on the generated change pattern, whether or not a fraudulent act by the caller against the recipient has occurred.

[0211] As a result, the information processing device 100 can accurately determine whether or not a fraudulent act by the sender has occurred against the recipient. The information processing device 100 can avoid generating the recipient's emotional value in sections that are relatively short in duration and where it is difficult to generate the recipient's emotional value, such as the nodding interval. The information processing device 100 can generate the recipient's emotional value in sections that are relatively long in duration and where it is easy to accurately generate the recipient's emotional value, such as the utterance interval. The information processing device 100 can accurately determine whether or not a fraudulent act by the sender has occurred against the recipient, for example, based on a relatively accurate pattern of change in the recipient's emotional value.

[0212] In Figure 13, the information processing device 100 refers to the filter information management table 600 to obtain a silence filter corresponding to "silence," and an acknowledgment filter corresponding to "acknowledgment," etc. A silence filter is, for example, a filter for removing noise based on respiration from biological data. An acknowledgment filter is, for example, a filter for removing noise based on body movement from biological data.

[0213] The information processing device 100 performs noise reduction processing by applying a silence filter to the partial biometric data corresponding to the silence intervals among the biometric data showing the time change of the receiver's heart rate during the call period. The information processing device 100 also performs noise reduction processing by applying an affirmation filter to the partial biometric data corresponding to the affirmation intervals among the biometric data showing the time change of the receiver's heart rate during the call period.

[0214] After performing noise reduction processing, the information processing device 100 calculates the amount of change from the heart rate at the start of each listening section to the heart rate at the end of each listening section, based on biometric data showing the time change of the recipient's heart rate during the call. If the calculated amount of change is greater than or equal to a threshold in at least one listening section, the information processing device 100 determines that a fraudulent act by the caller against the recipient has occurred. If the calculated amount of change is less than the threshold in any listening section, the information processing device 100 determines that a fraudulent act by the caller against the recipient has not occurred.

[0215] After performing noise reduction processing, the information processing device 100 may generate a change pattern for each listening section, from the heart rate at the start of the listening section to the heart rate at the end of the listening section, based on biometric data showing the time change of the recipient's heart rate during the call. The information processing device 100 may refer to the biometric pattern information management table 800 and determine whether or not a fraudulent act by the caller against the recipient has occurred based on the generated change pattern.

[0216] As a result, the information processing device 100 can accurately determine whether or not a fraudulent act by the sender has occurred against the recipient. The information processing device 100 can avoid generating heart rate change patterns in sections where noise superposition, such as body movement noise, tends to be relatively large, making it difficult to use in evaluating the occurrence of criminal activity, such as during speech. The information processing device 100 can generate heart rate change patterns in sections where noise superposition, such as body movement noise, tends to be relatively small, making it easy to use in evaluating the occurrence of criminal activity, such as during listening. Therefore, the information processing device 100 can accurately determine whether or not a fraudulent act by the sender has occurred against the recipient, for example, based on a heart rate change pattern with relatively small noise superposition, such as during body movement noise.

[0217] The information processing device 100 can appropriately perform noise reduction processing for each section corresponding to a voice pattern by using a filter corresponding to that voice pattern. For example, the information processing device 100 can use a filter that takes into account what type of noise tends to be included and what the magnitude of that noise tends to be, depending on the voice pattern. Therefore, the information processing device 100 can accurately remove noise from biometric data. Furthermore, the information processing device 100 can accurately generate heart rate change patterns and accurately determine whether or not fraudulent activity by the caller against the recipient has occurred.

[0218] (Overall processing procedure) Next, using Figure 14, an example of the overall processing procedure executed by the information processing device 100 will be described. The overall processing is realized, for example, by the CPU 501 shown in Figure 5, storage areas such as memory 502 and recording media 505, and network I / F 503.

[0219] Figure 14 is a flowchart showing an example of the overall processing procedure. The information processing device 100 detects the start of a call by the target person (step S1401).

[0220] Next, the information processing device 100 acquires the voice data of the target person during the target period at a predetermined timing (step S1402). The predetermined timing is, for example, the timing of the end of the call. The target period is, for example, the call period from the start to the end of the call. Then, the information processing device 100 acquires the biometric data of the target person during the target period at a predetermined timing (step S1403).

[0221] Next, the information processing device 100 analyzes the acquired audio data and classifies the target period into speaking intervals and listening intervals, and further classifies the listening intervals into silence intervals and response intervals (step S1404). Then, the information processing device 100 selects either the speaking intervals or the listening intervals as the target for processing, starting from the beginning of the target period (step S1405).

[0222] Next, the information processing device 100 determines whether the selected section is a listening section or not (step S1406). If the selected section is a listening section (step S1406: Yes), the information processing device 100 proceeds to step S1407. On the other hand, if the selected section is a speaking section and not a listening section (step S1406: No), the information processing device 100 proceeds to step S1410.

[0223] In step S1407, the information processing device 100 acquires the biometric data of the target person during the listening section from the biometric data of the target person during the target period (step S1407). Next, the information processing device 100 performs noise reduction processing on the biometric data of the target person during the listening section (step S1408).

[0224] The information processing device 100, for example, applies a filter corresponding to silence to the portion of the biometric data of the target person during the listening period that pertains to silence, thereby removing noise. Furthermore, the information processing device 100, for example, applies a filter corresponding to nodding to the portion of the biometric data of the target person during the listening period that pertains to nodding, thereby removing noise.

[0225] Then, the information processing device 100 calculates a first feature quantity of the biometric data of the target person in the acquired listening section (step S1409). After that, the information processing device 100 proceeds to the process in step S1412.

[0226] In step S1410, the information processing device 100 extracts the voice data of the target person from the voice data of the target person during the target period, specifically the voice data of the person in the speech interval (step S1410). Next, the information processing device 100 calculates a second feature quantity of the voice data of the target person in the extracted voice interval (step S1411). Then, the information processing device 100 proceeds to the processing in step S1412.

[0227] In step S1412, the information processing device 100 determines whether or not all sections have been selected as processing targets (step S1412). If no section has been selected (step S1412: No), the information processing device 100 returns to the process in step S1405. On the other hand, if all sections have been selected (step S1412: Yes), the information processing device 100 proceeds to the process in step S1413.

[0228] In step S1413, the information processing device 100 determines whether or not fraudulent activity has occurred based on the calculated change pattern of the first feature quantity and the calculated change pattern of the second feature quantity (step S1413). Then, the information processing device 100 terminates the entire process. As a result, the information processing device 100 can accurately determine whether or not fraudulent activity has occurred.

[0229] Here, we have described a case where the predetermined timing is the timing of the end of the call, but this is not the only case. For example, the predetermined timing may be a timing at regular intervals from the start of the call. In this case, the period in question may be the period from the predetermined timing to a certain time before. In this case, the information processing device 100 may return to the process of step S1402 after performing the process of step S1413.

[0230] Here, the information processing device 100 may perform some of the steps in Figure 14 in a different order. For example, the order of steps S1402 and S1403 can be changed. Also, the information processing device 100 may omit some of the steps in Figure 14. For example, steps S1408 and S1409 can be omitted.

[0231] As explained above, the information processing device 100 can acquire audio data showing the conversation of the target person during the target period. Based on the acquired audio data, the information processing device 100 can identify sections within the target period that match each of the one or more set audio patterns. For each identified section, the information processing device 100 can perform noise reduction processing on the biometric data of the target person in that section, corresponding to one of the one or more audio patterns that matched the section. Based on the biometric data of the target person after noise reduction processing in each identified section, the information processing device 100 can evaluate the occurrence of criminal acts against the target person. As a result, the information processing device 100 can appropriately remove noise from the biometric data of the target person and accurately evaluate the occurrence of criminal acts against the target person.

[0232] The information processing device 100 can have a storage unit that stores one or more voice patterns in association with a filter that removes noise from biometric data. The information processing device 100 can refer to the storage unit and, for each specified section, perform noise reduction processing on the biometric data of the target person in that section using a filter associated with one of the voice patterns that matches the section. In this way, the information processing device 100 can implement noise reduction processing using filters.

[0233] According to the information processing device 100, audio data showing the conversation of a target person during a specified period via telephone can be acquired. According to the information processing device 100, the occurrence of fraudulent activities against the target person via telephone can be evaluated based on the biometric data of the target person after noise reduction processing has been performed for each specified section. As a result, the information processing device 100 can make it easier to prevent fraud damage caused by fraudulent activities via telephone.

[0234] According to the information processing device 100, based on the biometric data of the target person after noise reduction processing corresponding to any voice pattern matching the specified interval, feature quantities related to the target person in each interval can be generated. According to the information processing device 100, the occurrence of criminal acts against the target person can be evaluated based on the change patterns of the generated feature quantities. As a result, the information processing device 100 can take into account the change patterns of the feature quantities and accurately evaluate the occurrence of criminal acts against the target person.

[0235] According to the information processing device 100, for each identified interval, noise reduction processing can be performed on biometric data representing the measured values ​​of the target person's heart rate or pulse in that interval, corresponding to any audio pattern that matches the interval. According to the information processing device 100, based on the biometric data after noise reduction processing for each identified interval, a feature quantity can be generated, which is the amount of change in the measured values ​​of the target person's heart rate or pulse in that interval. According to the information processing device 100, if the generated feature quantity is above a threshold, it can be evaluated that a criminal act against the target person has occurred. According to the information processing device 100, if the generated feature quantity is below the threshold, it can be evaluated that a criminal act against the target person has not occurred. As a result, the information processing device 100 can accurately evaluate whether or not a criminal act has occurred.

[0236] According to the information processing device 100, for each identified interval, noise reduction processing can be performed on biometric data representing the measured values ​​of the target person's heart rate or pulse in that interval, corresponding to any audio pattern that matches the interval. According to the information processing device 100, based on the biometric data after noise reduction processing for each identified interval, feature quantities, which are statistical values ​​of the measured values ​​of the target person's heart rate or pulse in that interval, can be generated. According to the information processing device 100, if the amount of change in the generated feature quantity is greater than or equal to a threshold, it can be evaluated that a criminal act against the target person has occurred. According to the information processing device 100, if the amount of change in the generated feature quantity is less than the threshold, it can be evaluated that a criminal act against the target person has not occurred. As a result, the information processing device 100 can accurately evaluate whether or not a criminal act has occurred.

[0237] The information processing method described in this embodiment can be implemented by executing a pre-prepared program on a computer such as a PC or workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium can be a hard disk, flexible disk, CD (Compact Disc)-ROM, MO (Magneto Optical Disc), DVD (Digital Versatile Disc), etc. Furthermore, the information processing program described in this embodiment may be distributed via a network such as the Internet.

[0238] With regard to the embodiments described above, the following additional information is disclosed.

[0239] (Note 1) Obtain audio data showing the conversations of the subject person over a certain period of time. Based on the acquired audio data, the intervals within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing program characterized by having a computer perform the processing.

[0240] (Note 2) The process to be carried out above is: The information processing program according to Appendix 1, characterized in that it refers to a storage unit that stores each of the one or more aforementioned voice patterns in association with a filter for removing noise from biometric data, and for each of the identified sections, performs noise removal processing on the biometric data of the target person in the section using a filter associated with one of the one or more aforementioned voice patterns that matches the section.

[0241] (Note 3) The audio data indicates the conversation of the person in question during the aforementioned period via telephone. The evaluation process described above is: An information processing program according to Appendix 1 or 2, characterized in that it evaluates the occurrence of fraudulent acts against the target person by another person via telephone, based on the biometric data of the target person after performing noise reduction processing corresponding to any voice pattern that matches the specified section in each of the specified sections.

[0242] (Note 4) Based on the biometric data of the target person after performing noise reduction processing corresponding to any of the voice patterns that match the identified interval, feature quantities relating to the target person in the interval are generated. The evaluation process described above is: An information processing program according to Appendix 1 or 2, characterized in that it evaluates the occurrence of criminal acts against the target person based on the change pattern of the generated feature quantities.

[0243] (Note 5) The process to be carried out above is: For each of the identified intervals, noise reduction processing is performed on the biometric data showing the measured values ​​of the heart rate or pulse of the subject person in that interval, corresponding to one of the one or more audio patterns that matches the interval. The aforementioned generation process is, Based on biometric data showing the measured values ​​of the target person's heart rate or pulse after applying noise reduction processing corresponding to any of the audio patterns that match the identified interval, a feature quantity is generated which is the amount of change in the measured values ​​of the target person's heart rate or pulse in the interval. The evaluation process described above is: The information processing program described in Appendix 4, characterized in that if the generated feature quantity is greater than or equal to a threshold, it is evaluated that a criminal act against the target person has occurred, and if the generated feature quantity is less than a threshold, it is evaluated that a criminal act against the target person has not occurred.

[0244] (Note 6) The process to be carried out above is: For each of the identified intervals, noise reduction processing is performed on the biometric data showing the measured values ​​of the heart rate or pulse of the subject person in that interval, corresponding to one of the one or more audio patterns that matches the interval. The aforementioned generation process is, Based on biometric data showing the measured values ​​of the target person's heart rate or pulse after applying noise reduction processing corresponding to any of the audio patterns that match the identified interval, a feature quantity is generated which is a statistical value of the measured values ​​of the target person's heart rate or pulse in the interval. The evaluation process described above is: The information processing program described in Appendix 4, characterized in that if the amount of change in the generated feature is greater than or equal to a threshold, it is evaluated that a criminal act against the target person has occurred, and if the amount of change in the generated feature is less than a threshold, it is evaluated that a criminal act against the target person has not occurred.

[0245] (Note 7) Obtain audio data showing the conversation of the subject person over a certain period of time, Based on the acquired audio data, the intervals within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing method characterized in that the processing is performed by a computer.

[0246] (Note 8) Obtain audio data showing the conversation of the subject person over a certain period of time, Based on the acquired audio data, the intervals within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing device characterized by having a control unit. [Explanation of Symbols]

[0247] 100 Information Processing Devices 101 Audio Data 102 Biometric data 200 Information Processing Systems 201 Audio data acquisition device 202 Biometric Data Acquisition Device 203 Alert Output Device 210 Network 301,401 users 302,402 Scammer 310,414 smartphones 311,907 Biosensors 403 Related parties 411 Wearable devices 412 Smart Speakers 413 Servers 415 Landline telephone 500,900 buses 501,901 CPU 502,902 memory 503,903 Network I / F 504,904 Recording medium I / F 505,905 recording media 600 Filter Information Management Table 700 Emotion Pattern Information Management Table 800 Biological Pattern Information Management Table 906 Voice Sensor 908 Output device 1000 storage section 1001 Acquisition Department 1002 Classification section 1003 Processing Department 1004 Calculation Unit 1005 Judgment section 1006 Output section 1101, 1102 Audio

Claims

1. By acquiring audio data that shows the conversations of a target person over a certain period, Based on the acquired audio data, the sections within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing program characterized by having a computer perform the processing.

2. The process to be carried out as described above is: The information processing program according to claim 1, characterized in that it refers to a storage unit that stores each of the one or more voice patterns in association with a filter for removing noise from biometric data, and for each of the identified sections, performs noise removal processing on the biometric data of the target person in the section using a filter associated with one of the one or more voice patterns that matches the section.

3. The aforementioned audio data shows the conversation of the person in question during the aforementioned period via telephone. The evaluation process described above is: The information processing program according to claim 1 or 2, characterized in that it evaluates the occurrence of fraudulent acts against the target person by another person via telephone, based on the biometric data of the target person after performing noise reduction processing corresponding to any voice pattern that matches the specified section in each of the specified sections.

4. By acquiring audio data that shows the conversations of a target person over a certain period, Based on the acquired audio data, the sections within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing method characterized in that the processing is performed by a computer.

5. By acquiring audio data that shows the conversations of a target person over a certain period, Based on the acquired audio data, the sections within the period that match each of the set one or more audio patterns are identified. For each of the identified intervals, noise reduction processing is performed on the biometric data of the target person in that interval, corresponding to one of the one or more voice patterns that matches the interval. Based on the biometric data of the target person after applying noise reduction processing corresponding to any of the audio patterns that match the identified section, the occurrence of criminal acts against the target person is evaluated. An information processing device characterized by having a control unit.

Images