Support device, system, support method, and support program

Abstract

An assistance device (10) comprises: a personal information extraction unit (11) that extracts, on the basis of account information acquired from a target account in virtual space, personal information by which a target user, who is the holder of the target account, can be identified; a position information extraction unit (12) that extracts position information pertaining to the target user on the basis of the account information; and an output unit (13) that outputs the extracted personal information and the extracted position information as assistance information for assisting in crime prevention in the vicinity of the position information of a physical space.

Description

[Technical field]

[0001] The present invention relates to an assisting device, a system, an assisting method and a non-transitory computer readable medium. [Background technology]

[0002] In recent years, internet services such as social media have become widespread and are used all over the world. However, due to their convenience and high anonymity, crimes using cyberspace are on the rise, and it is desirable to prevent such crimes before they occur. For example, Patent Document 1 is known as a related technology. Patent Document 1 describes how a gate facility at a public facility ensures safety from crime by comparing a person passing through the gate with a person on a list of suspicious people. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-167931 A Summary of the Invention [Problem to be solved by the invention]

[0004] According to related technologies such as Patent Document 1, it is possible to monitor suspicious people in physical space (real space) by using a prepared list of suspicious people. However, the related technologies do not take into consideration crimes that utilize cyberspace, and it is difficult to efficiently monitor and investigate in physical space using information in cyberspace.

[0005] In view of such problems, an object of the present disclosure is to provide an assistance device, a system, an assistance method, and a non-transitory computer-readable medium that enable efficient monitoring and investigation. [Means for solving the problem]

[0006] The support device disclosed herein comprises a personal information extraction means that extracts personal information capable of identifying a target user who holds a target account based on account information obtained from the target account in cyberspace, a location information extraction means that extracts location information related to the target user based on the account information, and an output means that outputs the extracted personal information and the extracted location information as support information that supports crime prevention in the vicinity of the location information in physical space.

[0007] The system disclosed herein comprises a plurality of monitoring systems that monitor different locations, and a support device, the support device comprising: a personal information extraction means that extracts personal information capable of identifying a target user who holds a target account based on account information obtained from the target account in cyberspace; a location information extraction means that extracts location information related to the target user based on the account information; and an output means that outputs the extracted personal information to the monitoring system selected based on the extracted location information.

[0008] The support method disclosed herein extracts personal information capable of identifying a target user who holds a target account based on account information obtained from the target account in cyberspace, extracts location information related to the target user based on the account information, and outputs the extracted personal information and the extracted location information as support information that supports crime prevention in the vicinity of the location information in physical space.

[0009] The non-transitory computer-readable medium of the present disclosure is a non-transitory computer-readable medium having stored therein an assistance program for causing a computer to execute a process of extracting personal information capable of identifying a target user who holds a target account based on account information obtained from the target account in cyberspace, extracting location information related to the target user based on the account information, and outputting the extracted personal information and the extracted location information as assistance information for assisting in crime prevention in the vicinity of the location information in physical space. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide an assistance device, a system, an assistance method, and a non-transitory computer-readable medium that enable efficient monitoring and investigation. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram showing an overview of a support device according to an embodiment; [Diagram 2] FIG. 1 is a configuration diagram showing a configuration example of a cyber-physical integrated monitoring system according to a first embodiment. [Diagram 3] 1 is a configuration diagram showing a configuration example of a monitoring support device according to a first embodiment; [Figure 4] 1 is a configuration diagram showing a configuration example of a monitoring system according to a first embodiment. [Diagram 5] 4 is a flowchart showing an operation example of the monitoring support device according to the first embodiment. [Figure 6] 13 is a flowchart showing an operation example of another account identification processing according to the second embodiment. [Figure 7] 13 is a flowchart showing an operation example of another account identification processing according to the third embodiment. [Figure 8] 13 is a flowchart showing an operation example of an account information aggregation process according to the fourth embodiment. [Figure 9] FIG. 13 is a configuration diagram showing a configuration example of an image position specifying unit according to the fifth embodiment. [Figure 10] FIG. 13 is a configuration diagram showing an example of the configuration of a discriminator according to a fifth embodiment. [Figure 11] 13 is a flowchart showing an operation example of a training process according to the fifth embodiment. [Figure 12] 23 is a flowchart showing an operation example of an activity area estimation process according to the sixth embodiment. [Figure 13] 13 is a flowchart showing an operation example of an activity area estimation process according to the seventh embodiment. [Figure 14] FIG. 2 is a configuration diagram showing an overview of the hardware of a computer according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment will be described with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and repeated explanations will be omitted as necessary.

[0013] (Considerations leading to the embodiment) In recent years, due to the convenience and anonymity of the Internet and social media, the focus of various crimes (planning, preparation, etc.) has shifted to cyberspace. For example, it is said that 90% of terrorist attacks and 70% of drug trafficking utilize social media.

[0014] One way to prevent such crimes is to register the facial photograph of the target person on a watchlist and detect the registered person through surveillance camera footage. However, as the methods of various crimes have become more complex, it is becoming difficult to prevent crimes through simple video surveillance that focuses on checking watchlists. For example, it is difficult to prevent crimes such as home-grown terrorism that sympathizes with extremist ideology through the Internet. In particular, it is not possible to detect first-time offenders who do not have a pre-registered facial photograph.

[0015] It is also possible to use video behavior analysis to detect suspicious behavior (loitering, leaving luggage, etc.) from surveillance camera footage without using a watchlist. However, with this method, it is difficult to define suspicious behavior, and there is a risk that many behaviors that are actually unrelated to crimes will be falsely detected, making it difficult to prevent crimes.

[0016] Therefore, in the following embodiments, by integrating and utilizing information from cyberspace and physical space, it is possible to identify the target of a crime in cyberspace (such as a crime warning) before it is transferred to physical space, thereby preventing the occurrence or spread of damage.

[0017] (Outline of the embodiment) Fig. 1 shows an overview of a support device according to an embodiment. The support device 10 according to the embodiment can be applied to, for example, investigation and security support for law enforcement agencies, and monitoring support for important facilities. As shown in Fig. 1, the support device 10 includes a personal information extraction unit 11, a location information extraction unit 12, and an output unit 13.

[0018] The personal information extraction unit 11 extracts personal information capable of identifying a target user (also called a target person) who holds a target account based on account information acquired from the target account in cyberspace. The location information extraction unit 12 extracts location information related to the target user based on the account information acquired from the target account. The account information acquired from the target account may include account information of the target account and account information of related accounts related to the target account.

[0019] The output unit 13 outputs the personal information extracted by the personal information extraction unit 11 and the location information extracted by the location information extraction unit 12 as support information for supporting crime prevention in the vicinity of the location information in the physical space. For example, the support information may be information for supporting the monitoring or investigation of the target user. When providing monitoring support, the output unit 13 may output the extracted personal information as information on the person being monitored to a monitoring system selected based on the extracted location information. When providing investigative support, the output unit 13 may output the extracted personal information as information on the person being investigated to an investigative agency investigating the vicinity of the extracted location information.

[0020] In this manner, in the embodiment, the personal information and location information of the target user who holds the target account are extracted based on the account information related to the target account, and this information is output to support crime prevention in the physical space. This makes it possible to efficiently monitor and investigate the person with the specified personal information in the vicinity of the location specified based on the information in cyberspace, and effectively prevent crimes that utilize cyberspace.

[0021] (Embodiment 1) Hereinafter, the first embodiment will be described with reference to the drawings. Fig. 2 shows a configuration example of a cyber-physical integrated monitoring system according to the present embodiment, Fig. 3 shows a configuration example of a monitoring support device in Fig. 2, and Fig. 4 shows a configuration example of the monitoring system in Fig. 2. Note that the configuration of each device is merely an example, and other configurations may be used as long as the operation (method) described below is possible. For example, part of the monitoring system may be included in the monitoring support device, or part of the monitoring support device may be included in the monitoring system.

[0022] The cyber-physical integrated monitoring system 1 is a system that monitors a target person in physical space based on information of the target account in cyberspace. In this embodiment, personal information of the target person who holds the target account and location information of the target person are acquired from account information such as posted information related to the target account in cyberspace, and the personal information of the target person is registered in a watch list of a monitoring system deployed in the vicinity of the acquired location information. Note that the system is not limited to monitoring of the target person, and the personal information and location information (support information) of the target person may be provided to a system (organization) for investigating the target person or for preventing other crimes.

[0023] 2, the cyber-physical integrated monitoring system 1 includes a monitoring support device 100, a plurality of monitoring systems 200, and a social media system 300. The monitoring support device 100 and the plurality of monitoring systems 200, and the monitoring support device 100 and the social media system 300 are connected to each other so as to be able to communicate with each other via the Internet or the like.

[0024] The social media system 300 is a system that provides social media services (cyber services) such as SNS (Social Networking Service) in cyberspace. The social media system 300 may include multiple social media services. The social media service is an online service that allows multiple accounts (users) to transmit (disclose) information and communicate with each other on the Internet (online). The social media service is not limited to SNS, but also includes messaging services such as chat, blogs, electronic bulletin boards (forum sites), video sharing sites, information sharing sites, social games, social bookmarks, and the like. For example, the social media system 300 includes a server and a user terminal on the cloud. The user terminal logs in with a user's account via an API (Application Programming Interface) provided by the server, inputs and views posts to the timeline and chat conversations, and also registers account connections such as friend relationships and follow relationships.

[0025] The monitoring support device 100 is a device that supports monitoring of the monitoring system 200 based on information from the social media system 300. As shown in Fig. 3, the monitoring support device 100 includes a social media information acquisition unit 101, an account identification unit 102, an account information extraction unit 103, a personal information extraction unit 104, a location information extraction unit 105, a monitoring system selection unit 106, a personal information output unit 107, and a storage unit 108.

[0026] The storage unit 108 stores information (data) necessary for the operation (processing) of the monitoring assistance device 100. The storage unit 108 is, for example, a non-volatile memory such as a flash memory, a hard disk drive, etc. The storage unit 108 stores a monitoring system list that associates multiple monitoring systems 200 (monitoring devices) with their monitoring areas (monitoring positions).

[0027] The social media information acquisition unit 101 acquires (collects) social media information from the social media system 300. The social media information is account information made public about each social media account. The account information includes account profile information and posted information (posted images, posted videos, posted text, posted audio, etc.).

[0028] The social media information acquisition unit 101 acquires all social media information that can be acquired from the social media system 300. The social media information acquisition unit 101 may acquire social media information for a plurality of social media. The social media information acquisition unit 101 may acquire the information from a server that provides a social media service via an API (acquisition tool), or may acquire the information from a database in which social media information is stored in advance.

[0029] The account identification unit 102 identifies an account from which personal information and location information are extracted. The account identification unit 102 identifies a target account (an account for extracting information of a target person) to be monitored, and also identifies related accounts related to the target account. The related accounts are accounts connected to the target account in a social media service in cyberspace. The related accounts include friend accounts with which a friend relationship is registered, and accounts with connections such as a follow relationship (follow or follower), connections through posts (comments on posts, quotations such as retweets, and reactions such as "likes"), connections through conversations (conversations in the same community), and connections through a history (footprints) of viewing account information including the profile and posted information of each account. In addition, the account identification unit 102 identifies, as related accounts, other accounts that are owned by the same user as the target account through an account matching process. That is, the account identification unit 102 is a target account identification unit that identifies the target account, and is also a different account identification unit (related account identification unit) that identifies another account (related account). For example, the different account identification unit identifies another account based on the account information of the target account and the account information of the related account.

[0030] The account information extraction unit 103 extracts account information related to the target account from the social media information collected by the social media information acquisition unit 101. The account information extraction unit 103 extracts account information of the specified target account as account information related to the target account, and also extracts account information of specified related accounts (friend accounts and other accounts).

[0031] The personal information extraction unit 104 extracts personal information of the target user (target person) based on the account information related to the extracted target account. The personal information extraction unit 104 extracts personal information of the target user who owns the target account from the profile information and posted information included in the account information by text analysis, image analysis technology, voice analysis technology, etc. The personal information is information that can identify the target user in the physical space. The personal information is, for example, biological information such as a face image, fingerprint information, and voiceprint information, but is not limited to this, and may include soft biometric information such as tattoos, belongings, name (account name, identification ID, etc.), age, gender, and other attribute information. The personal information is preferably information used to identify a person in the monitoring system 200 (monitoring and investigation in the physical space), but may also include other information.

[0032] The location information extraction unit 105 extracts location information of the target user based on the account information related to the extracted target account. The extracted location information includes an activity base such as a residence (residential area) extracted from the account information, a posting location where the posted information was posted, information that can be extracted from the posted information (GPS (Global Positioning System) information, place names, landmarks in images, etc.), and an activity area (activity range) of the target user estimated from them. The extracted location information is not limited to the current location or daily activity area of ​​the target user, but may also be a location mentioned in the posted text (location of a crime warning). The location mentioned in the posted text is extracted, for example, by natural language processing of the posted text. In this example, the location information extraction unit 105 includes an image location identification unit 110 and an activity area estimation unit 120. The image location identification unit 110 identifies a visiting location (posting location) of the target user from the reflection of the posted image, etc. The activity area estimation unit 120 estimates the activity area of ​​the target user based on a location identified from information of the target account and related accounts (including friend accounts).

[0033] The monitoring system selection unit 106 selects an appropriate monitoring system 200 from among the multiple monitoring systems 200 based on the extracted location information of the target user. The monitoring system selection unit 106 refers to the monitoring system list stored in the storage unit 108 and selects a monitoring system 200 that monitors the activity area (location information) of the target user. The monitoring system selection unit 106 selects a monitoring system 200 whose monitoring area includes the activity area of ​​the target user (the monitoring area overlaps part or all of the activity area). A monitoring system 200 whose monitoring area is a predetermined range of a location (the periphery of the activity area) from the activity area may be selected. In addition, if there are multiple applicable monitoring systems 200, multiple monitoring systems 200 may be selected. The personal information output unit 107 outputs the extracted personal information of the target user to the selected monitoring system 200.

[0034] The monitoring system 200 is installed in a public facility or the like, and monitors people in a monitoring area. For example, the multiple monitoring systems 200 monitor different locations (areas), but the monitoring areas may partially overlap. As shown in Fig. 4, the monitoring system 200 includes a monitoring device 201, a monitored person information extraction unit 202, a monitored person information matching unit 203, a matching result output unit 204, a watch list storage unit 205, and a watch list creation unit 206.

[0035] The monitoring device 201 is a detection device that detects information on a person to be monitored in a monitoring area. For example, the monitoring device 201 is a biometric information sensor that identifies biometric information, a monitoring camera, etc. The monitoring device 201 may be a monitoring camera or a microphone installed at the entrance or passage of a public facility, or a fingerprint sensor installed at an entrance / exit gate.

[0036] The monitored person information extraction unit 202 extracts personal information of the monitored person from the information detected by the monitoring device 201. For example, if the monitoring device 201 is a camera, the monitored person information extraction unit 202 extracts the face image and fingerprint of the person from the image captured by the camera, if the monitoring device 201 is a fingerprint sensor, the monitored person information extraction unit 202 acquires the fingerprint information of the person from the fingerprint sensor, and if the monitoring device 201 is a microphone, the monitored person information extraction unit 202 extracts the voiceprint information of the person from the voice picked up by the microphone. Other information such as soft biometric information, belongings, name, and attribute information may be extracted by analyzing the image of the camera, for example.

[0037] The watchlist storage unit 205 is a database that stores a watchlist, which is a list of people to be monitored. For example, the watchlist is a face database that stores face images, a fingerprint database that stores fingerprint information, a voiceprint database that stores voiceprint information, etc. The watchlist creation unit (registration unit) 206 registers personal information output from the monitoring support device 100 in the watchlist. That is, the watchlist creation unit 206 registers biometric information such as face images, fingerprint information, and voiceprint information of the target user, soft biometric information, belongings, name, and attribute information in the watchlist. When registering personal information (new personal information) of the target user, it may be added to an existing watchlist, or may be registered in a different watchlist (such as a watchlist that is different from a wanted criminal list).

[0038] The monitored person information matching unit 203 compares and matches the personal information of the monitored person extracted from the monitoring device 201 with the personal information of the watch list stored in the watch list storage unit 205. The matching result output unit 204 outputs the matching result of the monitored person's personal information and the personal information of the watch list to the monitor. When the matching result output unit 204 matches the personal information of the monitored person with the personal information of the watch list, it outputs an alert by displaying or sounding. The matching of the personal information may be determined, for example, by whether or not the similarity of the features extracted from each piece of information is greater than a predetermined threshold. In addition, when the personal information of the target user is registered in another watch list, an alert other than the existing alert may be output for the matching result of the other watch list. When the personal information includes multiple pieces of information (biometric information, soft biometric information, belongings, name, attribute information, etc.), the degree of matching (similarity) of each piece of information or a score obtained by adding up each degree of matching may be output. In addition, information included in the personal information that the monitoring device 201 cannot detect may be displayed as reference information.

[0039] Fig. 5 shows an example of the operation (monitoring support method) of the monitoring support device according to the present embodiment. As shown in Fig. 5, first, the monitoring support device 100 acquires social media information from the social media system 300 (S101). The social media information acquisition unit 101 accesses the server and database of the social media system 300 and acquires social media information of all accounts that are publicly available and can be acquired. For example, the social media information is acquired to the extent possible using the API (acquisition tool) of the social media service.

[0040] Next, the monitoring support device 100 identifies a target account to be monitored (S102). The account identification unit 102 may accept input of information related to the target account and identify the target account based on the input information. For example, a user of the system may prepare a list of targets who are likely to be involved in crimes based on information on the Internet and input information of the target account in the target list. The account may be identified by inputting the account ID (identification information) of the target account, or the account may be identified by searching social media information from the input name or the like. In addition, the account identification unit 102 may identify the target account from a predetermined keyword related to a crime such as a crime warning. For example, a list of predetermined keywords may be input or registered in the storage unit 108, and the target account may be identified by searching social media information from the keyword.

[0041] Next, the monitoring support device 100 identifies another account of the target account (S103). For example, the account identification unit 102 identifies related accounts related to the target account. The account identification unit 102 may use a social graph, which is data representing connections between users, to identify related accounts related to each account and obtain account information of the identified related accounts. For example, the related accounts may be accounts that have friendships with the target account, such as friends, follows, or followers, accounts that have posted information that quotes the posted information of the target account, accounts that have a history of giving "likes" or the like to the posted information of the target account, and accounts that have a history of viewing account information including the profile and posted information of the target account. Here, in particular, other accounts owned by the same user as the target account are identified. Based on the information of the target account, the account identification unit 102 searches for information on related accounts that are connected to the target account from social media information and extracts accounts that are likely to be owned by the same user. For example, the account identification unit 102 may calculate the similarity (similarity score) between the account information of the target account and the account information of the extracted related accounts, and determine the account information of the same user as the target account based on the calculated similarity.

[0042] Next, the monitoring support device 100 aggregates the account information of the identified accounts (S104). The account information extraction unit 103 extracts the account information of the identified target account and the account information of other accounts from the acquired social media information, and aggregates the extracted information. For example, when the account ID of an account is identified, the account information extraction unit 103 extracts and aggregates the profile information and posted information of the account associated with the account ID. Note that the account information is not limited to other accounts, and may be extracted as necessary of other related accounts.

[0043] Following S104, the monitoring support device 100 extracts personal information of the target user based on the aggregated account information (S105). The personal information extraction unit 104 extracts personal information of the target user based on the extracted and aggregated account information of the target account and other accounts. For example, the profile information in the account information includes text showing the profile of the account (user) and an image of the account, and the personal information extraction unit 104 performs text analysis and image analysis of these to extract attribute information such as the face image, name, age, and gender of the target user. In addition, the posted information includes text, images, videos, and audio posted by the account (user) to a timeline, etc., and the personal information extraction unit 104 performs text analysis, image analysis, and audio analysis of these to extract the above information as well as the target user's fingerprints, voiceprints, other soft biometric information, belongings, etc.

[0044] In addition, following S104, in S106 and S107, the monitoring support device 100 extracts the location information of the target user based on the aggregated account information. For example, the location information extraction unit 105 may acquire location information from the residence, hometown, etc. of the profile information included in the extracted and aggregated account information. In addition, the location information extraction unit 105 may acquire location information from words that can identify a location among the posted information included in the account information. Furthermore, when information that can identify the current location of the poster, called a GEO tag, is added to the posted information included in the account information, the location information extraction unit 105 may acquire location information from the GEO tag. Furthermore, the location information extraction unit 105 may acquire location information using geolocation. Furthermore, when using either the posted information or geolocation, the location information extraction unit 105 may use the location information that has been acquired the most frequently among the acquired location information.

[0045] Here, the location information of the target user is extracted by an image location specification process (S106) and an activity area estimation process (S107). In the image location specification process (S106), the image location specification unit 110 specifies a visited location (posted location) from the appearance of the posted image or video (acquired image) included in the aggregated account information. The appearance is, for example, an object related to a location, such as a building, a sign, or a road, that appears in the image. The image location specification unit 110 refers to an image database with location information (position image) associated with the location information, and compares the posted image with each position image in the image database with location information. The image database with location information may be stored in the storage unit 108, or may be an external database. For example, an object that appears in the posted image may be extracted by image analysis, and the object that appears may be compared with each position image in the image database with location information. Based on the result of this comparison, the image location specification unit 110 specifies the shooting location of the posted image from the position information associated with the matching position image.

[0046] In addition, the amount of images in the location information-attached image database may be enormous. For this reason, the search range of the location information-attached image database may be narrowed down based on account information, etc. That is, among the location images in the location information-attached image database, the location images related to the target account may be matched with the posted image (obtained image). For example, among the location images in the location information-attached image database, the location images corresponding to activity base information such as the residential area (e.g., Tokyo, Kawasaki City, Kanagawa Prefecture) described in the profile of the target account, or the location images corresponding to activity base information such as the residential area described in the profile of a related account (friend account) connected to the target account may be used as the matching target. This can improve the matching accuracy and search speed.

[0047] In the activity area estimation process (S107), the activity area estimation unit 120 estimates the activity area of ​​the target user from various pieces of location information extracted from the aggregated account information (including friend accounts). The activity area estimation unit 120 estimates the activity area from multiple pieces of location information including the location information extracted by the image location specification process. For example, the activity area estimation unit 120 extracts the target user's activity base, such as the residence, and visited places from the account information of the target account (including another account) and the friend account (related account), extracts the friend user's activity base, such as the residence, and visited places from the account information of the friend account, and defines the area including these places as the activity area.

[0048] The processes may be performed in the order of S106 and S107, or in the order of S107 and S106. That is, the location information extraction unit 105 may identify the target user's visited location from the appearance of the posted image in the aggregated account information (S106), estimate the target user's activity area from various location information including the friend account (including the location identified in S106) (S107), and extract the target user's activity area (location information). The location information extraction unit 105 may also estimate the target user's activity area from various location information of the aggregated account information (including the friend account) (S107), identify the target user's visited location from the appearance of the posted image within the range of the estimated activity area (S106), and extract the target user's activity area.

[0049] Next, the monitoring assistance device 100 selects a monitoring system 200 based on the location information of the target user extracted in S106 and S107 (S108). The monitoring system selection unit 106 refers to the monitoring system list stored in the storage unit 108 and selects a monitoring system whose monitoring area includes the activity area of ​​the target user (periphery of the activity area).

[0050] The monitoring system selection unit 106 may select a monitoring system 200 for a public facility such as a railroad or an airport in the vicinity of the target user's location information. The monitoring system selection unit 106 may, for example, calculate the degree of congestion (people or vehicles) of a place or facility, and select the monitoring system 200 based on the calculated degree of congestion. For example, the degree of congestion is calculated using the number of people or the number of vehicles. The monitoring system selection unit 106 may select a place or facility that is currently or usually congested, or is expected to be congested in the future, among the places or facilities in the vicinity of the target user's location information. This makes it possible to monitor places that may be soft targets. In addition, the monitoring system selection unit 106 may select a monitoring system 200 for a public transportation facility such as a railroad or a bus that may be a route of travel for the target user, based on the target user's location information.

[0051] Furthermore, when there are multiple candidates for location information of the target user, the monitoring system selection unit 106 may select multiple monitoring systems 200 in the vicinity of the multiple pieces of location information. For example, the monitoring system selection unit 106 may set a score indicating the possibility that the target user is located to the candidates for location information of the target user, and select the monitoring system 200 based on the set score. The score is set based on, for example, the number of visits or frequency of visits of the target account or friend accounts, the distance between locations, the weight of friend relationships, and the like. The monitoring system selection unit 106 may select the monitoring systems 200 in the vicinity of location information of only the top N candidates with the set score.

[0052] Following S105 and S108, the monitoring assistance device 100 outputs the personal information of the target user (S109). The personal information output unit 107 outputs the personal information of the target user extracted in S105 to the monitoring system 200 selected in S108. As a result, the extracted personal information of the target user is registered in the watch list of the monitoring system 200 deployed around the activity area of ​​the target user. Note that the personal information output unit 107 may output the personal information and location information of the target user to all monitoring systems 200. In this case, the monitoring system 200 compares the received location information of the target user with the monitoring area of ​​its own system, and if they match, registers the received personal information of the target user in the watch area.

[0053] As described above, in this embodiment, the monitoring support device extracts personal information and location information of the target user from the account information related to the target account, and registers the extracted personal information in a watch list of a monitoring system deployed in the vicinity of the extracted location information. This makes it possible to identify the location information of a target person involved in a crime using cyberspace, and to monitor locations where the target person is likely to be located. This makes it possible to efficiently monitor the target person, and effectively detect the target person before he or she commits a crime in physical space.

[0054] In general, it is difficult to obtain location information of a person, and law enforcement agencies in particular have difficulty identifying the location of a person involved in a crime using cyberspace. In this embodiment, it is possible to reliably obtain location information of a target user by using an account matching technology that identifies another account of the target user, an image location identification technology that identifies the target user's visited locations from the appearance of the posted image, and an activity area estimation technology that estimates the target user's activity range by utilizing information from friend users.

[0055] (Embodiment 2) Next, a second embodiment will be described with reference to the drawings. In this embodiment, an example of the different account identification process (S103 in FIG. 5) in the first embodiment will be described. Note that the configuration of the monitoring support device 100 and other processes are the same as those in the first embodiment.

[0056] FIG. 6 shows an example of another account identification process according to the present embodiment. Here, an example will be described in which it is determined whether two accounts to be determined (referred to as determination accounts) are accounts owned by the same user. That is, the two accounts that are ultimately determined to be accounts owned by the same user correspond to the target account and another account identified in the first embodiment. Note that the following process is mainly executed by the account identification unit 102 of the monitoring support device 100, but may be executed by other units as necessary. In this example, the account identification unit 102 identifies another account based on location information acquired from the account information of the related account, and in particular, identifies hierarchical location information in which the acquired location information is hierarchical according to the granularity level of the location, and identifies the other account based on the identified hierarchical location information.

[0057] 6, first, the account identifying unit 102 acquires information on related accounts related to two determined accounts (S201). The account identifying unit 102 identifies two determined accounts from the collected social media information, and acquires account information on the related accounts related to the two determined accounts. As in the first embodiment, the account identifying unit 102 may identify related accounts connected to each determined account, and acquire account information on the identified related accounts.

[0058] Next, the account identification unit 102 acquires location information associated with each related account (S202). The location information of the related account may be acquired in the same manner as the location information extraction unit 105 of the first embodiment. For example, the account identification unit 102 may acquire location information from the residence, birthplace, etc. of profile information included in the account information of the related account, or may acquire location information from images, text, etc. of posted information included in the account information of the related account.

[0059] Next, the account identifying unit 102 identifies hierarchical location information of each related account based on the location information of each related account (S203). The account identifying unit 102 identifies hierarchical location information indicating location information hierarchicalized according to the granularity level of the location based on the acquired location information of the related account. Furthermore, the account identifying unit 102 generates a hierarchical location information table in which the hierarchical location information of each related account is set for each determination account.

[0060] The granularity level may be, for example, a level corresponding to a country or an administrative district. For example, when three levels of granularity are defined, the lowest level of granularity may be a country level, the second lowest level of granularity may be a prefecture level, and the third lowest level of granularity may be a city, ward, town, or village level. The account identification unit 102 identifies the granularity level of the acquired location information, and identifies the location information of the "country", the location information of the "prefecture", or the location information of the "city, ward, town, or village" based on the acquired location information. For example, when the SNS prepares the user's place of residence or place of origin included in the profile information as a format for registering information of "country", "prefecture", and "city, ward, town, or village", the hierarchical location information of the granularity levels of "country", "prefecture", and "city, ward, town, or village" may be identified according to the above format. For example, if the acquired location information is "Fuchu City," the hierarchical location information of the acquired location information may be identified as location information at a granularity level of "city, ward, town, or village," and the hierarchical location information at a "prefecture" level, which has a lower granularity level than the "city, ward, town, or village" level, may be identified as "Tokyo," and the hierarchical location information at a "country" level may be identified as "Japan."

[0061] Next, the account identifying unit 102 calculates the similarity between the two determined accounts (S204). The account identifying unit 102 refers to the generated hierarchical location information table for each determined account, and calculates the similarity between the determined accounts using the hierarchical location information set in the hierarchical location information table. Specifically, the account identifying unit 102 counts the number of pieces of data of hierarchical location information for each granularity level in the hierarchical location information table for each determined account, and normalizes the number of counted pieces of data. The account identifying unit 102 multiplies the normalized values ​​in the two determined accounts, and sets the multiplied value as the evaluation value of each piece of data. The account identifying unit 102 calculates the sum of the evaluation values ​​of all data common to the two determined accounts as the similarity for each granularity level between the two determined accounts. Furthermore, the account identifying unit 102 calculates the sum of the similarities for each granularity level as the similarity between the two determined accounts.

[0062] Next, the account identifying unit 102 determines whether the two determined accounts are accounts of the same user (S205). Based on the calculated similarity between the determined accounts, the account identifying unit 102 determines whether the two determined accounts are accounts owned by the same user. Specifically, when the similarity between the two determined accounts is equal to or greater than a predetermined threshold, the account identifying unit 102 determines that the two accounts are owned by the same user. Note that the account identifying unit 102 may identify accounts owned by the same user from the similarity of the location information (hierarchical location information table) of related accounts for all accounts included in the social media information.

[0063] As described above, in this embodiment, another account held by the same user is identified based on location information acquired from account information of a related account related to a determined account. In addition, based on the location information of the related account, hierarchical location information indicating location information hierarchicalized according to the granularity level of the location is identified, and another account is identified using the identified hierarchical location information. Furthermore, hierarchical location information is identified for each determined account, the similarity between the determined accounts is calculated using the hierarchical location information, and another account is identified based on the calculated similarity. This makes it possible to accurately identify accounts held by the same user even if the information of the determined account contains false content or information different from the actual information is registered. Therefore, it is possible to accurately identify accounts owned by the same user, regardless of the information registered by the user.

[0064] (Embodiment 3) Next, a third embodiment will be described with reference to the drawings. In this embodiment, another example of the different account identification process (S103 in FIG. 5) in the first embodiment will be described. Note that the configuration of the monitoring support device 100 and other processes are the same as those in the first embodiment.

[0065] FIG. 7 shows an example of another account identification process according to this embodiment. Here, an example will be described in which it is determined whether two accounts to be determined (referred to as determination accounts) are accounts owned by the same user. That is, the two accounts that are ultimately determined to be accounts owned by the same user correspond to the target account and another account identified in the first embodiment. Note that the following process is mainly executed by the account identification unit 102 of the monitoring support device 100, but may be executed by other units as necessary. In this example, the account identification unit 102 identifies the other account based on content data acquired from account information of a related account.

[0066] As shown in FIG. 7, first, the account identification unit 102 acquires content of an associated account related to a first determined account (S301). The account identification unit 102 identifies the first determined account from the collected social media information, and acquires account information of the associated account related to the first determined account. As in the first embodiment, the account identification unit 102 may identify an associated account connected to the first determined account, and acquire account information of the identified associated account. Furthermore, the account identification unit 102 extracts content associated with the associated account from the acquired account information of the associated account. For example, the content is image data uploaded in association with the associated account, and the content is acquired from the posting information of the account information.

[0067] Next, the account identification unit 102 acquires the content of the related account related to the second judgment account (S302). As in S301, the account identification unit 102 identifies the second judgment account, acquires the account information of the related account related to the second judgment account, and extracts the content associated with the related account from the acquired account information.

[0068] Next, the account identification unit 102 determines whether the first determination account and the second determination account are accounts of the same user (S303). Specifically, the account identification unit 102 determines whether the content of the associated account related to the acquired first determination account and the content of the associated account related to the second determination account are similar, and if they are similar, determines that the two determination accounts are accounts owned by the same user. For example, if the similarity is higher than a predetermined threshold, it may be determined that the accounts are owned by the same user.

[0069] The account identification unit 102 may determine the similarity of all acquired content, or may determine only a predetermined type of content, such as image data. The account identification unit 102 may, for example, obtain the similarity of an object detected from the image data. The object to be determined may be any type of object, or may be a specific type of object. When determining a specific type of object, for example, the similarity of only a person among the objects included in the image data may be obtained.

[0070] The account identifying unit 102 may also determine the similarity of topics of image data included in the content. A topic is a main thing or event expressed by the data, such as work, food, sports, travel, games, or politics. The account identifying unit 102 may also extract keywords from text data included in the content and determine the similarity of the text data. The account identifying unit 102 may also extract keywords or voiceprints from audio data, such as data of a single voice included in the content or data of an audio included in a video, and determine the similarity of the audio data. The account identifying unit 102 may also identify accounts owned by the same user from the similarity of content of related accounts for all accounts included in the social media information.

[0071] As described above, in this embodiment, other accounts owned by the same user are identified based on content data acquired from account information of related accounts related to the judged account. In addition, for each judged account, content data associated with the judged account is acquired, and other accounts are identified depending on whether the acquired content data is similar (similarity). Since there is a high probability that the user has made similar information public in accounts owned by the same user, accounts owned by the same user can be accurately identified.

[0072] (Embodiment 4) Next, a fourth embodiment will be described with reference to the drawings. In this embodiment, an example of the account information aggregation process (S104 in FIG. 5) in the first to third embodiments will be described. Note that the configuration of the monitoring support device 100 and other processes are similar to those in the first to third embodiments.

[0073] FIG. 8 shows an example of the account information aggregation process according to the present embodiment. Here, an example will be described in which the reliability of an account to be judged (referred to as a judged account) is calculated and an account to be aggregated is judged. For example, in the first embodiment, when the reliability of the specified other account is higher than the reliability of the target account, information of only the other account may be aggregated. That is, among the judged accounts including the target account and the other account, the account information of the account that is finally judged to be an account with a high reliability may be aggregated. Note that the following process is mainly executed by the account information extraction unit 103 of the monitoring support device 100, but may be executed by other units as necessary. The account information extraction unit 103 can also be said to be a reliability calculation unit that calculates the reliability of the target account and related accounts (other accounts). For example, the personal information extraction unit 104 and the position information extraction unit 105 extract personal information and position information based on the account information of one of the target account and the related accounts, and in particular, extract personal information and position information based on the account information of the account with a high reliability among the target account and the related accounts. In this example, the reliability is based on person attribute information acquired from the account information of the target account and the related accounts.

[0074] 8, first, the account information extraction unit 103 acquires personal attribute information of a determined account (S401). The account information extraction unit 103 may acquire the account information of the determined account from the collected social media information, as in the first embodiment. Furthermore, the account information extraction unit 103 extracts personal attribute information included in the profile information from the acquired account information of the determined account.

[0075] Next, the account information extraction unit 103 acquires personal attribute information of the related account (S402). As in the first embodiment, the account information extraction unit 103 may acquire account information of the related account related to the determined account from the collected social media information. Furthermore, the account information extraction unit 103 extracts personal attribute information included in the profile information from the acquired account information of the related account. For example, the related account may be a friend account included in a friend account list of the determined account.

[0076] Next, the account information extraction unit 103 estimates personal attributes of the user (determined user) of the determined account (S403). The account information extraction unit 103 estimates personal attributes of the determined user who owns the determined account based on the personal attribute information of the acquired related account (friend account). For example, if the personal attribute information of the related account includes a residence, the determined user's residence is estimated based on the physical distance from the residence.

[0077] Next, the account information extraction unit 103 calculates the distance between the person attribute information of the determined account acquired in S401 and the person attribute of the determined user estimated in S403 (S404). For example, the account information extraction unit 103 calculates the distance using information of the same category among the acquired person attribute information and the estimated person attribute. Specifically, the account information extraction unit 103 may calculate the physical distance between the residence included in the profile of the determined account and the residence of the determined user estimated from the related account.

[0078] The category for calculating the distance may be at least one of the differences in demographic attributes such as age, sex, income, educational background (e.g., deviation score or distance between fields), occupation (e.g., blue-collar or white-collar, distance between industries), and family structure. The distance may be calculated by a method based on the distance between fields / industries (e.g., transfer rate / job change to a different field / industry (transition probability)). The category for calculating the distance may be at least one of the differences in psychographic attributes such as hobbies and preferences (e.g., indoor / outdoor) and purchasing trends.

[0079] Next, the account information extraction unit 103 calculates the reliability of the determined account based on the calculated distance (S405). The reliability may be a numerical index calculated from the distance.

[0080] Next, the account information extraction unit 103 determines the account to be aggregated based on the calculated reliability (S406). When the reliability of the determined account is greater than a predetermined threshold, the account information extraction unit 103 determines that the determined account is an account to be aggregated. For example, the reliability of two determined accounts (an account other than the target account) may be calculated, and only the account with the higher reliability may be determined to be the account to be aggregated.

[0081] As described above, in this embodiment, for each judged account, the reliability of the judged account is calculated based on the personal attribute information acquired from the account information of the judged account. In addition, the reliability of the judged account is calculated based on the personal attribute information of the related account related to the judged account. Furthermore, the personal attribute of the judged account is estimated based on the personal attribute information of the related account, and the reliability of the judged account is calculated based on the distance between the acquired personal attribute information of the judged account and the estimated personal attribute of the judged account. This makes it possible to judge the reliability of the judged account (whether it is a fake account, etc.), so that only the information of highly reliable accounts can be aggregated. Note that the reliability calculated in this embodiment may be used to identify other accounts held by the same user.

[0082] (Embodiment 5) Next, a fifth embodiment will be described with reference to the drawings. In this embodiment, an example of the image position identifying unit (image position identifying unit 110 in FIG. 3) and the image position identifying process (S106 in FIG. 5) in the first to fourth embodiments will be described. Other configurations of the surveillance support device 100 and other processes are the same as those in the first to fourth embodiments.

[0083] 9 shows a configuration example of the image position identifying unit 110 of the surveillance support device 100 according to the present embodiment. As shown in FIG. 9, the image position identifying unit 110 includes a search unit 111, a discriminator 112, and a position database 113. For example, the position database 113 may be included in the storage unit 108 of the surveillance support device 100.

[0084] A ground-view image is input to the image position identification unit 110. The ground-view image is an image of a certain place (position) captured from a ground camera of a pedestrian, a vehicle, or the like. The ground-view image may be a panoramic image having a 360-degree field of view, or may be an image having a predetermined field of view less than 360 degrees. For example, the input ground-view image is a posted image included in the account information of the target account in the first embodiment.

[0085] The position database 113 is an image database with position information, and stores a plurality of overhead images (position images) associated with the position information. For example, the position information is the GPS coordinates of the position where the overhead image was captured. The overhead image is an image captured from an overhead view (planar view) of a certain location by a camera in the sky, such as a drone, an airplane, or a satellite.

[0086] The search unit 111 acquires a ground-view image for identifying location information. The search unit 111 searches the location database 113 for an overhead image that matches the acquired ground-view image, and determines the location where the ground-view image was captured. Specifically, the process of sequentially acquiring overhead images from the location database 113 is repeated until an overhead image that matches the ground-view image is detected. In this example, the ground-view image and the overhead image are input to the classifier 112, and the output of the classifier 112 is determined to indicate whether the ground-view image and the overhead image match, thereby searching for an overhead image including the location where the ground-view image was captured. The search unit 111 identifies the location where the ground-view image (an acquired image such as a posted image) was captured, based on location information associated with the detected overhead image.

[0087] The classifier 112 acquires a ground-view image and an overhead image, and identifies whether or not the acquired ground-view image and the overhead image match. Note that "the ground-view image and the overhead image match" means that the position where the ground-view image was captured is included in the overhead image. The classification by the classifier 112 can be realized by various methods. For example, the classifier 112 extracts features of the ground-view image and the overhead image, and calculates the similarity between the features of the ground-view image and the overhead image. If the calculated similarity is high (e.g., equal to or greater than a predetermined threshold), the classifier 112 determines that the ground-view image and the overhead image match, whereas if the calculated similarity is low (e.g., less than a predetermined threshold), the classifier 112 determines that the ground-view image and the overhead image do not match. For example, the classifier 112 is generated by machine learning (training) in advance on the relationship between the ground-view image and a plurality of overhead images.

[0088] Fig. 10 shows an example of the configuration of the classifier 112 according to this embodiment. Fig. 10 shows an example in which the classifier 112 is implemented using a plurality of neural networks. As shown in Fig. 10, the classifier 112 includes an extraction network 114, an extraction network 115, and a judgment network 116.

[0089] The extraction network (first extraction unit) 114 is a neural network that acquires a ground-view image, generates a feature map of the acquired ground-view image (extracts features of the ground-view image), and outputs the generated feature map. The extraction network (second extraction unit) 115 is a neural network that acquires an overhead image, generates a feature map of the acquired overhead image (extracts features of the overhead image), and outputs the generated feature map. The judgment network (judgment unit) 116 is a neural network that analyzes the generated feature map of the ground-view image and the generated feature map of the overhead image, and outputs whether the ground-view image and the overhead image match.

[0090] 11 shows a training process (learning method) of the classifier 112 according to this embodiment. This training process may be performed by the monitoring support device 100 or by another training device (not shown). Here, the description will be given assuming that the training process is performed by a training device.

[0091] First, the training device acquires a training data set (S501). The training device acquires a training data set including ground-view images and overhead images associated with position information, which are prepared in advance. The training data set includes ground-view images, positive examples of overhead images, first-level negative examples of overhead images, and second-level negative examples of overhead images. Note that a positive example is an overhead image that matches a corresponding ground-view image (the distance between images is equal to or less than a predetermined threshold). A negative example is an overhead image that does not match a corresponding ground-view image (the distance between images is greater than a predetermined threshold).

[0092] The similarity of the first level negative example to the ground-view image is different from the similarity of the second level negative example to the horizon-view image. For example, each overhead image is associated with information indicating the type of scenery included in the overhead image. The first level negative example includes a different type of scenery from the scenery included in the corresponding ground-view image, and the second level negative example includes the same type of scenery as the scenery included in the corresponding ground-view image. This means that the similarity of the first level negative example to the corresponding ground-view image is lower than the similarity of the second level negative example to the corresponding ground-view image.

[0093] Next, the training device executes the first stage training of the classifier 112 (S502). The training device inputs ground-view images and positive examples to the classifier 112, and updates the parameters of the classifier 112 using the output of the classifier 112. The training device also inputs ground-view images and first-level negative examples to the classifier 112, and updates the parameters of the classifier 112 using the output of the classifier 112. First, in the first stage training, a set of neural networks is trained using ground-view images, positive examples, and a loss function of the positive examples (positive loss function). The positive loss function is designed to train the classifier 112 to output a greater similarity between the ground-view images and the positive examples.

[0094] In the classifier 112 of FIG. 10, the ground-view images and the positive examples are input to the extraction network 114 and the extraction network 115, respectively. Then, the output from the set of neural networks is input to the positive loss function, and the parameters (weights) assigned to each connection between nodes in the neural network constituting the classifier 112 are updated based on the calculated loss. Furthermore, in the first stage of training, the set of neural networks is trained using the ground-view images, the negative examples, and a loss function for the negative examples (negative loss function). The negative loss function is designed to train the classifier 112 to output a smaller similarity between the ground-view images and the negative examples.

[0095] 10, the ground-view image and the negative example are input to extraction network 114 and extraction network 115, respectively. Then, the output from the set of neural networks is input to a negative loss function, and the parameters (weights) assigned to each connection between nodes in the neural network constituting the classifier 112 are updated based on the calculated loss.

[0096] Next, the training device executes second-stage training of the classifier 112 (S503). The second-stage training is similar to the first-stage training except that second-level negative examples are used. That is, ground-view images and positive examples are input to the classifier 112, and the parameters of the classifier 112 are updated using the output of the classifier 112. Also, ground-view images and second-level negative examples are input to the classifier 112, and the parameters of the classifier 112 are updated using the output of the classifier 112.

[0097] As described above, according to the present embodiment, a classifier is trained (learned) using an overhead image and a ground-view image that are associated with position information in advance, and the location where the ground-view image was taken is identified using the obtained classifier. This makes it possible to reliably identify the location where the posted image was taken.

[0098] (Embodiment 6) Next, a sixth embodiment will be described with reference to the drawings. In this embodiment, an example of the activity area estimation process (S107 in FIG. 5) in the first to fifth embodiments will be described. Note that the configuration of the monitoring support device 100 and other processes are similar to those in the first to fifth embodiments.

[0099] FIG. 12 shows an example of an activity area estimation process according to the present embodiment. Here, an example of determining whether a posting location is ordinary / unordinary will be described. That is, a location determined to be highly ordinary is a location included in the activity area of ​​the target user in the first embodiment. Note that the following process is mainly executed by the activity area estimation unit 120 of the monitoring support device 100, but may be executed by other units as necessary. In this example, the activity area estimation unit 120 estimates the activity area of ​​the target user depending on whether a location specified from the account information of the target account and related accounts is an ordinary or unordinary activity location of the target user.

[0100] As shown in FIG. 12, first, the activity area estimation unit 120 acquires residence information of related accounts (S601). As in the first embodiment, the activity area estimation unit 120 may acquire account information of related accounts related to the target account from the collected social information. Furthermore, the activity area estimation unit 120 acquires residence information (activity base information) of the related accounts from the acquired account information of the related accounts. For example, the activity area estimation unit 120 may acquire residence information from the residence, birthplace, etc. of profile information included in the account information of the related accounts, or may acquire residence information based on words that can identify the residence from the posted information included in the related account information.

[0101] The residence information is information that geographically identifies the residence of a user who holds an account. The user's residence is a place that is the base of the user's life, and is intended to be an area such as a prefecture or a city, town, or village, but there is no particular limitation on the unit by which the area is divided. For example, the user's residence may be an area identified by the latitude and longitude of the east, west, north, and south end points. The user's residence may also include multiple geographically separated areas. Furthermore, the user's residence may also include the workplace of related users and stations on the commute route.

[0102] Next, the activity area estimation unit 120 estimates the residence of the target user (S602). The activity area estimation unit 120 estimates the residence (activity base) of the target user who holds the target account based on the residence information of the acquired related account. The activity area estimation unit 120 regards each of the multiple residence information of the related accounts as a residence candidate of the target user, calculates a score for each of the residence candidates indicating the possibility that the target user resides in the residence candidate, and estimates the residence candidate with the highest score or N residence candidates with the top N scores (N is a positive integer of 1 or more) as the residence of the target user. For example, the score may be based on the presence or absence of a friendship relationship, the distance between the residences of friends, etc.

[0103] The estimated residence (estimated residence) is information that geographically identifies the residence of the target user estimated from the residence information. Since the estimated residence is estimated from the residence information of the related account, it represents an area such as a prefecture or a city, town, or village, similar to the original residence information. In addition, the estimated residence may represent an area identified by the latitude and longitude of the east-west, north-south, and south ends, or may include multiple geographically separated areas, or may include the workplace or a station on the commute route.

[0104] Next, the activity area estimation unit 120 extracts the posting location from the account information of the target account (S603). As in the first embodiment, the activity area estimation unit 120 acquires the posting information (such as an obtainable image) included in the account information of the target account (which may include related accounts), and extracts the posting location where the acquired posting information was posted. When the longitude and latitude of the shooting location or the current location is linked to the posted content by information such as a GEO tag, the activity area estimation unit 120 may acquire the longitude and latitude of the posting location from the linked information. In addition, when information such as a GEO tag is not linked to the post, the activity area estimation unit 120 may estimate the posting location using words or hashtags specific to the region included in the posted text. The posting location is information that geographically identifies the location where the content was posted to the social media by the target user. The posting location may be the address of the posting location, or may be the longitude and latitude of the posting location.

[0105] Next, the activity area estimation unit 120 compares the posting location acquired in S603 with the residence estimated in S602 (S604). The activity area estimation unit 120 compares the posting location of the acquired account information of the target account with the estimated residence of the target user. The comparison result indicates, for example, whether the posting location is within the estimated residence or outside the estimated residence.

[0106] Next, the activity area estimation unit 120 judges whether the posting location is ordinary or extraordinary (S605). Based on a comparison result between the acquired posting location and the estimated residence, the activity area estimation unit 120 judges whether the posting location is an ordinary activity location of the target user or an extraordinary activity location. For example, when the comparison result indicates that the posting location is within the estimated residence, the activity area estimation unit 120 judges that the posting location is an ordinary activity location of the target user. Also, when the comparison result indicates that the posting location is outside the estimated residence, the activity area estimation unit 120 judges that the posting location is an extraordinary activity location of the target user. For example, when it is determined that the posting location is an ordinary activity location of the target user, the posting location is estimated to be the activity area of ​​the target user.

[0107] As described above, in this embodiment, the activity area of ​​the target user can be identified depending on whether the posting location acquired from the account information is a place where the target user engages in everyday or unusual activities. According to this embodiment, based on the knowledge that friends who have some kind of connection are geographically close to each other, the residence (activity base) of the target user is estimated from the residence information (activity base information) of an associated account related to the target account. Then, the residence estimated from the associated account is compared with the posting location of the posted information of the target account to determine whether the posting location is everyday or unusual. This makes it possible to accurately estimate the activity area of ​​the target user.

[0108] The residence of the target user may be estimated from residence information of other users (offline friends) who have interactions with the target user in physical space. For example, when estimating the residence, a score may be calculated by weighting the residence candidates of related users who are determined to be offline friends. From among the related users, related users whose related accounts are local accounts related to a specific region may be selected as offline friends of the target user.

[0109] Furthermore, the ordinaryness / unordinaryness of a posting location may be determined based on the relationship between a location attribute that represents the attribute of the posting location and a person attribute that represents the attribute of the target user. For example, the location attribute is information that represents whether the posting location is a famous tourist spot or not, whether it is a high-end restaurant or not, etc. For example, the person attribute is information that represents the hobbies, preferences, income, occupation, etc. of the target user. When there is an association (high association) between the location attribute and the person attribute, the posting location is determined to be a place for ordinary activity.

[0110] Furthermore, it may be determined whether the schedule of the target user at the posting date and time is ordinary or extraordinary based on the relationship between the target user's past behavior history and future schedule and the posting date and time. If there is a correlation between the schedule at the posting date and time and the location attribute, it is determined whether the target user's schedule at the posting date and time is ordinary or extraordinary based on the purpose of the behavior, periodicity, and the like. For example, if the schedule at the posting date and time is a hospital visit that is carried out for a certain period of time or with a certain frequency, it is determined to be ordinary. Also, if the schedule at the posting date and time is a business trip or a homecoming for a certain period of time, or participation in an event that is participated in every year, it is determined to be ordinary. Also, if the schedule at the posting date and time is participation in a one-off event or business trip, it is determined to be extraordinary.

[0111] In addition, the ordinaryness / unordinaryness of the posting location may be determined based on the relationship between the posting location and the friend posting area of ​​the friend account. The friend posting area is information about the area of ​​the posting location of the related user generated based on the location where the user of the related account posted content on social media. The friend posting area and the posting location are compared geographically, and the ordinaryness / unordinaryness is determined based on the comparison result between the posting location and the estimated residence and the comparison result between the friend posting area and the posting location. For example, if the posting location indicates that it is outside the estimated residence, and if the posting location indicates that it is within the friend posting area, the posting location is determined to be within the target user's everyday activity location. Also, if the posting location indicates that it is within the estimated residence, the posting location is determined to be the target user's everyday location.

[0112] (Embodiment 7) Next, a seventh embodiment will be described with reference to the drawings. In this embodiment, another example of the activity area estimation process (S107 in FIG. 5) in the first to fifth embodiments will be described. Note that the configuration of the monitoring support device 100 and other processes are similar to those in the first to fifth embodiments.

[0113] 13 shows an example of an activity area estimation process according to the present embodiment. The following process is mainly executed by the activity area estimation unit 120 of the monitoring support device 100, but may be executed by other units as necessary. In this example, the activity area estimation unit 120 estimates the activity area of ​​the target account based on a location identified from the account information of a related account (offline friend) that has a friend relationship with the target account in physical space.

[0114] 13, first, the activity area estimation unit 120 acquires information on friend accounts (S701). As in the first embodiment, the activity area estimation unit 120 acquires account information on friend accounts (related accounts) related to the target account from the collected social information.

[0115] Next, the activity area estimation unit 120 determines whether the user of the friend account (friend user) is an offline friend (S702). Based on the acquired account information of the friend account, the activity area estimation unit 120 determines whether each friend user who has a friend account is also a friend of the target user in the physical world or is not a friend in the physical world.

[0116] As a result of the offline friend determination, the activity area estimation unit 120 calculates an offline friend degree indicating whether or not a friend relationship is formed in the physical space (offline) between the friend user and the target user. For example, a score indicating the degree of offline friend is calculated for each friend account of the target user, and if the score exceeds a certain threshold, the offline friend degree may be a value indicating offline friend (e.g., "1"), and if the score is equal to or less than the threshold, the offline friend degree may be a value indicating not offline friend (e.g., "0").

[0117] The activity area estimation unit 120 may also determine whether the friend account of the target user is a local account related to a specific region. For example, a local account is a social media account operated for a specific location or region among social media accounts. Examples of local accounts include accounts operated by local newspapers, local governments, and community-based businesses such as privately-run restaurants. The activity area estimation unit 120 may calculate the offline friend degree of the friend user based on the result of the determination of whether the friend account is a local account.

[0118] Furthermore, the activity area estimation unit 120 may calculate the offline friend degree according to the administrative level of the area targeted by each friend account. For example, the offline friend degree of an official account of a city, ward, town, or village that targets a small area may be set to a high value (e.g., "1"), the offline friend degree of an account targeting a prefecture level may be set to a medium value (e.g., "0.7"), and the offline friend degree of an account targeting a country level may be set to a low value (e.g., "0.2").

[0119] In addition, when it is determined that it is unclear whether a friend account is a local account, the activity area estimation unit 120 may refer to the friend information of the friend account to determine whether the friend account is a local account. For example, it may determine whether the friend account of the target user is a local account based on whether the account of the friend of the friend account is a local account.

[0120] The activity area estimation unit 120 may calculate the reliability of the offline friend degree (determination result) in addition to the offline friend degree. The reliability indicates the reliability of the determination result with an offline friend. For example, the reliability is determined depending on what information or method was used to determine the offline friend. For example, if a friend user of the target user is determined to be an offline friend based on friend information of a friend account of the target account, the reliability of the determination may be considered to be high, and if a friend account is determined to be an offline friend based on friend information of a friend of the friend account, the reliability may be considered to be low.

[0121] Next, the activity area estimation unit 120 determines a weight to be assigned to each determined friend user (S703). The activity area estimation unit 120 determines a weight indicating the degree of importance to be attached to the friend information based on the calculated offline friend degree and its reliability. For friend users determined to be offline friends, the activity area estimation unit 120 sets a relatively large weight to the friend information, and for friend users determined not to be offline friends, sets a relatively small weight to the friend information. In addition, when determining the weight, the weight may be increased or decreased based on the reliability.

[0122] Next, the activity area estimation unit 120 calculates a score for the candidate activity positions of the target user based on the weighted friend user information (S704). The activity area estimation unit 120 calculates a score representing the possibility of the target user's activity at each candidate position based on the weighted friend information. This score indicates the possibility that the target user will be active at each candidate position. Here, the "candidate position" refers to a candidate space in which the target user is thought to be active. The candidate positions may be selected in advance, or the candidate positions may be selected from the activity positions of the friend users.

[0123] For example, the activity area estimation unit 120 calculates the distance between each candidate location and the activity location of each friend user, and calculates a score that indicates the relationship between the presence or absence of a friendship and the distance. In calculating the score, the degree to which friend information is emphasized may be increased or decreased according to the calculated weight of each friend. For example, the larger the weight value, the more importance is placed on the friend information in calculating the score. In other words, the larger the weight value, the greater the influence of friend information on the score calculation.

[0124] Next, the activity area estimation unit 120 estimates an activity range (activity area) based on the calculated score (S705). The activity area estimation unit 120 selects candidate locations based on the score for each candidate location and determines any activity range related to the target user. For example, the candidate location with the highest score may be searched for. The candidate location with the highest score is considered to correspond to the location where the target user is based, such as the target user's residence or workplace. The activity area estimation unit 120 selects the candidate location with the highest score as the user's activity range. In this case, the location where the target user is based can be estimated.

[0125] The activity area estimation unit 120 may also compare the score with a threshold value and select one or more candidate locations with a score equal to or greater than the threshold value as the user's activity range. Candidate locations with a score equal to or greater than the threshold value are considered to correspond to the target user's base, such as a place of residence, and the range of movement in daily life. In this case, the target user's base location and the range of movement in daily life can be estimated.

[0126] As described above, in this embodiment, the activity area of ​​the target user is identified based on the offline friend degree indicating the degree of friendship in physical space between the target user of the target account and related users (friend users) of related accounts related to the target account. In addition, the score of the candidate location is calculated based on the offline friend degree of the friend users, and the activity area of ​​the target user is estimated from the calculated score. This makes it possible to accurately estimate the activity area of ​​the target user.

[0127] In addition, the activity range of the target user may be estimated by using only information of active users among the acquired friend information. It is determined whether each of the friend users of the target user is an active user who utilizes social media or an inactive user. For example, it may be determined whether the friend user is an active user based on the posting frequency of the friend account, or it may be determined whether the friend user is an active user based on the interval of login of the friend account by receiving information about the login of the friend account.

[0128] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present disclosure.

[0129] Each configuration in the above-described embodiment may be configured by hardware or software, or both, and may be configured by one piece of hardware or software, or may be configured by multiple pieces of hardware or software. Each device and each function (processing) may be realized by a computer 20 having a processor 21 such as a CPU (Central Processing Unit) and a memory 22 which is a storage device, as shown in Fig. 14. For example, a program for performing a method in the embodiment (such as a monitoring support method) may be stored in the memory 22, and each function may be realized by having the processor 21 execute the program stored in the memory 22.

[0130] These programs can be stored and provided to a computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)). The programs may also be provided to a computer by various types of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. The transitory computer readable media can provide the programs to a computer via a wired communication path such as an electric wire and an optical fiber, or via a wireless communication path.

[0131] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0132] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0133] (Appendix 1) A personal information extraction means for extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; A location information extraction means for extracting location information related to the target user based on the account information; an output means for outputting the extracted personal information and the extracted location information as support information for supporting crime prevention in the vicinity of the location information in a physical space; A support device comprising: (Appendix 2) The support information is information for supporting monitoring or investigation of the target user. 2. The support device according to claim 1. (Appendix 3) The account information includes account information of the target account or account information of an associated account related to the target account; 3. The assistance device according to claim 1 or 2. (Appendix 4) The related account is an account that is connected to the target account in the cyberspace. 4. The support device according to claim 3. (Appendix 5) The related account includes an account other than the target account held by the target user. 5. The assistive device according to claim 3 or 4. (Appendix 6) an account identification means for identifying the other account based on account information of the target account and account information of the related account; 6. The support device according to claim 5. (Appendix 7) The account identification means identifies the other account based on location information acquired from account information of the related account. 7. The support device according to claim 6. (Appendix 8) The account identification means identifies hierarchical location information obtained by hierarchically classifying the acquired location information according to a granularity level of the location, and identifies the other account based on the identified hierarchical location information. 8. The support device according to claim 7. (Appendix 9) the account identification means identifies the other account based on content data acquired from account information of the related account; 7. The support device according to claim 6. (Appendix 10) The personal information extraction means and the location information extraction means extract the personal information and the location information based on account information of either the target account or the related account. 10. The assistance device according to any one of claims 3 to 9. (Appendix 11) The personal information extraction means and the location information extraction means extract the personal information and the location information based on account information of an account having a high reliability among the target account and the related accounts. 11. The support device of claim 10. (Appendix 12) The reliability is based on personal attribute information obtained from account information of the target account and the related accounts. 12. The support device according to claim 11. (Appendix 13) The account information includes profile information or posting information. 13. An assistance device according to any one of claims 1 to 12. (Appendix 14) The personal information includes any one of the target user's biometric information, soft biometric information, belongings, name, and attribute information; 14. An assistance device according to any one of claims 1 to 13. (Appendix 15) The location information extraction means identifies the location information based on the captured image acquired from the account information. 15. An assistance device according to any one of claims 1 to 14. (Appendix 16) the position information extraction means identifies the position information based on a comparison between the acquired image and a plurality of position images to which position information is previously associated. 16. The support device of claim 15. (Appendix 17) The location information extraction means compares a location image related to the target account among the plurality of location images with the acquired image. 17. The support device of claim 16. (Appendix 18) The acquired image is a ground-view image captured from a ground perspective, and the plurality of position images is a plurality of overhead images captured from an overhead perspective. 18. The assistance device of claim 16 or 17. (Appendix 19) the location information extraction means identifies the overhead image that matches the acquired image by using a classifier that has machine-learned the relationship between the ground-view image and the plurality of overhead images; 19. The support device of claim 18. (Appendix 20) The classifier is a first extraction means for extracting features of the ground-view image; A second extraction means for extracting features of the overhead image; a determination means for determining whether or not the ground-view image and the overhead-view image match based on the extracted features of the ground-view image and the features of the overhead-view image; 20. The assistance device of claim 19, comprising: (Appendix 21) The location information extraction means estimates an activity area of ​​the target user as the extracted location information. 21. An assistance device according to any one of claims 1 to 20. (Appendix 22) The location information extraction means estimates the activity area based on a location identified from account information of the target account and a related account related to the target account. 22. The support device of claim 21. (Appendix 23) The location information extraction means estimates the activity area depending on whether a location specified from the account information is a place where the target user regularly or irregularly engages in activities. 23. The assistance device according to claim 21 or 22. (Appendix 24) The location information extraction means estimates the activity area based on account information of the related account that is in a friend relationship with the target account in a physical space. 23. The support device of claim 22. (Appendix 25) A plurality of monitoring systems for monitoring different locations and a support device are provided; The support device includes: A personal information extraction means for extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; A location information extraction means for extracting location information related to the target user based on the account information; an output means for outputting the extracted personal information to the monitoring system selected based on the extracted location information; A system comprising: (Appendix 26) The monitoring system registers the output personal information in a watch list, which is a list of people to be monitored. 26. The system of claim 25. (Appendix 27) the output means selects the surveillance system in a public facility in the vicinity of the location information; 27. The system of claim 25 or 26. (Appendix 28) The output means selects the monitoring system based on a score indicating the possibility that the target user is located. 28. The system of any one of claims 25 to 27. (Appendix 29) the output means selects the monitoring system based on a degree of congestion in the vicinity of the location information. 29. The system of any one of claims 25 to 28. (Appendix 30) the output means selects the monitoring system in a public transportation facility along a travel route of the target user estimated from the location information; 30. The system of any one of claims 25 to 29. (Appendix 31) extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; Extracting location information related to the target user based on the account information; outputting the extracted personal information and the extracted location information as support information for supporting crime prevention in the vicinity of the location information in a physical space; How to help. (Appendix 32) extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; Extracting location information related to the target user based on the account information; outputting the extracted personal information and the extracted location information as support information for supporting crime prevention in the vicinity of the location information in a physical space; A non-transitory computer-readable medium having stored thereon a support program for causing a computer to carry out a process. [Explanation of symbols]

[0134] 1 Cyber-physical integrated monitoring system 10 Support equipment 11 Personal information extraction section 12 Location information extraction section 13 Output section 20. Computers 21 Processors 22 Memory 100 Monitoring support equipment 101 Social Media Information Acquisition Department 102 Account Identification Department 103 Account Information Extraction Unit 104 Personal information extraction section 105 Location information extraction section 106 Surveillance System Selection Section 107 Personal Information Output Section 108 Storage section 110 Image position identification section 111 Search Department 112 Classifier 113 Location Database 114 Extraction Network 115 Extraction Network 116 Judgment Network 120 Activity Area Estimation Department 200 Surveillance System 201 Surveillance Devices 202 Surveillance person information extraction unit 203 Surveillance Person Information Verification Unit 204 Matching result output unit 205 Watchlist Storage 206 Watchlist Creation Department 300 Social Media System

Claims

1. A plurality of monitoring systems for monitoring different locations and a support device are provided; The support device includes: A personal information extraction means for extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; A location information extraction means for extracting location information related to the target user based on the account information; an output means for outputting the extracted personal information to the monitoring system selected based on the extracted location information; Equipped with the output means selects the monitoring system based on a degree of congestion in the vicinity of the location information. system.

2. The account information includes account information of the target account or account information of an associated account related to the target account; The system of claim 1 .

3. The related account is an account that is connected to the target account in the cyberspace. The system of claim 2.

4. The related account includes an account other than the target account held by the target user.

4. A system according to claim 2 or 3.

5. The support device includes an account identification means for identifying the other account based on account information of the target account and account information of the related account. The system of claim 4.

6. The account identification means identifies the other account based on location information acquired from account information of the related account. The system of claim 5.

7. A support device that constitutes a system together with a plurality of monitoring systems that monitor different locations, A personal information extraction means for extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; A location information extraction means for extracting location information related to the target user based on the account information; an output means for outputting the extracted personal information to the monitoring system selected based on the extracted location information; Equipped with the output means selects the monitoring system based on a degree of congestion in the vicinity of the location information. Support equipment.

8. A support method executed by a support device constituting a system together with a plurality of monitoring systems monitoring different locations, comprising: extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; Extracting location information related to the target user based on the account information; outputting the extracted personal information to the monitoring system selected based on the extracted location information; In the output to the monitoring system, the monitoring system is selected based on a degree of congestion in the vicinity of the position information. How to help.

9. A support program for causing a computer to execute a support method for a support device constituting a system together with a plurality of monitoring systems for monitoring different locations, comprising: extracting personal information capable of identifying a target user who holds a target account based on account information acquired from the target account in cyberspace; Extracting location information related to the target user based on the account information; outputting the extracted personal information to the monitoring system selected based on the extracted location information; In the output to the monitoring system, the monitoring system is selected based on a degree of congestion in the vicinity of the position information. Support programs, including processing.

Images