Hazard Identification Program

Abstract

To detect a danger level in advance and prevent an incident from happening.SOLUTION: A program for identifying a danger level of appearance of a suspicious person on a public road consisting of a road or station premises causes a computer to perform an information acquisition step of, in identifying the danger level, acquiring person information obtained by extracting the person by analyzing a newly captured image of the public road and audio information recorded at the time of capturing the image and an identification step of referring to three or more stages of association levels between the reference person information obtained by extracting the person by analyzing the captured image of the public road and the danger level, and on the basis of the reference person information according to the person information acquired through the information acquisition step, and prioritizing the one with the higher association degree to identify the danger level of the appearance of a suspicious person on the public road on the basis of the acquired audio information.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a risk discrimination program for discriminating the risk of intrusion of suspicious persons or animals into architectural structures, cultivated land, public roads, and the like. [Background technology]

[0002] Numerous attempts have been made to prevent intrusion of suspicious individuals into buildings such as houses and buildings housing businesses. In particular, detached buildings have many more intrusion routes than apartment buildings, so it is necessary to take more robust measures to prevent intrusion of suspicious individuals in order to protect the safety of residents. In addition, damage caused by animals such as bears and wild boars has become a problem in recent years. In particular, detached buildings have many more intrusion routes than apartment buildings, so it is necessary to take more robust measures to prevent intrusion of suspicious individuals in order to protect the safety of residents.

[0003] In recent years, there have been frequent cases of suspicious people and animals invading public roads and cultivated land, which has become a social problem. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent application 2016-210848 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology disclosed in the above-mentioned Patent Document 1 describes a technology for identifying a person from a face image using artificial intelligence, but it does not allow for quantitative determination of the risk of a suspicious person intruding into an actual building structure using artificial intelligence. It was necessary to accurately predict the actual risk in order to promptly contact a security company or the police when danger to a resident is imminent, and to avoid needlessly calling for the dispatch of security guards or police officers from the security company when the actual risk is not that high.

[0006] Such a viewpoint is not particularly mentioned in the disclosed technology of the cited document 1. Similarly, there is no mention of animal damage.

[0007] The present invention has been devised in consideration of the above-mentioned problems, and its object is to provide a danger discrimination program that uses artificial intelligence to automatically determine the danger level, in order to detect in advance the danger level of intrusion of suspicious persons or animals into building structures, cultivated land, or public roads, and to protect the safety of residents. [Means for solving the problem]

[0008] The danger level discrimination program of the present invention is a danger level discrimination program for discriminating the risk level of a suspicious person appearing on a public way consisting of a road or a station premises, and is characterized in that, when discriminating the risk level, the program has a computer execute an information acquisition step of acquiring personal information extracted by analyzing a newly taken image of the public way and audio information recorded at the time of taking the image, and a discrimination step of referring to reference personal information extracted by analyzing the image of the public way and a correlation level of three or more with the risk level, giving priority to reference personal information with a higher correlation level based on the personal information acquired through the information acquisition step, and discriminating the risk level of a suspicious person appearing on the public way based on the acquired audio information. Effect of the Invention

[0009] In order to detect the risk of intrusion of suspicious people or animals onto public roads, etc. in advance and prevent incidents from occurring, the level of danger can be determined automatically and with a high degree of accuracy using artificial intelligence. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an overall configuration of a risk determination system in which a risk determination program to which the present invention is applied is implemented. [Diagram 2] FIG. 2 is a diagram illustrating a specific example of the configuration of a discrimination device. [Diagram 3] 11 is a diagram for explaining the operation of a risk determination program to which the present invention is applied. FIG. [Figure 4] FIG. 13 is a diagram showing an example in which three or more levels of association are set, including reference location information. [Diagram 5] FIG. 13 is a diagram showing an example in which three or more stages of association degrees are set, including reference at-home pattern information. [Figure 6] FIG. 13 is a diagram showing an example in which three or more levels of correlation are set, including reference time zone information. [Figure 7] FIG. 13 is a diagram showing an example in which three or more levels of association are set, including reference person information. [Figure 8] FIG. 13 is a diagram showing an example in which three or more levels of association are set, including reference audio information. [Figure 9] FIG. 13 is a diagram showing an example in which three or more levels of correlation are set, including reference security service information. [Figure 10] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 11] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 12] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 13] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 14] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 15] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 16] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 17] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 18] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 19] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 20] FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 21]FIG. 2 is a diagram for explaining the operation of the present invention. [Figure 22] FIG. 2 is a diagram for explaining the operation of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a risk determination program to which the present invention is applied will be described in detail with reference to the drawings.

[0012] 1 is a block diagram showing the overall configuration of a risk determination system 1 in which a risk determination program according to the present invention is implemented. The risk determination system 1 includes an information acquisition unit 9, a determination device 2 connected to the information acquisition unit 9, and a database 3 connected to the determination device 2.

[0013] The information acquisition unit 9 is a device for a person who uses the system to input various commands and information, and specifically includes a keyboard, buttons, a touch panel, a mouse, a switch, and the like. The information acquisition unit 9 is not limited to a device for inputting text information, and may be configured as a device capable of detecting voice such as a microphone and converting it into text information. The information acquisition unit 9 may also be configured as an imaging device capable of capturing images such as a camera. The information acquisition unit 9 may also be configured as a scanner having a function of recognizing character strings from a paper document. The information acquisition unit 9 may also be integrated with the discrimination device 2 described later. The information acquisition unit 9 outputs the detected information to the discrimination device 2.

[0014] The database 3 stores information on the risk of suspicious intrusion into architectural structures, such as previous incidents involving the intrusion of suspicious intruders into architectural structures, or cases that did not result in an incident but were highly dangerous. The database 3 stores reference image information previously captured by the camera that actually constitutes the information acquisition unit 9, reference structural information including structural information on the architectural structure, reference location information for identifying the location of the architectural structure, reference at-home pattern information indicating the at-home pattern of the resident of the architectural structure, reference time period information indicating the time period at the time of capturing the reference image information, reference person information extracted by analyzing the reference image information, reference voice information recorded at the time of capturing the reference image information, reference security service information indicating the contract status of the architectural structure for the security service of a security company, etc. In addition to the above, the database 3 stores reference traffic volume information indicating the traffic volume of the roads around the architectural structure, and reference crime history information regarding the past crime history in the area of ​​the architectural structure. The architectural structure includes not only houses such as detached houses, apartment buildings, condominiums, and apartments, but also buildings occupied by companies, schools, hospitals, public facilities, and amusement facilities. The present invention can be applied not only to determining the risk of intrusion of a suspicious person into an architectural structure, but also to determining the risk of appearance of a suspicious person on a public road. The public road here includes roads (including not only sidewalks but also crosswalks), as well as underground passages leading to railway stations and passages within railway stations. The risk of appearance of a suspicious person among pedestrians walking on this public road is determined (hereinafter referred to as the risk of appearance of a suspicious person). This risk of appearance of a suspicious person may indicate the risk of a person who actually behaves like a suspicious person among pedestrians, or may indicate the probability that a person who behaves like a suspicious person will appear later even if there is no pedestrian actually behaving like that among pedestrians. Also, it may indicate the possibility that a pedestrian will enter the shooting range later and a suspicious person will appear among the pedestrians even if there is no pedestrian in the actual shooting range.

[0015] The present invention can also be applied to determining the risk of a suspicious person appearing inside a public transportation vehicle. The public transportation vehicle referred to here is a train (including a monorail, rail bus, bullet train, ropeway, cable car, etc.), a bus, a taxi, etc. The risk of a suspicious person appearing inside such a public transportation vehicle is determined (hereinafter, referred to as the risk of a suspicious person appearing). This risk of a suspicious person appearing may indicate the risk of a person actually behaving like a suspicious person inside the vehicle, or may indicate the probability that a person behaving like a suspicious person will appear later even if there is no person actually behaving like that inside the vehicle. Also, even if there are no passengers in the actual photographed range, it may indicate the possibility that a passenger will enter the photographed range later and a suspicious person will appear among the passengers.

[0016] The discrimination device 2 is configured with electronic devices such as a personal computer (PC), but may be embodied in any other electronic devices such as a mobile phone, a smartphone, a tablet terminal, a wearable terminal, etc. By obtaining a search solution by the discrimination device 2, a user can discern the degree of risk of a suspicious person's intrusion into a building structure.

[0017] 2 shows a specific example of the configuration of the discrimination device 2. In this discrimination device 2, a control unit 24 for controlling the entire discrimination device 2, an operation unit 25 for inputting various control commands via an operation button, a keyboard, or the like, a communication unit 26 for performing wired or wireless communication, a judgment unit 27 for making various judgments, and a storage unit 28, typified by a hard disk or the like, for storing a program for performing a search to be executed, are all connected to an internal bus 21. Furthermore, a display unit 23 serving as a monitor for actually displaying information is connected to this internal bus 21.

[0018] The control unit 24 is a so-called central control unit for controlling each component implemented in the discrimination device 2 by transmitting a control signal via the internal bus 21. In addition, the control unit 24 transmits various control commands via the internal bus 21 in response to an operation via the operation unit 25.

[0019] The operation unit 25 is embodied by a keyboard or a touch panel, and an execution command for executing a program is input by the user. When the execution command is input by the user, the operation unit 25 notifies the control unit 24. The control unit 24, which receives the notification, executes the desired processing operation in cooperation with each component including the determination unit 27. The operation unit 25 may be embodied as the information acquisition unit 9 described above.

[0020] The judgment unit 27 is responsible for various judgments regarding the risk of intrusion of a suspicious person into a building structure. When performing an estimation operation, the judgment unit 27 reads out various pieces of information stored in the memory unit 28 as necessary information and various pieces of information stored in the database 3. The judgment unit 27 may be controlled by artificial intelligence. This artificial intelligence may be based on any well-known artificial intelligence technology.

[0021] The display unit 23 is configured with a graphic controller that creates a display image based on the control of the control unit 24. The display unit 23 is realized by, for example, a liquid crystal display (LCD) or the like.

[0022] When the storage unit 28 is configured with a hard disk, predetermined information is written to each address and is read out as necessary under the control of the control unit 24. The storage unit 28 also stores a program for carrying out the present invention. The program is read out and executed by the control unit 24.

[0023] The operation of the risk determination system 1 having the above-mentioned configuration will be described below.

[0024] The risk level determination system 1 is premised on the formation of a combination having reference image information and reference audio information, for example, as shown in Fig. 3. The reference image information is an image of the outside of an architectural structure taken by a camera installed in the architectural structure, its gate, entrance, garden, exterior, fence, barrier, parking space, service entrance, etc., and may be, for example, an image of a passerby walking on a road outside the architectural structure, an image of a resident of the architectural structure entering the architectural structure through a gate, or even an image of a suspicious person climbing over the fence of the architectural structure to enter. This image may be either a still image or a video.

[0025] The reference image information is composed of images captured by street cameras installed on public roads, cameras installed in vehicles capable of capturing images of public roads, etc., when determining the risk of a suspicious person appearing on a public road. Images captured by cameras installed in train stations may also be included.

[0026] Furthermore, when determining the risk of a suspicious person appearing inside a public transportation vehicle, the reference image information is composed of an image captured by a camera installed inside the vehicle.

[0027] The reference audio information is audio recorded through a microphone or the like at the actual shooting location when the reference image information is shot (or for a predetermined period of time before and after the shooting). The audio information may be analyzed by a known method or in detail using a neural network. The audio information may also be analyzed on the frequency axis.

[0028] In the example of Figure 3, for example, the reference image information is reference image information P11 to P13 captured at each location or each time series of the exterior of the building structure, and the reference audio information is reference audio information F to reference audio information I, etc.

[0029] Such reference image information and reference audio information are arranged as input data. The intermediate node shown in Fig. 3 is a combination of the reference image information and the reference audio information as input data. Each intermediate node is further connected to an output. In this output, the risk of a suspicious person intruding into the building structure as an output solution is displayed as a percentage.

[0030] Each combination (intermediate node) of the reference image information and the reference audio information is associated with each other through three or more levels of association with respect to the risk level as the output solution. The reference image information and the reference audio information are arranged on the left side through the association levels, and each risk level is arranged on the right side through the association levels. The association level indicates the degree of association with which risk level is highly related to the reference image information and the reference audio information arranged on the left side. In other words, the association level is an index indicating the likelihood that each reference image information and the reference audio information are associated with what risk level, and indicates the accuracy in selecting the most likely risk level from the reference image information and the reference audio information. In the example of FIG. 3, association levels w13 to w22 are shown. As shown in Table 1 below, w13 to w22 are expressed on a 10-point scale. The closer to 10, the higher the degree of correlation between the combinations as intermediate nodes and the risk as outputs. Conversely, the closer to 1, the lower the degree of correlation between the combinations as intermediate nodes and the risk as outputs.

[0031] [Table 1]

[0032] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 3. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference audio information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 3 by analyzing these.

[0033] For example, the reference image information P11 is an image of an inhabitant of the building structure entering the building structure through a gate. In addition, when the sound at the time of capturing the image is the reference sound information F, whether or not an incident occurred that could cause harm to the inhabitant of the building structure in the previous data, whether or not the danger level was such that an incident could occur even if it did not result in an incident, or whether or not the building structure was particularly safe, etc. are extracted. These data can be obtained from the record of image data captured by a camera in the past for the reference image information, and the reference sound information may be extracted from sound data recorded at that time. The degree of danger may be quantified by having multiple people visually view the reference image information and tallying up the results of a questionnaire survey on the degree of danger.

[0034] This analysis may be performed by artificial intelligence. In such a case, for example, when the reference image information P11 and the reference audio information F are used, past data is analyzed to determine whether an incident actually occurred or whether the risk was high even if the incident did not result in an incident. The more incidents there are or the higher the risk of an incident, the higher the correlation degree leading to a high-risk output is set, and the fewer incidents there are, the higher the correlation degree leading to a low-risk output is set. In the example of the intermediate node 61a linked when the reference image information P11 and the reference audio information H are used, the output is linked to a risk level of 90% and a risk level of 30%, but since the risk level is extremely high from previous cases, the correlation degree of w13 linked to a risk level of 90% is set to 7 points, and the correlation degree of w14 linked to a risk level of 30% is set to 2 points.

[0035] The correlation shown in Fig. 3 may be configured by a node of a neural network in artificial intelligence. That is, the weighting coefficient for the output of the node of the neural network corresponds to the correlation. Also, it is not limited to a neural network, and may be configured by any decision-making factor that constitutes artificial intelligence.

[0036] 3, node 61b is a node for a combination of reference image information P11 and reference audio information F, with a degree of association of 60% risk being w15 and a degree of association of 0% risk being w16. Node 61c is a node for a combination of reference image information P12 and reference audio information G and further reference audio information I, with a degree of association of 30% risk being w17 and a degree of association of 70% risk being w18.

[0037] Such correlation becomes what is called learned data in artificial intelligence. After creating such learned data, when actually determining a new risk level, the learned data described above will be used to determine the risk level. In such a case, new image information is obtained and audio information is also obtained. The image information corresponds to reference image information, and the audio information corresponds to reference audio information.

[0038] The newly acquired image information is captured by the camera of the information acquisition unit 9. This capture is the same as the image of the outside of the building structure captured to obtain the above-mentioned reference image information. Furthermore, the capture conditions (capture angle, angle of view, resolution) are not necessarily required to be all the same as those to obtain the above-mentioned reference image information.

[0039] The voice information is acquired via the information acquisition unit 9 having a recording function.

[0040] In this way, the risk level at the time when the newly acquired image information is actually acquired is calculated based on the newly acquired image information and the voice information. In this case, the previously acquired correlation level shown in FIG. 3 (Table 1) is referenced. For example, when the newly acquired image information is the same as or similar to P12 and is the acquired voice information I, the node 61d is associated through the correlation level, and this node 61d is associated with "risk level 60%" at correlation level w19 and with "risk level 70%" at correlation level w20. In this case, the "risk level 60%" with the highest correlation level is selected as the optimal solution. However, it is not essential to select the one with the highest correlation level as the optimal solution, and the "risk level 70%" with a low correlation level but a correlation level itself that is recognized may be selected as the optimal solution. In addition, it is of course possible to select an output solution that is not connected to the arrow, and any other priority order may be selected as long as it is based on the correlation level.

[0041] In this way, the selection of these optimal solutions can be realized through artificial intelligence that uses the trained model shown in Fig. 3 and outputs an output solution (risk level) when input data (image information, audio information) for which a solution is actually desired is input. However, the present invention does not necessarily require the use of artificial intelligence, and may be realized in any form that uses three or more levels of correlation between a combination having reference image information and reference audio information and the risk level for that combination.

[0042] Moreover, examples of the degrees of association w1 to w12 extending from the input are shown in Table 2 below.

[0043] [Table 2]

[0044] The intermediate node 61 may be selected based on the degrees of association w1 to w12 extending from this input. In other words, the greater the degrees of association w1 to w12, the heavier the weighting in the selection of the intermediate node 61. However, the degrees of association w1 to w12 may all be the same value, and the weighting in the selection of the intermediate node 61 may all be the same. The risk determination system 1 may be embodied in a form shown in FIG.

[0045] The reference housing structure information is data that indicates the structure of the house, such as a photograph of the entire house taken from the air, a photograph of the house taken from the road side, electronic data of the completion drawings of the house, and a structural drawing of the entire house shown on a map.

[0046] In the example of Figure 4, for example, the reference image information is reference image information P11 to P13 taken of the exterior of a house at each location or for each time series, and the reference house structure information is structure F to structure I, etc.

[0047] As input data, such reference image information and reference house structure information are arranged side by side. The intermediate node shown in Figure 4 is a combination of the reference image information as input data and the reference house structure information. Each intermediate node is further connected to an output. In this output, the risk of a suspicious person breaking into the house is displayed as a percentage as an output solution.

[0048] Each combination (intermediate node) of the reference image information and the reference house structure information is related to each other through three or more levels of correlation with respect to the risk level as the output solution. The reference image information and the reference house structure information are arranged on the left side through the correlation, and each risk level is arranged on the right side through the correlation. The correlation indicates the degree to which risk level is highly related to the reference image information and the reference house structure information arranged on the left side. In other words, the correlation is an index indicating what risk level each reference image information and the reference house structure information is likely to be linked to, and indicates the accuracy in selecting the most likely risk level from the reference image information and the reference house structure information. In the example of FIG. 4, the correlations w13 to w22 are shown. As shown in Table 1 below, w13 to w22 are expressed on a 10-point scale. The closer to 10, the higher the degree of correlation between the combinations as intermediate nodes and the risk as outputs. Conversely, the closer to 1, the lower the degree of correlation between the combinations as intermediate nodes and the risk as outputs.

[0049] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 4. That is, when discriminating the actual degree of danger, the discrimination device 2 accumulates reference image information, reference approach information, and data on the degree of danger in each case, and creates the correlations shown in Fig. 4 by analyzing these.

[0050] For example, the reference image information P11 is an image of a resident of a house entering the house through a gate. In addition, when the structure of the house to be discriminated is F, whether or not an incident that caused harm to the resident of the house occurred in previous data, whether or not the danger level was such that an incident could occur even if it did not result in an incident, or whether or not the house was particularly safe, etc. are extracted. These data can be obtained from the record of image data taken by a camera in the past for the reference image information, and the reference structure information may be extracted from the completion drawing or map data recorded by the house manufacturer or the government office, or from the image data of the house taken from the air or from the road in the past. The degree of danger may be quantified by visually checking the reference image information by multiple people and tallying up the results of a questionnaire survey on the degree of danger.

[0051] This analysis may be performed by artificial intelligence. In such a case, for example, when the reference image information P11 and the structure F are used, past data is analyzed to determine whether an incident actually occurred or whether the risk was high even if the incident did not result in an incident. The more incidents there are or the higher the risk of an incident, the higher the correlation degree leading to a high-risk output is set, and the fewer incidents there are, the higher the correlation degree leading to a low-risk output is set. In the example of the intermediate node 61a linked when the reference image information P11 and the structure H are used, the output is linked to a risk level of 90% and a risk level of 30%, but since the risk level is extremely high from previous cases, the correlation degree of w13 leading to a risk level of 90% is set to 7 points, and the correlation degree of w14 leading to a risk level of 30% is set to 2 points.

[0052] The correlation shown in Fig. 4 may be configured by a node of a neural network in artificial intelligence. That is, the weighting coefficient for the output of the node of the neural network corresponds to the correlation. Also, it is not limited to a neural network, and may be configured by any decision-making factor that constitutes artificial intelligence.

[0053] 4, node 61b is a node for a combination of structure F of the reference home structure information with reference image information P11, with a degree of association of a risk of 60% being w15 and a degree of association of a risk of 0% being w16. Node 61c is a node for a combination of structure G (e.g., structural information on the layout of a garden) of the reference home structure information with structure I (e.g., structural information on the layout of a fence) with reference image information P12, with a degree of association of a risk of 30% being w17 and a degree of association of a risk of 70% being w18.

[0054] This degree of correlation is what is called learned data in artificial intelligence. After creating this learned data, when actually determining a new level of risk, the learned data described above will be used to determine the level of risk. In such cases, new image information is acquired and housing structure information is also acquired. The image information corresponds to reference image information, and the housing structure information corresponds to reference housing structure information.

[0055] The newly acquired image information is captured by the camera of the information acquisition unit 9. This capture is the same as the image of the exterior of the house captured to obtain the above-mentioned reference image information. Furthermore, the capture conditions (capture angle, angle of view, resolution) are not necessarily required to be all the same as those to obtain the above-mentioned reference image information.

[0056] The housing structure information is acquired by accessing a database in which the housing structure information of the house for which the risk level is to be determined is recorded. For example, if the housing structure information is recorded in the database in association with an address, the housing structure information linked to the address may be read out by inputting the address.

[0057] In this way, the risk level at the time when the newly acquired image information is actually obtained is calculated based on the newly acquired image information and the approaching train information. In such a case, the previously acquired correlation level shown in FIG. 4 (Table 1) is referenced. For example, when the newly acquired image information is the same as or similar to P12, and the acquired house structure information is structure I, the node 61d is associated through the correlation level, and this node 61d is associated with "risk level 60%" at correlation level w19 and with "risk level 70%" at correlation level w20. In such a case, the "risk level 60%" with the highest correlation level is selected as the optimal solution. However, it is not essential to select the one with the highest correlation level as the optimal solution, and the "risk level 70%" with a low correlation level but a correlation level itself that is recognized may be selected as the optimal solution. In addition, it is of course possible to select an output solution that is not connected to the arrow, and any other priority order may be selected as long as it is based on the correlation level.

[0058] In this way, the selection of these optimal solutions can be realized through artificial intelligence that uses the trained model shown in Fig. 4 and outputs an output solution (risk level) when input data (image information, home structure information) for which a solution is actually sought is input. However, the present invention does not require the use of artificial intelligence, and may be realized in any form that uses three or more levels of correlation between a combination of reference image information and reference home structure information and the risk level for that combination.

[0059] In addition, when determining the risk of a suspicious person appearing on a public road, a correlation may be formed between the reference map structure information and the reference image information on the map and road structure of the public road that was photographed, instead of the reference house structure information. The reference map structure information here is information on the map that includes at least the range in which the reference image information is actually photographed. Since there are areas where alleys are blind spots for crimes when viewed on a map, and in busy areas, etc., where crimes are likely to occur, the purpose is to increase the accuracy of determining the risk by including a map that includes such information. The information on the road structure is, for example, composed of multi-level intersections and pedestrian bridges, and various structures such as stairs, escalators, moving walkways, branches, crossroads, etc. are adopted in the station premises. The information on this structure may be further combined with map information to be embodied as map information within the station premises. This makes it possible to obtain the positional relationship of shops within the station premises as the reference map structure information.

[0060] In such a case, in addition to the image information, map structure information on the structure of the map and roads that includes at least the range in which the image information is captured is obtained. The details of the map structure information are the same as those of the reference map structure information. Then, a solution search is performed based on the reference map structure information that corresponds to the map structure information.

[0061] FIG. 5 shows an example in which three or more levels of correlation are set between a combination having the above-mentioned reference image information and reference location information and a risk level for the combination.

[0062] As input data, such reference image information and reference location information are arranged side by side. The intermediate node shown in FIG. 5 is a combination of the reference image information and the reference location information as input data.

[0063] The reference location information includes all information for identifying the location of the building structure. The reference location information includes address information, map information, and also images of the surrounding environment including the building structure. The reference location information also includes the status of buildings on the land adjacent to the building structure and what purpose the land adjacent to the building structure is used for, such as a store, building, restaurant, vacant lot, parking lot, school, etc. In other words, if the adjacent land is vacant, it is considered that it is easy for someone to enter the building structure from the vacant lot, and this is also a factor that governs the risk level. In addition, when determining the risk level of a suspicious person appearing on a public road, the information is composed of information for identifying the location on the public road, and includes information on the status of buildings on the land adjacent to the building structure on the public road and what purpose the land adjacent to the building structure is used for, such as a store, building, restaurant, vacant lot, parking lot, school, etc.

[0064] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 5. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference location information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 5 by analyzing these.

[0065] In the example of correlation degrees shown in Figure 5, node 61b is a node where the reference location information is location J for the reference image information P11, and the correlation degree for a risk level of 60% is w15 and the correlation degree for a risk level of 0% is w16.

[0066] Similarly, when such a degree of association is set, new image information is acquired and location information is also acquired. The image information corresponds to the reference image information, and the location information corresponds to the reference location information.

[0067] The location information is acquired by accessing a database in which audio information of a building structure for which a risk level is to be determined is recorded. For example, if location information is associated with an address and recorded in a database, the location information linked to the address (such as an adjacent restaurant being relatively bright, or an adjacent parking lot where suspicious people are likely to enter) may be read out by inputting the address. The risk level of a public road can be similarly searched, and the location information linked to the address (such as an adjacent restaurant being relatively bright, or an adjacent parking lot where suspicious people are likely to enter) may be read out by inputting each address adjacent to the public road.

[0068] In calculating the risk level, the previously acquired correlation shown in Fig. 5 is referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the location information is location M, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation level w17 and a risk level of 70% at correlation level w18. Based on the correlation results w17 and w18, the risk level at the time when the new image information and approach information are actually acquired is calculated.

[0069] FIG. 6 shows an example in which three or more levels of correlation are set between a combination having the above-mentioned reference image information and reference at-home pattern information, and the risk level for that combination.

[0070] As input data, such reference image information and reference at-home pattern information are arranged side by side. The intermediate node shown in Figure 6 is a combination of reference location information and reference image information as input data.

[0071] The reference at-home pattern information includes all information for identifying the at-home pattern of the resident residing in the building structure. The reference at-home pattern information is, for example, a pattern of the time periods when the resident is at home and the time periods when the resident is out, based on the amount of power used in the building structure and images of the resident entering and leaving taken by a camera installed at the entrance or gate. For each time period on each day of the week between Monday and Sunday, the time periods when the resident is at home and the time periods when the resident is out are patterned based on the above-mentioned amount of power used and images. At this time, the above-mentioned amount of power used and images may be machine-learned in advance to determine the pattern.

[0072] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 6. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference at-home patterns, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 6 by analyzing these.

[0073] In the example of correlation degrees shown in Figure 6, node 61b is a node where the reference location information is location J for reference image information P11, and the correlation degree for a risk level of 60% is w15 and the correlation degree for a risk level of 0% is w16.

[0074] Similarly, when such a degree of association is set, new image information is acquired and at-home pattern information is also acquired. The image information corresponds to the reference image information, and the at-home pattern information corresponds to the reference at-home pattern information.

[0075] The at-home pattern information is acquired by accessing a database in which the at-home pattern information of the building structure for which the risk level is to be determined is recorded. For example, if the at-home pattern information is associated with an address and recorded in the database, by inputting the address, the at-home pattern information linked to the address (Monday to Friday, out from 9:00 to 21:00 and at home the rest of the time, Saturday and Sunday, out from 10:00 to 15:00 and at home the rest of the time, etc.) may be read out.

[0076] In calculating the risk level, the previously acquired correlation degrees shown in Fig. 6 are referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the at-home pattern information is at-home pattern R, the combination is associated with node 61c, and this node 61c is associated with a risk level of 30% at correlation degree w17 and a risk level of 70% at correlation degree w18. Based on the results of such correlation degrees w17 and w18, the risk level at the time when the new image information and approach information are actually acquired is calculated.

[0077] Figure 7 shows an example in which a combination of the above-mentioned reference image information and reference time zone information indicating the time zone at the time the reference image information was captured is set with three or more levels of correlation between the risk level for the combination.

[0078] The input data includes such reference image information and reference time period information. The intermediate node shown in FIG. 7 is a combination of the reference image information and the reference time period information as input data.

[0079] The reference time zone information means the time zone at the time the reference image information was captured, and may be not only a specific time, but also a range of time, such as 10:10 to 10:20.

[0080] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 7. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference at-home patterns, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 7 by analyzing these.

[0081] In the example of correlation shown in Fig. 7, node 61b is a node that combines reference image information P11 with reference time period information time period S, and has a correlation degree of 60% of risk w15 and a correlation degree of 0% of risk w16. In other words, there are time periods when crimes are rare and times when they are common. In particular, if there are few suspicious intrusions during the day and many at night, the risk degree is analyzed in advance in relation to each of the reference image information for each of the time periods S to V, ..., and is stored in association with the correlation degree as the intermediate node 61.

[0082] Similarly, when such a degree of association is set, new image information is acquired and at-home time period information is acquired. The image information corresponds to the reference image information, and the time period information corresponds to the reference at-home time period information.

[0083] To obtain time zone information, the time zone at the time of capturing image information of the building structure for which the degree of danger is to be determined is extracted. The extracted time zone information is then compared with the reference time zone information to perform the determination. In such a case, a database in which the reference time zone information is recorded is accessed. For example, if the time zone information is 9:20 to 9:30 (time zone V), the degree of danger is obtained via time zone V (9:20 to 9:30) of the same reference time zone information.

[0084] In calculating the risk level, the previously acquired correlation shown in Fig. 7 is referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the time period information is time period V, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation level w17 and a risk level of 70% at correlation level w18. Based on the correlation results w17 and w18, the risk level at the time when the new image information and approach information are actually acquired is calculated.

[0085] The same method can also be used to learn reference time zone information for determining the risk of a suspicious person appearing on a public road or inside a public transportation vehicle, making it possible to search for a solution using the newly acquired time zone information.

[0086] FIG. 8 shows an example in which a combination having reference image information, reference audio information, and further reference person information is set, and three or more levels of correlation are set between the combination and the risk level.

[0087] The reference person information is information extracted from a person depicted in the reference image information. The person information is facial features (face outline, facial features, hairstyle, whether or not the person wears glasses, wrinkles, scars, etc.), age, sex, etc. estimated from the facial features. The facial features may be extracted using a neural network. In this case, well-known methods may be used, such as (1) extracting edges from the image data of the face part, (2) extracting skin-colored areas from the image data of the face part, (3) extracting areas corresponding to the skin-colored areas extracted in (2) from the edge image generated in (1), (4) calculating cumulative histograms in the vertical and horizontal directions from the edge image corresponding to the skin-colored areas, and (5) inputting the number of pixels of the cumulative histograms whose brightness is equal to or exceeds a predetermined threshold value as input data for the neural network (NN). In addition, (6) the age and sex of the visitor may be estimated by the NN based on the input data and the teacher data.

[0088] In such a case, the degree of association is expressed as a set of combinations having reference image information, reference audio information, and reference person information as intermediate nodes 61a to 61e, as described above, as shown in FIG. 8.

[0089] 8, for example, node 61c is associated with reference image information P12 at a degree of association w3, reference audio information "structure G" at a degree of association w7, and "person β (for example, the person information is a wife living next door who frequently visits)" as reference personal information at a degree of association w11. Similarly, node 61e is associated with reference image information P13 at a degree of association w5, reference audio information G at a degree of association w8, and "person α (for example, a man in his 40s who has never been met before)" as reference personal information at a degree of association w10. Even if a person is not registered in the reference personal information, the fact that he or she is not registered may be treated as one piece of reference personal information.

[0090] Even when such a correlation is set, the risk level at the time when the newly acquired image information is actually acquired is calculated based on the newly acquired image information, audio information, and person information. The person information here is extracted from the actually newly acquired image information by performing image analysis using a method similar to that for extracting the reference person information.

[0091] In calculating this risk level, the previously acquired correlation degrees shown in Fig. 8 are referenced. For example, if the acquired image is identical or similar to the reference image information P12, the reference audio information is G, and the extracted person information is person β, the combination is associated with node 61c, and this node 61c is associated with a risk level of 30% at correlation degree w17 and a risk level of 70% at correlation degree w18. Based on the correlation results w17 and w18, the risk level at the time when the new image information and approach information are actually acquired is calculated.

[0092] If a person is not registered in the reference person information, the person is assumed to be person β, and even if the extracted person information is not registered in the reference person information, the solution search is performed assuming that person β has been entered.

[0093] In addition, for a combination having reference image information and reference location information, a combination having reference image information and reference at-home pattern information, and a combination having reference image information and reference time information, the degree of association may be defined by further including a combination having this reference person information. Also, the degree of association may be defined for a combination having reference image information and reference person information.

[0094] FIG. 9 shows an example in which a combination of the above-mentioned reference image information and reference audio information, as well as reference security service information, is furthermore set with three or more levels of correlation between the combination and the risk level.

[0095] The reference security service information is related to whether or not the building structure has a contract for security services from a security company, and the specific contract details, such as whether security guards from the security company will patrol the building structure if a door or window is broken.

[0096] In such a case, the degree of correlation is expressed as a set of combinations having reference image information, reference audio information, and reference security service information as intermediate nodes 61a to 61e, as described above, as shown in Figure 9.

[0097] For example, in FIG. 9, the node 61c is associated with the reference image information P12 at a degree of association w3, the reference audio information G at a degree of association w7, and the reference security service information "uncontracted" at a degree of association w11.

[0098] Even when such correlation is set, the degree of danger at the time when the newly acquired image information is actually acquired is calculated based on the newly acquired image information, audio information, and security service information. The security service information here refers to whether or not the target building structure actually has a contract for security services from a security company, or the contract status regarding the specific contents of the contract.

[0099] In calculating this risk level, the previously acquired correlation degree shown in Fig. 9 is referenced. For example, if the acquired image is identical or similar to the reference image information P12, the reference audio information G, and the extracted security service information is "uncontracted", the combination is associated with node 61c, and this node 61c is associated with a risk level of 30% at correlation degree w17 and a risk level of 70% at correlation degree w18. Based on the correlation results w17 and w18, the risk level at the time when the new image information and approach information are actually acquired is calculated.

[0100] Figure 10 shows an example in which a combination of the above-mentioned reference image information and reference weather information indicating the weather at the time the reference image information was captured is set with three or more levels of correlation between the combination and the risk level.

[0101] The reference weather information indicates information such as the weather (sunny, cloudy, rainy), disasters (typhoons, heavy rain, etc.), temperature, humidity, etc. on the time of the photograph. On rainy days, fewer people are out and about, and it may be difficult for noises to be detected, making it easier for intruders to break in, so this is also added as an explanatory variable.

[0102] As input data, such reference image information and reference weather information are arranged side by side. The intermediate node shown in FIG. 10 is a combination of reference time zone information and reference image information as input data.

[0103] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 10. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference weather information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 10 by analyzing these.

[0104] In the example of correlation shown in Fig. 10, node 61b is a node of a combination of reference image information P11 and reference weather information S, with a correlation degree of 60% being w15 and a correlation degree of 0% being w16. In other words, there are weather conditions in which crimes are unlikely to occur due to intrusion of suspicious people and weather conditions in which crimes are likely to occur due to intrusion of suspicious people. In particular, when intrusion of suspicious people is unlikely on sunny days and conversely is likely to occur on rainy days, the risk degree is analyzed in advance in relation to each of the reference weather information S to V, ... in relation to the reference image information, and the correlation degree is stored as an intermediate node 61 in association with the risk degree.

[0105] Similarly, when such a correlation is set, new image information is acquired and weather information is also acquired. The image information corresponds to the reference image information, and the weather information corresponds to the reference weather information.

[0106] Weather information may be acquired by importing the weather at that time from data from the Japan Meteorological Agency, or by inputting the weather as understood by the user. The extracted weather information is then compared with reference weather information for judgment. In such cases, a database in which the reference weather information is recorded is accessed. For example, if the weather information is heavy rain, the degree of danger is obtained based on the weather information (heavy rain) of the same reference time period information.

[0107] In calculating the risk level, the previously acquired correlation degrees shown in Fig. 10 are referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the weather information is V, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation degree w17 and a risk level of 70% at correlation degree w18. Based on these correlation results, w17 and w18, the risk level at the time when the new image information and weather information are actually acquired is calculated.

[0108] FIG. 11 shows an example of using reference traffic volume information as a substitute for the above-mentioned reference weather information. The reference traffic volume information is information on the traffic volume of vehicles or pedestrians on roads around the building structure. This traffic volume is the number of vehicles or pedestrians passing by per unit time. It has already been reported that the more a building structure faces a road with a lot of traffic, the less likely it is that a suspicious person will enter, so this is included in the explanatory variables to determine the risk level. The same is true for risk detection of public roads, and it has been reported that the possibility of a suspicious person appearing changes depending on the number of pedestrians and vehicles passing by on the public road, so this is included in the explanatory variables to determine the risk level.

[0109] For example, assume that the reference traffic volume information S indicates a traffic volume of 100 people per unit time, and the reference traffic volume information T indicates a traffic volume of 1 person per unit time. In this case, if the risk level is low as in the reference traffic volume information S, a process for lowering the weighting of the risk level, in other words, a process for lowering the risk level itself is set in advance. On the other hand, if the risk level is high as in the reference traffic volume information T, a process for increasing the weighting of the risk level, in other words, a process for increasing the risk level itself is set in advance.

[0110] As input data, such reference image information and reference traffic volume information are arranged side by side. The intermediate node shown in FIG. 11 is a combination of the reference image information and the reference traffic volume information as input data.

[0111] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 11. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference traffic volume information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 11 by analyzing these.

[0112] In the example of correlation shown in Fig. 11, node 61b is a node of a combination of reference image information P11 and reference traffic information S, with a correlation degree of 60% being w15 and a correlation degree of 0% being w16. In other words, there are roads with heavy traffic and roads with light traffic. In particular, in cases where building structures facing roads with heavy traffic have few intruders, and conversely roads with light traffic have many intruders, the risk degree is analyzed in advance for each of the reference traffic information S to V, ... in relation to the reference image information, and the correlation degree is stored in association with the intermediate node 61.

[0113] Similarly, when such a correlation is set, new image information is acquired and traffic volume information is also acquired. The image information corresponds to the reference image information, and the traffic volume information corresponds to the reference traffic volume information.

[0114] The traffic volume information and reference traffic volume information may be directly based on data from traffic volume surveys conducted by municipalities, the country, or other organizations, or may be determined based on images of building structures on roads captured per unit time. In such cases, an inspector may count the number of vehicles and pedestrians captured in the images one by one, or well-known deep learning technology may be used to extract and identify vehicles and pedestrians, and the number of identified vehicles and pedestrians per unit time may be counted.

[0115] The extracted traffic information is then compared with the reference traffic information to make a judgment. In such a case, a database in which the reference traffic information is recorded is accessed. For example, if the traffic information is the same as or similar to the reference traffic information, the degree of risk is calculated based on the reference traffic information.

[0116] In calculating the risk level, the previously acquired correlation degrees shown in Fig. 11 are referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the traffic volume information is V, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation degree w17 and a risk level of 70% at correlation degree w18. Based on the correlation results w17 and w18, the risk level at the time when the new image information and weather information are actually acquired is calculated.

[0117] When searching for the danger level inside a public transportation vehicle, a solution search may be performed based on the degree of correlation between the danger level and a combination of reference passenger number information indicating the number of passengers inside the vehicle and reference image information as an alternative to reference traffic volume information. The more passengers inside the vehicle, the higher the incidence of theft and sexual crimes, so this is introduced as an explanatory variable. In this case, passenger number information on the number of passengers inside the vehicle to be newly determined is also obtained. The method of obtaining the reference passenger number information and passenger number information may be to capture an image of the vehicle interior with a camera and estimate the number of people from the image, or to obtain the information through a sensor or the like, or to actually count the number of people manually.

[0118] When new passenger number information is acquired, the degree of risk is searched for based on the corresponding reference passenger number information.

[0119] FIG. 12 is an example of using reference crime history information as a substitute for the above-mentioned reference weather information. The reference crime history information is information on the past crime history in the area of ​​the building structure. The crime history includes all crimes such as murder, housebreaking, burglary, kidnapping, robbery, and sexual crimes. This crime history may be quantified by the number of crimes per unit period, and the numerical value may be further weighted according to the severity of the crime. An adjustment is made so that the higher the crime history of an area, the higher the risk. The same applies to risk detection of public roads, and the past crime history in the area where the public road is laid may be included in the explanatory variables. The same applies to risk detection inside public transportation vehicles, and the past crime history in the area where the public transportation is laid may be included in the explanatory variables.

[0120] The crime history of each region may be information publicly available on the Internet, or may be extracted and input from articles reported on television, in newspapers, etc.

[0121] As input data, such reference image information and reference criminal history information are arranged side by side. The intermediate node shown in FIG. 12 is a combination of the reference image information and the reference criminal history information as input data.

[0122] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 12. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference criminal history information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 12 by analyzing these.

[0123] In the example of correlation shown in Fig. 12, node 61b is a node that combines reference image information P11 with reference criminal history information S, and the correlation level for a risk level of 60% is w15, and the correlation level for a risk level of 0% is w16. In other words, there are areas with many crime histories and areas with few. In particular, if building structures facing areas with many criminal histories have few intruders, and conversely, roads with few criminal histories have many intruders, the risk level of each of the reference criminal history information S to V, ... is analyzed in advance in relation to the reference image information, and the correlation level is stored in association with the reference image information as intermediate node 61.

[0124] Similarly, when such a degree of correlation is set, new image information is obtained and criminal history information is also obtained. The image information corresponds to the reference image information, and the criminal history information corresponds to the reference criminal history information.

[0125] The extracted criminal history information is then compared with the reference criminal history information to make a judgment. In such cases, a database in which the reference criminal history information is recorded is accessed. For example, if the criminal history information is the same as or similar to the reference criminal history information, the risk level is calculated based on the reference criminal history information.

[0126] In calculating the risk level, the previously acquired correlation levels shown in Fig. 12 are referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the criminal history information is V, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation level w17 and a risk level of 70% at correlation level w18. Based on these correlation levels w17 and w18, the risk level at the time when the image information and criminal history information are actually newly acquired is calculated.

[0127] In addition, for a combination having reference image information and reference location information, a combination having reference image information and reference at-home pattern information, and a combination having reference image information and reference time information, the degree of association may be defined by further including a combination having this reference security service information. Also, the degree of association may be defined for a combination having reference image information and reference security service information.

[0128] The example of FIG. 13 is an example in which three or more levels of correlation between reference image information and risk are used.

[0129] It is assumed that a correlation is formed in which the reference image information and the risk level are linked to each other. In the example of Fig. 13, the input data is, for example, the reference image information P11 to P13. Such reference image information as input data is linked to the output. In this output, the risk level is an output solution.

[0130] The reference image information is related to each other through three or more levels of correlation with respect to the risk level as the output solution. The reference image information is arranged on the left side through the correlation levels, and each risk level is arranged on the right side through the correlation levels.

[0131] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w19 as shown in Fig. 13. That is, when discriminating an actual search solution, the discrimination device 2 accumulates past data on the level of risk when reference image information was captured and acquired in the past, and creates the correlations shown in Fig. 13 by analyzing the accumulated data.

[0132] For example, it is assumed that the reference image information is P11. It is assumed that the risk level for such P11 is often 90%. By collecting and analyzing such data sets, the correlation between the reference image information P11 and the risk level becomes stronger.

[0133] This analysis may be performed by artificial intelligence. Also, image discrimination may be performed using deep learning. Also, the correlation shown in FIG. 13 may be configured by a node of a neural network in artificial intelligence. That is, the weighting coefficient for the output of the node of this neural network corresponds to the above-mentioned correlation. Also, it is not limited to a neural network, and may be configured by any decision-making factor that constitutes artificial intelligence.

[0134] In the process of constructing a trained model based on such correlations, reference housing structure information including housing structure information related to the building structure is also acquired. This reference housing structure information is not included in the correlations described above.

[0135] Such correlations are what is called "trained data" in artificial intelligence. After creating such trained data, a solution is searched for. In such a case, the captured image information is acquired, and the house structure information is also acquired.

[0136] First, the risk level is searched based on the newly acquired image information. In this case, the correlation level shown in FIG. 13, which is acquired in advance, is used. For example, if the newly acquired image information is the same as or similar to the reference image information P12, the risk level of 30% is associated with the correlation level w15 and the risk level of 60% is associated with the correlation level w16. In this case, the risk level of 30% with the highest correlation level is selected as the optimal solution. However, it is not essential to select the one with the highest correlation level as the optimal solution, and the correlation level of 60% with a low correlation level but with a correlation level that is recognized as a solution may be selected. In addition, it is of course possible to select an output solution with no arrows connected to it, and the output solution may be selected based on any other priority level as long as it is based on the correlation level. In addition, the output solution to be selected is not limited to one, and two or more may be selected. In this case, two or more may be selected in order from the top of the correlation level, but this is not limited to this, and the output solution may be selected based on any other priority level of the correlation level.

[0137] The risk level calculated through the correlation level may be further modified or weighted based on the house structure information.

[0138] For example, assume that structure F in the reference housing structure information is a single-family home with an open exterior and no surrounding fences, and structure G in the reference housing structure information is a structure surrounded by a 2m-high fence. In this case, for a structure with a low degree of danger like structure G in the reference housing structure information, a process is set up in advance to lower the weighting of the danger level, in other words, to lower the danger level itself. On the other hand, for a structure with a high degree of danger like structure F in the reference housing structure information, a process is set up in advance to increase the weighting of the danger level, in other words, to increase the danger level itself.

[0139] After setting with the reference housing structure information in this way, if the actually acquired housing structure information is identical or similar to structure G of the reference housing structure information, a process is performed to lower the weighting of the risk level, in other words, a process is performed to lower the risk level itself. On the other hand, if the actually acquired housing structure information is identical or similar to structure F of the reference housing structure information, a process is performed to increase the weighting of the risk level, in other words, a process is performed to increase the risk level itself.

[0140] In such a case, when the safety is high, such as when the actually acquired housing structure information is identical or similar to structure G, a process for lowering the weight of the risk level may be performed, and the risk level may be determined as 0% without making a judgment using the above-mentioned correlation degree. When determining the risk level of a suspicious person appearing on a public road, reference map structure information on the map and road structure of the photographed public road is used as a substitute for reference housing structure information, and map structure information on the map and road structure that includes at least the range in which the image information is photographed is obtained. The details of the map structure information are the same as those of the reference map structure information. Then, a solution search is performed based on the reference map structure information corresponding to the map structure information.

[0141] In addition, the present invention is not limited to searching for a solution including the above-mentioned housing structure information, and it is not essential to determine the risk level based on the correlation alone, correct it based on the housing structure information, or change the weighting.

[0142] In other words, three or more levels of correlation with the risk level for the reference image information can be obtained in advance, and when determining a new risk level, image information can be obtained by photographing the target, and the previously obtained correlation can be referred to, and the risk level of a suspicious person entering a vehicle or appearing on a public road can be determined based on the new image information.

[0143] The same applies to the case where reference location information is used instead of the reference house structure information.

[0144] 14 shows an example of using reference location information instead of reference house structure information, and the same processing operation is executed in this case. The weighting of the risk level is changed depending on whether the location of the building structure is safe or not, depending on the location of the building structure.

[0145] Furthermore, in the case shown in FIG. 13, it goes without saying that reference home pattern information, reference time period information, reference security service information, and reference weather information may be applied instead of reference house structure information.

[0146] Also, Fig. 15 is an example of using reference traffic volume information as a substitute for reference housing structure information. Reference traffic volume information is information on the volume of vehicles or pedestrians passing on roads around a building structure. This traffic volume is the number of vehicles or pedestrians passing per unit time. It has already been reported that the more traffic a building structure faces a road with a lot of traffic, the less likely it is that a suspicious person will intrude into the building structure. Therefore, this is included as an explanatory variable to determine the degree of risk.

[0147] For example, assume that the reference traffic volume information F has a traffic volume of 100 people per unit time, and the reference traffic volume information G has a traffic volume of 1 person per unit time. In this case, when the risk level is low as in the reference traffic volume information F, a process for lowering the weighting of the risk level, in other words, a process for lowering the risk level itself is set in advance. On the other hand, when the risk level is high as in the reference traffic volume information G, a process for increasing the weighting of the risk level, in other words, a process for increasing the risk level itself is set in advance.

[0148] After setting the reference traffic information in this way, if the actually acquired traffic information is identical or similar to the reference traffic information G, the weighting of the risk level is increased, in other words, the risk level itself is increased. On the other hand, if the actually acquired traffic information is identical or similar to the reference traffic information F, the weighting of the risk level is decreased, in other words, the risk level itself is decreased.

[0149] In such a case, when the result of acquiring traffic information indicates that safety is high, such as when the actually acquired traffic information is identical or similar to the reference traffic information F, a process of lowering the weighting of the risk level may be performed, and the risk level may be determined to be 0% without making a judgment using the above-mentioned correlation degree.

[0150] The traffic volume information and reference traffic volume information may be directly based on data from traffic volume surveys conducted by municipalities, the country, or other organizations, or may be determined based on images of building structures on roads captured per unit time. In such cases, an inspector may count the number of vehicles and pedestrians captured in the images one by one, or well-known deep learning technology may be used to extract and identify vehicles and pedestrians, and the number of identified vehicles and pedestrians per unit time may be counted.

[0151] FIG. 16 is an example of using reference crime history information as an alternative to reference housing structure information. Reference crime history information is information on the past crime history in the area of ​​the building structure. Crime history includes all crimes such as murder, housebreaking, burglary, kidnapping, robbery, and sexual crimes. This crime history may be quantified based on the number of crimes per unit period, and the numbers may be further weighted according to the severity of the crime. An adjustment is made so that the risk level is higher in areas with higher crime histories.

[0152] The crime history of each region may be information publicly available on the Internet, or may be extracted and input from articles reported on television, in newspapers, etc.

[0153] For example, suppose that the reference crime history information F indicates that there has been only one burglary in the past ten years in that area. In this case, if the risk level is low, as in the reference crime history information F, the system is set up to lower the weighting of the risk level, in other words, to lower the risk level itself. On the other hand, if the reference crime history information G indicates a high risk level, such as ten burglaries in the past ten years, the system is set up to increase the weighting of the risk level, in other words, to increase the risk level itself.

[0154] After this setting with the reference criminal history information, if the actually acquired criminal history information is identical or similar to the reference criminal history information G, the weighting of the danger level is increased, in other words, the danger level itself is increased. On the other hand, if the actually acquired criminal history information is identical or similar to the reference criminal history information F, the weighting of the danger level is decreased, in other words, the danger level itself is decreased.

[0155] In such a case, when the result of obtaining the criminal history information is that the actually obtained criminal history information is identical or similar to the reference criminal history information F, and thus the safety is high, for example, a process of lowering the weighting of the risk level may be performed, and the risk level may be determined to be 0% without making a judgment using the above-mentioned correlation degree.

[0156] Also, reference time period information may be used as an alternative to the reference housing structure information shown in Fig. 13. Then, a solution search is performed in the same manner as described above based on the reference time period information corresponding to the newly acquired time period information. Also, when searching for the risk of a suspicious person appearing inside a public transportation vehicle, reference passenger number information may be used as an alternative to the reference housing structure information shown in Fig. 13. Then, a solution search is performed in the same manner as described above based on the reference passenger number information corresponding to the newly acquired passenger number information.

[0157] As shown in Fig. 13 to 16, the degree of association is not limited to the case where it is formed in relation to the reference image information and the risk level, but may be obtained in advance with three or more levels of association between a combination of reference image information and reference audio information recorded when the reference image information is photographed, and the risk level for the combination, as shown in Fig. 17. In addition, the risk level of a suspicious person's intrusion may be determined by referring to reference information linked to any of reference audio information, reference location information, reference at-home pattern information, reference time period information, reference security service information, reference weather information, reference traffic volume information, and reference crime history information.

[0158] In the example of Fig. 18, the same processing operation is performed when forming a correlation between the behavioral information that typifies a series of movements and poses of a person shown in the reference image information and the risk level. That is, three or more levels of correlation of the risk level for each combination described above including the reference behavioral information that extracts the person's behavior by analyzing the reference image information is acquired in advance. Next, when determining a new risk level, further behavioral information that extracts the person's behavior by analyzing image information acquired by photographing the outside of the building structure is acquired. The risk level is determined based on the reference behavioral information corresponding to this acquired behavioral information.

[0159] The reference behavior information and behavior information may utilize so-called posture estimation technology. The posture estimation technology may use deep learning, for example, to detect feature points of the nose, eyes, ears, neck, shoulders, elbows, wrists, waist, knees, and ankles using deep learning. Then, posture estimation is performed based on dynamic changes of these feature points. As free software for posture estimation, for example, OpenPose (registered trademark) developed by Carnegie Mellon University may be used.

[0160] The correlation degree obtained by replacing reference image information and image information with such reference behavior information and behavior information is not limited to the reference correlation degree shown in Figure 18, but can also be applied when forming combinations with other reference housing structure information, reference location information, reference home pattern information, reference time period information, reference security service information, reference weather information, reference traffic volume information, and reference crime history information. Also, a solution search may be performed only by the correlation degree replaced with the reference behavior information and behavior information. The present invention includes all embodiments embodied by replacing the reference image information and image information with such reference behavior information and behavior information.

[0161] As an alternative to the reference image information, reference sensing information obtained by sensing an intruder from outside the building structure using a sensor may be used, as shown in Fig. 19. This reference sensing information includes infrared sensors and laser sensors capable of detecting an intruder within the building structure premises, passive sensors that detect an intrusion by detecting the difference between a person's body temperature and the surrounding temperature, and any information obtained by magnet switches, glass breakage sensors, etc. that detect the opening and closing, breaking, and vibration of windows in the building structure.

[0162] Three or more levels of correlation of the risk level for each of the above combinations including such reference sensing information are obtained in advance. Next, when determining a new risk level, sensing information is obtained by sensing an intruder from outside the building structure using a sensor. The risk level is determined via the reference sensing information corresponding to such sensing information.

[0163] The correlation obtained by replacing the reference image information and image information with such reference sensing information and sensing information is not limited to the reference correlation shown in Figure 19, but can also be applied when forming combinations with other reference housing structure information, reference location information, reference home pattern information, reference time period information, reference security service information, reference weather information, reference traffic volume information, and reference crime history information.

[0164] In the present invention, as shown in Fig. 20, reference image information may be input as input data, a risk level may be output as output data, and at least one hidden layer may be provided between the input node and the output node for machine learning. The above-mentioned correlation level may be set in either one or both of the input node and the hidden layer node, which serves as a weighting for each node, and an output may be selected based on this. When this correlation level exceeds a certain threshold, the output may be selected.

[0165] According to the present invention having the above-mentioned configuration, it is possible to easily judge the risk of intrusion of a suspicious person into a building structure with little effort, without requiring any particular skill. Furthermore, according to the present invention, it is possible to judge the risk with higher accuracy than a human would. Furthermore, by configuring the above-mentioned correlation with artificial intelligence (neural network, etc.), it is possible to further improve the discrimination accuracy by learning this.

[0166] In addition, the present invention is characterized in that optimal properties and generation mechanisms are searched for through the correlation degree set to three or more stages. The correlation degree can be described, for example, by a numerical value from 0 to 100% in addition to the above-mentioned five stages, but is not limited thereto and may be configured in any stage as long as it can be described by a numerical value of three or more stages.

[0167] By searching for the most likely risk level based on the correlation level expressed in three or more numerical levels, it is possible to search and display in descending order of the correlation level in a situation where multiple possible candidates for increased risk are considered. If the correlation levels can be displayed to the user in descending order in this way, it is also possible to display the more likely risk levels preferentially, and to alert users to the increased risk.

[0168] If the danger level is high, the danger can be avoided by contacting a security service company or the police, or by immediately alerting the residents of the building structure. If the danger level is particularly high, safety can be ensured by alerting residents through voice or by generating sound outside to discourage suspicious people from entering. Furthermore, according to the present invention, the accuracy of detecting the danger level is high, so the police can be called only when it is truly necessary, without needlessly calling the police.

[0169] In addition, according to the present invention, it is possible to judge even a very low output discrimination result such as a correlation of 1% without overlooking it. Even an output discrimination result with a very low correlation is connected as a slight sign, and it is possible to alert the user that the discrimination result may be useful once in tens or hundreds of times.

[0170] In addition, the present invention discriminates based on the degree of association between a combination of two or more types of information, that is, reference information U and reference information V, as shown in Fig. 21. These reference information U and V are each piece of reference information including the above-mentioned reference image information.

[0171] At this time, as shown in Fig. 21, the output obtained for the reference information U may be used as input data as it is and associated with the output via an intermediate node 61 in combination with the reference information V. For example, after an output solution is obtained for the reference information U (reference image information) as shown in Fig. 3, this may be used as input as it is to search for an output by utilizing the degree of association with other reference information V.

[0172] Furthermore, according to the present invention, by performing a search based on such three or more levels of correlation, there is an advantage that the search policy can be determined by the way the threshold is set. If the threshold is set low, even if the correlation is 1% as described above, it is possible to pick up without omission, but there is a low possibility that a more appropriate discrimination result can be suitably detected, and there is a possibility that a lot of noise will be picked up. On the other hand, if the threshold is set high, there is a high possibility that the optimal risk level can be detected with a high probability, but there is a possibility that a suitable solution that usually has a low correlation and is ignored but appears once in tens or hundreds of times will be overlooked. It is possible to decide which one to emphasize based on the thinking of the user side or the system side, and it is possible to increase the degree of freedom in selecting such points to emphasize.

[0173] Furthermore, in the present invention, the above-mentioned correlation degree may be updated. This update may be made to reflect information provided via a public communication network such as the Internet. Furthermore, when new knowledge is found about the relationship between the input parameters and the output solution (risk level) based on a camera image taken in the building structure, acquired reference image information, reference location information, reference at-home pattern information, reference time information, reference voice information, reference security service information, etc., the correlation degree is increased or decreased according to the new knowledge.

[0174] In other words, this update corresponds to learning in artificial intelligence. Since new data is acquired and reflected in the learned data, it can be considered a learning action.

[0175] Furthermore, the process of initially constructing the trained model and the above-mentioned updates may use not only supervised learning, but also unsupervised learning, deep learning, reinforcement learning, etc. In the case of unsupervised learning, instead of loading and learning a data set of input data and output data, information equivalent to the input data (reference image information, reference location information, reference at-home pattern information, reference time information, reference audio information, reference security service information, etc.) may be loaded and trained, and the degree of association related to the output data may be formed automatically from there.

[0176] The degree of association may be updated manually or automatically by the system or user based on information available from public communication networks, or based on research data and papers by experts, academic presentations, newspaper articles, books, etc. Artificial intelligence may be utilized in these update processes.

[0177] Furthermore, the process of initially constructing the trained model and the above-mentioned updates may use not only supervised learning, but also unsupervised learning, deep learning, reinforcement learning, etc. In the case of unsupervised learning, instead of loading and learning a data set of input data and output data, information equivalent to the input data may be loaded and learned, and the degree of association related to the output data may be formed from the input data.

[0178] Furthermore, in the case where the danger level determination system 1 to which the present invention is applied is determined to be high, in addition to reporting the danger to the residents of the building structure, the management company, and even the security company, it is of course possible to generate an alarm or emit a voice message such as "Please do not enter without permission."

[0179] In the above embodiment, the case of detecting the danger of an intrusion of a suspicious person has been described as an example, but the present invention is not limited to this. The present invention can also detect the danger of an intrusion of animals other than humans, i.e., beasts, in a similar manner.

[0180] The beasts referred to here include all animals that are harmful to humans, such as bears, brown bears, wild boars, palm civets, raccoon dogs, wild dogs, cats, pyongs, etc. Animals are not limited to mammals, but also include birds such as crows.

[0181] Similarly, when detecting the risk of intrusion by such animals, a correlation between reference information and the risk is formed in advance, and when an actual intrusion is detected, various information such as image information is similarly obtained by a camera, and the risk is determined by referring to the correlation.

[0182] When referring to each of the above-mentioned reference information, a correlation is formed between the reference sighting history information showing the history of animal sightings in each area as a substitute for the reference crime history information, and the danger level. Then, during actual discrimination, sighting history information showing the history of animal sightings in that area is obtained, and a solution search is performed by referring to the correlation based on the reference sighting history information corresponding to this sighting history information.

[0183] This animal intrusion detection method can be applied not only to building structures, but also to cultivated land such as rice paddies and farmland.

[0184] In addition, the risk determination system 1 to which the present invention is applied can determine high-risk architectural structures, public roads, public transport facilities, and their areas. For architectural structures, public roads, public transport facilities, and their areas whose risk exceeds a certain threshold, for example, a layout plan may be proposed, such as displaying that more buses should be placed.

[0185] Hereinafter, a further description will be given of an embodiment that can be applied when determining the risk of a suspicious person appearing on a public road and when determining the risk of a suspicious person appearing inside a public transportation vehicle. Fig. 22 shows an example of using reference environmental information instead of the above-mentioned reference weather information. The reference environmental information is a parameter necessary for determining the quality of the environment of a public road, inside a train, or at a station, such as odor or amount of accumulated garbage. Odor can be measured using an odor sensor, and the amount of accumulated garbage can be analyzed by image analysis using a camera or by sensors installed on the road, inside a train, or in a garbage bin.

[0186] As input data, such reference image information and reference environment information are arranged side by side. The intermediate node shown in FIG. 22 is a combination of the reference image information and the reference environment information as input data.

[0187] The discrimination device 2 acquires in advance three or more stages of correlation w13 to w22 as shown in Fig. 22. That is, when discriminating the actual risk level, the discrimination device 2 accumulates reference image information, reference environment information, and data on the level of risk in each case, and creates the correlation levels shown in Fig. 22 by analyzing these.

[0188] In the example of correlation shown in Fig. 22, node 61b is a node of a combination of reference image information P11 and reference environment information S, with a correlation degree of 60% being w15 and a correlation degree of 0% being w16. In other words, there are areas with many crime histories and areas with few. In particular, building structures facing areas with many crime histories have few intruders, while roads with few crime histories have many intruders. In this case, the risk of each of the reference environment information S to V, ... is analyzed in advance in relation to the reference image information, and the correlation degree is stored as an intermediate node 61 in association with the reference image information.

[0189] When such a degree of association is set, similarly, new image information is acquired and environmental information is also acquired. The image information corresponds to the reference image information, and the environmental information corresponds to the reference environmental information.

[0190] The extracted environmental information is then compared with the reference environmental information to make a judgment. In such a case, a database in which the reference environmental information is recorded is accessed. For example, if the environmental information is identical or similar to the reference environmental information, the degree of risk is calculated based on the reference environmental information.

[0191] In calculating the risk level, the previously acquired correlation shown in Fig. 22 is referenced. For example, if the acquired image is identical or similar to the reference image information P12 and the environmental information is V, the combination is associated with node 61c, which is associated with a risk level of 30% at correlation level w17 and a risk level of 70% at correlation level w18. Based on the correlation results w17 and w18, the risk level at the time when the image information and environmental information are actually newly acquired is calculated.

[0192] Moreover, instead of identifying suspicious people on public roads, it is of course also possible to identify the danger level of animals. [Explanation of symbols]

[0193] 1. Risk assessment system 2 Discrimination device 21 Internal Bus 23 Display section 24 Control Unit 25 Control section 26 Communications Department 27 Estimation part 28 Memory section 61 nodes

Claims

1. A risk discrimination program for discriminating the risk of a suspicious person appearing on a public road consisting of a road or a station premises, an information acquisition step of acquiring person information by analyzing a newly captured image of a public road when determining a risk level, and voice information recorded at the time of capturing the image; a determination step of referring to reference person information obtained by analyzing an image of a public road to extract a person and a degree of association with a danger level of three or more stages, prioritizing information with a higher degree of association based on the reference person information corresponding to the person information obtained through the information obtaining step, and determining a danger level of a suspicious person appearing on the public road based on the obtained voice information, by the computer; A risk assessment program characterized by:

2. The above correlations are composed of nodes in a neural network in artificial intelligence.

2. The risk determination program according to claim 1,

Images