Risk discrimination program

Abstract

To detect a risk in advance and prevent an incident before the incident happens.SOLUTION: A risk determination program for determining a risk of a suspicious person appearing in a vehicle of public transport causes a computer, in determining the risk, to execute: an information acquisition step for acquiring person information, which is obtained by extracting persons by analyzing a newly captured image of the inside of the vehicle, and audio information of sound recorded during the image capture; and a determination step for referring three or more grades of a correlation degree between the risk and reference person information, which is obtained by extracting persons by analyzing the captured image of the inside of the vehicle, so as to determine the risk of a suspicious person appearing in the vehicle on the basis of the audio information acquired while prioritizing high grades of the correlation degree on the basis of the reference person information corresponding to the person information acquired through the information acquisition step.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a risk determination program for determining the risk of intrusion by suspicious persons or animals into a building structure, cultivated land, public road, etc.

Background Art

[0002] Conventionally, various attempts have been made to prevent the intrusion of suspicious persons into building structures such as houses and buildings occupied by companies. In particular, since single-family housing structures have many more intrusion routes compared to condominiums, it is necessary to take stronger measures to prevent the intrusion of suspicious persons in order to protect the safety of the residents. In recent years, damage caused by animals such as bears and wild boars has become a problem. In particular, since single-family housing structures have many more animal intrusion routes compared to condominiums, it is necessary to take stronger measures to prevent the intrusion of suspicious persons in order to protect the safety of the residents.

[0003] In recent years, cases of intrusion by suspicious persons or animals into public roads and cultivated land have occurred frequently, becoming a social problem.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the disclosed technology of Patent Document 1 described above, a technology for identifying a person from a face image by artificial intelligence is described, but it is not possible to quantitatively determine the risk of intrusion of suspicious persons into an actual building structure by utilizing artificial intelligence. By accurately predicting the actual risk, it is necessary to promptly contact a security company or the police when a danger is approaching the residents, and to avoid unnecessarily dispatching security guards or police officers by the security company when the actual risk is not so high.

[0006] Such a perspective is not particularly described in the disclosed technology of Citation Document 1. The same applies to damage caused by wild animals.

[0007] Therefore, the present invention has been devised in view of the above-described problems, and its object is to provide a risk determination program that automatically determines the risk level using artificial intelligence in order to detect in advance the risk of intrusion by suspicious persons or animals into a building structure, cultivated land, or public road and protect the safety of residents.

Means for Solving the Problems

[0008] The risk determination program according to the present invention is a risk determination program for determining the risk of appearance of a suspicious person in a vehicle of a public transportation vehicle. When determining the risk, an information acquisition step of acquiring person information obtained by extracting a person by analyzing an image newly taken inside the vehicle and voice information recorded at the time of the shooting, a reference person information obtained by extracting a person by analyzing an image taken inside the vehicle, and a reference person information corresponding to the person information obtained through the information acquisition step with a three-level or higher correlation with the risk level, giving priority to those with a higher correlation, and a determination step of determining the risk of appearance of a suspicious person in the vehicle based on the acquired voice information are executed by a computer.

Effects of the Invention

[0009] It is possible to automatically and highly accurately determine the risk level using artificial intelligence in order to detect in advance the risk of intrusion by suspicious persons or animals into a public road or the like and prevent incidents.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Embodiments for Carrying Out the Invention

[0011] Hereinafter, a risk determination program to which the present invention is applied will be described in detail with reference to the drawings.

[0012] FIG. 1 is a block diagram showing the overall configuration of a risk determination system 1 in which a risk determination program to which the present invention is applied is implemented. The risk determination system 1 includes an information acquisition unit 9, a determination device 2 connected to the information acquisition unit 9, and a database 3 connected to the determination device 2.

[0013] The information acquisition unit 9 is a device for a person using this system to input various commands and information. Specifically, it is composed of a keyboard, buttons, a touch panel, a mouse, a switch, etc. The information acquisition unit 9 is not limited to a device for inputting text information, and may be composed of a device such as a microphone that can detect sound and convert it into text information. Also, the information acquisition unit 9 may be configured as an imaging device capable of taking images such as a camera. The information acquisition unit 9 may be composed of a scanner having a function of recognizing character strings from documents on paper media. Further, the information acquisition unit 9 may be integrated with the determination device 2 described later. The information acquisition unit 9 outputs the detected information to the determination device 2.

[0014] The database 3 stores information regarding the risk of suspicious persons intruding into a building structure, such as previous incidents that have occurred regarding the intrusion of suspicious persons into the building structure, or cases where no incident occurred but the risk was high. The database 3 stores reference image information previously captured by the camera that actually constitutes the information acquisition unit 9, reference structure information including the structural information regarding the building structure, reference location information for identifying the location of the building structure, reference occupancy pattern information indicating the occupancy pattern of the residents of the building structure, reference time zone information indicating the time zone at the time of shooting the reference image information, reference person information obtained by extracting a person by analyzing the reference image information, reference audio information recorded at the time of shooting the reference image information, reference security service information indicating the contract status of the security service of the security company for the building structure, etc. In addition to this, the database 3 stores reference traffic volume information indicating the traffic volume of the roads around the building structure, and reference criminal history information regarding the past criminal history in the area of the building structure. The building structure includes, in addition to single-family houses, condominiums, mansions, apartments and other houses, buildings occupied by companies, schools, hospitals, public facilities, amusement facilities, etc. Note that the present invention can be applied not only when determining the risk of suspicious persons intruding into a building structure, but also when determining the risk of suspicious persons appearing on a public road. Here, the public road referred to includes, in addition to roads (including not only sidewalks but also crosswalks), underground passages leading to railway stations and passages within the precincts of railway stations. The risk of a suspicious person appearing among pedestrians walking on this public road is determined (hereinafter referred to as the risk of a suspicious person appearing). This risk of a suspicious person appearing may indicate the risk for a person who is actually taking suspicious actions among pedestrians, or may indicate the probability that such a person will appear later even if there is no such person actually taking such actions among pedestrians. Also, even if there are no pedestrians in the actually photographed area, it may indicate the possibility that a pedestrian will enter the photographed area later and a suspicious person will appear among the pedestrians.

[0015] Furthermore, the present invention can also be applied when determining the risk of the appearance of suspicious persons inside public transportation vehicles. Here, the public transportation vehicles referred to include trains (including monorails, rail buses, bullet trains, ropeways, cable cars, etc.), buses, taxis, and the like. The risk of the appearance of suspicious persons inside such public transportation vehicles is determined (hereinafter referred to as the risk of the appearance of suspicious persons). This risk of the appearance of suspicious persons may indicate the risk for a person who is actually acting suspiciously inside the vehicle, or may indicate the probability that such a person will appear later even if there is no such person actually acting inside the vehicle at present. Also, even if there are no passengers in the actually photographed area, it may indicate the possibility that passengers will enter the photographed area later and a suspicious person will appear among those passengers.

[0016] The discrimination device 2 is composed of, for example, an electronic device such as a personal computer (PC), but in addition to the PC, it may be embodied by any other electronic device such as a mobile phone, smartphone, tablet terminal, wearable terminal, etc. The user can determine the risk of the intrusion of suspicious persons into the building structure by obtaining the search solution by this discrimination device 2.

[0017] FIG. 2 shows a specific configuration example of the discrimination device 2. This discrimination device 2 includes a control unit 24 for controlling the entire discrimination device 2, an operation unit 25 for inputting various control commands via operation buttons, a keyboard, etc., a communication unit 26 for performing wired or wireless communication, a judgment unit 27 for performing various judgments, and a storage unit 28 represented by a hard disk, etc., for storing a program for performing the search to be executed, which are respectively connected to an internal bus 21. Further, a display unit 23 as a monitor for actually displaying information is connected to this internal bus 21.

[0018] The control unit 24 is a so-called central control unit for controlling each component implemented in the discrimination device 2 by transmitting a control signal via the internal bus 21. Further, this control unit 24 transmits various control commands via the internal bus 21 in response to an operation via the operation unit 25.

[0019] The operation unit 25 is embodied by a keyboard or a touch panel, and an execution command for executing a program is input from a user. When this execution command is input from the user, this operation unit 25 notifies the control unit 24 of this. The control unit 24 that has received this notification will execute a desired processing operation in cooperation with each component, starting with the determination unit 27. This operation unit 25 may be embodied as the information acquisition unit 9 described above.

[0020] The determination unit 27 is responsible for making various determinations regarding the risk of an intruder entering a building structure. When executing the estimation operation, this determination unit 27 reads out various information stored in the storage unit 28 and various information stored in the database 3 as necessary information. This determination unit 27 may be controlled by artificial intelligence. This artificial intelligence may be based on any well-known artificial intelligence technology.

[0021] The display unit 23 is composed of a graphic controller that creates a display image based on the control by the control unit 24. This display unit 23 is realized by, for example, a liquid crystal display (LCD) or the like.

[0022] When the storage unit 28 is composed of a hard disk, based on the control by the control unit 24, predetermined information is written to each address and read out as necessary. Further, a program for executing the present invention is stored in this storage unit 28. This program will be read out and executed by the control unit 24.

[0023] The operation of the risk discrimination system 1 having the above-described configuration will be described.

[0024] In the risk determination system 1, for example, as shown in FIG. 3, it is assumed that a combination having reference image information and reference audio information is formed. The reference image information is an image obtained by photographing the exterior of a building structure with a camera installed on the building structure, the door of the building structure, the entrance, the garden, the exterior structure, the fence, the parking space, the passageway, etc. For example, it includes various images such as an image of a passerby passing on the road outside the building structure, an image of a resident of the building structure entering the building through the door, and an image of a suspicious person trying to climb over the fence of the building structure. This image may be either a still image or a moving image.

[0025] In addition, the reference image information is composed of images captured by street cameras installed on public roads, cameras installed on vehicles that can image public roads, etc. when determining the risk of appearance of suspicious persons on public roads. Images captured by cameras installed inside the station building may also be included.

[0026] In addition, when determining the risk of appearance of suspicious persons inside a public transportation vehicle, the reference image information is composed of images captured by cameras installed inside the vehicle.

[0027] The reference audio information is audio recorded through a microphone or the like at the actual shooting location during the shooting of the reference image information (or a predetermined time before and after the shooting). The analysis of the audio information may be performed by well-known means or may be finely analyzed using an NN. It may also be the result of analyzing the audio information on the frequency axis.

[0028] In the example of FIG. 3, for example, the reference image information is the reference image information P11 to P13 obtained by photographing the exterior of the building structure for each location or for each time series, and the reference audio information is the reference audio information F to the reference audio information I, etc.

[0029] As input data, such reference image information and reference audio information are arranged side by side. What is combined with the reference image information as such input data and the reference audio information is the intermediate node shown in FIG. 3. Each intermediate node is further connected to the output. In this output, the degree of danger of a suspicious person intruding into the building structure as the output solution is displayed as a percentage.

[0030] Each combination (intermediate node) of the reference image information and the reference audio information is correlated with each other through a correlation degree of three or more levels with respect to the degree of danger as this output solution. The reference image information and the reference audio information are arranged on the left side through this correlation degree, and each degree of danger is arranged on the right side through the correlation degree. The correlation degree indicates the degree to which any of the degrees of danger has a high relevance to the reference image information and the reference audio information arranged on the left side. In other words, this correlation degree is an index indicating the degree to which each reference image information and reference audio information is likely to be associated with any degree of danger, and indicates the accuracy in selecting the most probable degree of danger from the reference image information and the reference audio information. In the example of FIG. 3, w13 to w22 are shown as the correlation degrees. These w13 to w22 are shown in 10 levels as shown in Table 1 below. The closer to 10 points, the higher the degree of correlation between each combination as the intermediate node and the degree of danger as the output. Conversely, the closer to 1 point, the lower the degree of correlation between each combination as the intermediate node and the degree of danger as the output.

[0031]

Table 1

[0032] The discrimination device 2 acquires in advance such correlation degrees w13 to w22 shown in FIG. 3 of three or more levels. That is, when actually discriminating the degree of danger, the discrimination device 2 accumulates data on the reference image information, the reference audio information, and the degree of danger in that case, and creates the correlation degree shown in FIG. 3 by analyzing and parsing these.

[0033] For example, assume that the reference image information P11 is an image of a resident of a building entering the building from a door. Also, when the audio at the time of image capture is the reference audio information F, it is determined whether an event occurred in the previous data that caused harm to the resident of the building, whether the degree of risk was such that it would not be strange even if an event did not occur or an event occurred, or whether it was particularly safe, etc. These data can be obtained from the recorded image data captured by the camera in the past for the reference image information, and the reference audio information may be extracted from the audio data recorded at that time. The quantification of the degree of risk may also be obtained by having a plurality of people visually recognize the above reference image information and tabulating a questionnaire survey or the like regarding the degree of risk.

[0034] This analysis and interpretation may also be performed by artificial intelligence. In such a case, for example, when it is the reference image information P11 and the reference audio information F, it is analyzed from past data whether an actual event occurred, or whether the risk was high even though the event did not occur. The higher the degree of association between cases where an event occurred or a high risk leading to an event, the higher the degree of risk of the output, and the higher the degree of association between cases where fewer events occurred and the lower the degree of risk of the output. In the example of the intermediate node 61a linked in the case of the reference image information P11 and the reference audio information H, it is linked to the outputs of a 90% degree of risk and a 30% degree of risk. However, since it is a case with an extremely high degree of risk from previous cases, the degree of association w13 leading to a 90% degree of risk is set to 7 points, and the degree of association w14 leading to a 30% degree of risk is set to 2 points.

[0035] Also, the degree of association shown in this Figure 3 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficient for the output of the nodes of this neural network will correspond to the degree of association described above. Also, not limited to neural networks, it may be composed of any decision-making factor constituting artificial intelligence.

[0036] In the example of the degree of association shown in FIG. 3, node 61b is a node of the combination of the reference audio information F with respect to the reference image information P11, and the degree of association with a risk level of 60% is w15, and the degree of association with a risk level of 0% is w16. Node 61c is a node of the combination of the reference audio information G and further the reference audio information I with respect to the reference image information P12, and the degree of association with a risk level of 30% is w17, and the degree of association with a risk level of 70% is w18.

[0037] Such a degree of association becomes learned data in the context of artificial intelligence. After creating such learned data, when actually determining a new risk level from now on, the risk level will be determined using the above-described learned data. In such a case, image information is newly acquired, and audio information is acquired. The image information corresponds to the reference image information, and the audio information corresponds to the reference audio information.

[0038] The newly acquired image information is obtained by taking a picture with the camera of the information acquisition unit 9 described above. This photographing is the same as the photograph of the exterior of the building structure taken to obtain the above-described reference image information. Also, the photographing conditions are not necessarily required to be the same up to the point where all of the above-described photographing conditions (photographing angle, field angle, resolution) for obtaining the reference image information are the same.

[0039] The acquisition of the audio information is obtained via the information acquisition unit 9 having a recording function.

[0040] Based on the newly acquired image information and audio information in this way, the risk level at the time when the image information was actually acquired is determined. In such a case, the correlation shown in the previously acquired Figure 3 (Table 1) is referred to. For example, when the newly acquired image information is the same as or similar to P12, and the acquired audio information is I, the node 61d is associated through the correlation. This node 61d has "risk level 60%" associated with w19 and "risk level 70%" associated with the correlation w20. In such a case, the "risk level 60%" with the highest correlation is selected as the optimal solution. However, it is not essential to select the one with the highest correlation as the optimal solution. It is also possible to select the "risk level 70%" whose correlation is low but the relevance itself is recognized as the optimal solution. Of course, it is also possible to select an output solution that is not connected by an arrow other than this. As long as it is based on the correlation, it may be selected in any other priority order.

[0041] In this way, the selection of these optimal solutions can be realized through an artificial intelligence that uses the learned model shown in Figure 3 and inputs the input data (image information, audio information) for which the solution is actually desired, and then outputs the output solution (risk level). However, it is not essential for the present invention to utilize artificial intelligence. As long as it uses the correlation of three or more levels between a combination having reference image information and reference audio information and the risk level for the combination, it may be realized in any form.

[0042] Also, examples of the correlations w1 to w12 extending from the input are shown in Table 2 below.

[0043]

Table 2

[0044] The intermediate node 61 may be selected based on the relevance degrees w1 to w12 extending from this input. That is, the higher the relevance degrees w1 to w12, the heavier the weighting in the selection of the intermediate node 61 may be. However, each of the relevance degrees w1 to w12 may be the same value, and the weighting in the selection of the intermediate node 61 may all be the same. In the risk determination system 1, for example, it may embody the form shown in FIG. 4.

[0045] The reference housing structure information is data indicating the structure of the housing, and is, for example, electronic data such as a photograph of the entire housing taken from the air, a photograph of the housing taken from the roadside, electronic data of the completion drawing of the housing, and a structural diagram of the entire housing shown on a map.

[0046] In the example of FIG. 4, for example, the reference image information is assumed to be reference image information P11 to P13 obtained by photographing the exterior of the housing for each location or for each time series, and the reference housing structure information is structures F to I, etc.

[0047] As the input data, such reference image information and reference housing structure information are arranged. What is combined with the reference image information as such input data is the intermediate node shown in FIG. 4. Each intermediate node is further connected to the output. In this output, the risk degree of a suspicious person's intrusion into the housing as the output solution is displayed as a percentage.

[0048] Each combination (intermediate node) of reference image information and reference housing structure information is correlated with each other through a correlation degree of three or more levels with respect to the risk level as this output solution. The reference image information and the reference housing structure information are arranged on the left side through this correlation degree, and each risk level is arranged on the right side through the correlation degree. The correlation degree indicates the degree of high relevance with any of the risk levels for the reference image information and the reference housing structure information arranged on the left side. In other words, this correlation degree is an index indicating the likelihood of each reference image information and reference housing structure information being associated with a certain risk level, and indicates the accuracy in selecting the most probable risk level from the reference image information and the reference housing structure information. In the example of FIG. 4, w13 to w22 are shown as the correlation degrees. These w13 to w22 are shown in 10 levels as shown in Table 1 below. The closer it is to 10 points, the higher the degree of correlation between each combination as an intermediate node and the risk level as the output. Conversely, the closer it is to 1 point, the lower the degree of correlation between each combination as an intermediate node and the risk level as the output.

[0049] The discrimination device 2 acquires in advance such correlation degrees w13 to w22 shown in FIG. 4. That is, the discrimination device 2 accumulates the reference image information, the reference proximity information, and the data on the degree of the risk level in that case when actually performing the discrimination of the risk level, and creates the correlation degree shown in FIG. 4 by analyzing and parsing these.

[0050] For example, assume that the reference image information P11 is an image of a resident of a house entering the house from the door. Also, when the structure F of the house to be discriminated is considered, it is determined whether an incident occurred in the past in which the resident of the house was harmed, or whether, even if no incident occurred, the degree of risk was such that an incident could have occurred without being strange, or whether it was particularly safe, etc. These data can be obtained from the records of past image data captured by cameras for the reference image information, and the reference structure information may be extracted from as-built drawings and map data recorded by house manufacturers or government offices, or from aerial images or image data of the house captured from the road in the past. The quantification of the degree of risk may be obtained by visually inspecting the above reference image information by multiple persons and aggregating questionnaire surveys and the like regarding the degree of risk.

[0051] This analysis and interpretation may be performed by artificial intelligence. In such a case, for example, in the case of the reference image information P11 and the structure F, it is analyzed from past data whether an incident actually occurred, or whether, although no incident occurred, the risk was high. The higher the degree of association leading to a higher output of the degree of risk for cases where an incident occurred or the risk leading to an incident is high, and the higher the degree of association leading to a lower output of the degree of risk for cases where fewer incidents occurred. In the example of the intermediate node 61a linked in the case of the reference image information P11 and the structure H, it is linked to outputs of a 90% degree of risk and a 30% degree of risk. However, since it is a case with an extremely high degree of risk from past examples, the degree of association w13 leading to a 90% degree of risk is set to 7 points, and the degree of association w14 leading to a 30% degree of risk is set to 2 points.

[0052] Also, the degree of association shown in FIG. 4 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficient for the output of the nodes of this neural network will correspond to the degree of association described above. Also, not limited to neural networks, it may be composed of any decision-making factor constituting artificial intelligence.

[0053] In the example of the degree of association shown in FIG. 4, node 61b is a node of the combination of the structure F of the reference housing structure information with respect to the reference image information P11, and the degree of association with a risk level of 60% is w15, and the degree of association with a risk level of 0% is w16. Node 61c is a node of the combination of the structure G (for example, structure information regarding the arrangement of a garden) of the reference housing structure information and further the structure I (for example, structure information regarding the arrangement of a fence) with respect to the reference image information P12, and the degree of association with a risk level of 30% is w17, and the degree of association with a risk level of 70% is w18.

[0054] Such a degree of association becomes the learned data in the context of artificial intelligence. After creating such learned data, when actually determining a new risk level from now on, the risk level will be determined using the above-described learned data. In such a case, image information is newly acquired, and housing structure information is acquired. The image information corresponds to the reference image information, and the housing structure information corresponds to the reference housing structure information.

[0055] The newly acquired image information is obtained by taking an image with the camera of the above-described information acquisition unit 9. This imaging is the same as the image of the exterior of the house taken to obtain the above-described reference image information. Also, the imaging conditions are not necessarily required to be exactly the same up to the point where all of the above-described imaging conditions (imaging angle, angle of view, resolution) for obtaining the reference image information are the same.

[0056] The acquisition of the housing structure information accesses a database in which the housing structure information of the house for which the risk level is to be determined is recorded. For example, when the housing structure information is associated through an address and recorded in the database, the housing structure information linked to that address may be read out by inputting the address.

[0057] Based on the newly acquired image information and the approaching information of the train in this way, the risk level at the time when the image information was actually acquired is determined. In such a case, the correlation shown in Fig. 4 (Table 1) acquired in advance is referred to. For example, when the newly acquired image information is the same as or similar to P12, and the acquired housing structure information is Structure I, Node 61d is associated through the correlation. This Node 61d has "Risk level 60%" associated with w19 and "Risk level 70%" associated with the correlation w20. In such a case, the "Risk level 60%" with the highest correlation is selected as the optimal solution. However, it is not essential to select the one with the highest correlation as the optimal solution. Instead, the "Risk level 70%" with a lower correlation but with its relevance itself recognized may be selected as the optimal solution. Of course, it is also possible to select an output solution not connected by an arrow other than this. As long as it is based on the correlation, it may be selected in any other priority order.

[0058] In this way, the selection of these optimal solutions can be realized through an artificial intelligence that uses the learned model shown in Fig. 4 and inputs the input data (image information, housing structure information) for which the solution is actually desired, and then outputs the output solution (risk level). However, it is not essential for the present invention to utilize artificial intelligence. As long as it uses the correlation of three or more levels between the reference image information, the combination of the reference housing structure information, and the risk level for the combination, it may be realized in any form.

[0059] When determining the risk level of the appearance of suspicious persons on public roads, instead of using the reference housing structure information, the relevance may be formed between the reference map structure information and the reference image information regarding the map and road structure on the photographed public road. The reference map structure information referred to here is information regarding a map that at least includes the range where the reference image information is actually photographed. Since areas that become blind spots for crimes where there are alleys when viewed on a map, or areas such as busy streets where crimes are likely to occur, the aim is to improve the discrimination accuracy of the risk level by including maps containing such information. The information regarding the road structure is composed of, for example, overpasses or pedestrian bridges, or in the station building, various structures such as stairs, escalators, moving sidewalks, branches, intersections, etc. It may be materialized as map information within the station building by combining the information regarding this structure with map information. As a result, it becomes possible to obtain this as reference map structure information including the positional relationship of stores within the station building.

[0060] In such a case, in addition to the image information, map structure information regarding a map and the road structure that at least includes the range where the image information is photographed is acquired. The details of the map structure information are the same as those of the reference map structure information. Then, a solution search is performed based on the reference map structure information corresponding to the map structure information.

[0061] Figure 5 shows an example in which a relevance of three or more levels is set between the above-described combination having the reference image information and the reference location information, and the risk level for the combination.

[0062] As input data, such reference image information and reference location information are arranged. What is obtained by combining the reference location information with the reference image information as such input data is the intermediate node shown in Figure 5.

[0063] The reference location information includes all information for identifying the location of the building structure. In addition to address information and map information, the reference location information also includes images of the surrounding environment including the building structure. Also included in this reference location information are the conditions of the buildings on the adjacent land of the building structure and the uses to which the adjacent land of the building structure is put, such as shops, buildings, restaurants, vacant lots, parking lots, schools, etc. That is, if the adjacent land is a vacant lot, it is also considered that it is easy to break into the building structure from that vacant lot, so this also becomes a factor in controlling the risk level. When determining the risk level of the appearance of a suspicious person on a public road, it is composed of information for identifying the location on that public road, and also includes information about the conditions of the buildings on the adjacent land of the public road and the uses to which the adjacent land of the building structure is put, such as shops, buildings, restaurants, vacant lots, parking lots, schools, etc.

[0064] The discrimination device 2 acquires in advance such relevance levels w13 to w22 of three or more levels shown in FIG. 5. That is, when actually discriminating the risk level, the discrimination device 2 accumulates reference image information, reference location information, and data on the degree of the risk level in that case, and creates the relevance levels shown in FIG. 5 by analyzing and analyzing these.

[0065] In the example of the relevance level shown in FIG. 5, the node 61b is a node of a combination in which the reference location information is location J with respect to the reference image information P11, and the relevance level of a risk level of 60% is w15 and the relevance level of a risk level of 0% is w16.

[0066] Similarly, when such a relevance level is set, new image information is acquired and location information is acquired. The image information corresponds to the reference image information, and the location information corresponds to the reference location information.

[0067] To obtain location information, access the database in which the voice information of the building structure for which the risk level is to be determined is recorded. For example, when the location information is associated through an address and recorded in the database, by inputting the address, the location information linked to that address (such as the adjacent lot being a restaurant and relatively bright, or the adjacent lot being a parking lot and easy for suspicious persons to enter) may be read out. The same applies to the risk level search of a public road. By inputting each address to which the public road is connected, the location information linked to that address (such as the adjacent lot being a restaurant and relatively bright, or the adjacent lot being a parking lot and easy for suspicious persons to enter) may be read out.

[0068] In determining the risk level, refer to the correlation shown in FIG. 5 obtained in advance. For example, when the acquired image is the same as or similar to the reference image information P12 and the location information is location M, the combination is associated with node 61c. This node 61c is associated with a risk level of 30% with a correlation of w17 and a risk level of 70% with a correlation of w18. As a result of such correlations, based on w17 and w18, the risk level at the time when the new image information and proximity information are actually acquired is determined.

[0069] FIG. 6 shows an example in which a correlation of three or more levels is set between the above-described combination of reference image information and reference home pattern information and the risk level for the combination.

[0070] As input data, such reference image information and reference home pattern information are arranged. What is obtained by combining the reference location information with the reference image information as such input data is the intermediate node shown in FIG. 6.

[0071] The reference home pattern information includes all information for identifying the home patterns of the residents living in the building structure. As the reference home pattern information, for example, based on the electricity consumption in the building structure and the images of the residents' coming and going taken by the cameras installed at the entrance and the door, the time periods when the residents are at home and the time periods when they are out are patterned. During Monday to Sunday, for each time period of each day of the week, the time periods when the residents are at home and the time periods when they are out are patterned based on the above-mentioned electricity consumption and images. At this time, the above-mentioned electricity consumption and images may be pre-trained by machine learning so as to discriminate the patterns.

[0072] The discrimination device 2 pre-acquires the correlation degrees w13 to w22 of three or more levels shown in FIG. 6 as described above. That is, the discrimination device 2 accumulates the reference image information, the reference home pattern, and the data on the degree of risk in that case when actually discriminating the degree of risk, and creates the correlation degrees shown in FIG. 6 by analyzing and parsing these.

[0073] In the example of the correlation degree shown in FIG. 6, the node 61b is a node of a combination in which the reference location information is location J with respect to the reference image information P11, and the correlation degree of a risk degree of 60% is w15 and the correlation degree of a risk degree of 0% is w16.

[0074] Similarly, when such a correlation degree is set, new image information is acquired and home pattern information is acquired. The image information corresponds to the reference image information, and the home pattern information corresponds to the reference home pattern information.

[0075] To acquire the home pattern information, access is made to the database in which the home pattern information of the building structure for which the risk degree is to be discriminated is recorded. For example, when the home pattern information is recorded in the database in association with the address, the home pattern information linked to the address (from Monday to Friday, going out from 9:00 to 21:00 and being at home otherwise, on Saturdays and Sundays, going out from 10:00 to 15:00 and being at home otherwise, etc.) may be read by inputting the address.

[0076] In determining the risk level, refer to the relevance shown in FIG. 6 obtained in advance. For example, when the acquired image is the same as or similar to the reference image information P12 and the home pattern information is the home pattern R, the combination is associated with the node 61c. This node 61c is associated with a risk level of 30% with a relevance w17 and a risk level of 70% with a relevance w18. Based on the results of such relevance, w17 and w18, the risk level at the time when the new image information and proximity information are actually acquired is determined.

[0077] FIG. 7 shows an example in which a relevance of three or more levels is set between the above-described reference image information, a combination having reference time zone information indicating the time zone at the time of shooting the reference image information, and the risk level for the combination.

[0078] As input data, such reference image information and reference time zone information are arranged. What is obtained by combining the reference time zone information with the reference image information as such input data is the intermediate node shown in FIG. 7.

[0079] The reference time zone information means the time zone at the time of shooting the reference image information, and may have not only the time of a single point but also a time width such as 10:10 to 10:20.

[0080] The discrimination device 2 acquires in advance the relevance of three or more levels w13 to w22 shown in FIG. 7. That is, when actually discriminating the risk level, the discrimination device 2 accumulates data on the reference image information, the reference home pattern, and the risk level in that case, and creates the relevance shown in FIG. 7 by analyzing and analyzing these.

[0081] In the example of the degree of association shown in FIG. 7, node 61b is a node with a combination of reference time zone information being time zone S for the reference image information P11, with the degree of association of 60% risk being w15 and the degree of association of 0% risk being w16. That is, there are time zones with few suspicious intrusions by criminals and time zones with many. In particular, if there are few suspicious intrusions during the day and many at night, the risk for each time zone S to V, ··· is analyzed in advance in relation to the reference image information, and is stored in association with the degree of association as the intermediate node 61.

[0082] Similarly, when such a degree of association is set, new image information is acquired and the in-home time zone information is acquired. The image information corresponds to the reference image information, and the time zone information corresponds to the reference in-home time zone information.

[0083] The acquisition of the time zone information extracts the time zone at the time of shooting the image information in the building structure for which the risk is to be determined. Then, the extracted time zone information is compared with the reference time zone information for discrimination. In such a case, access is made to the database in which the reference time zone information is recorded. For example, when the time zone information is from 9:20 to 9:30 (time zone V), the risk is obtained via the same time zone V (9:20 to 9:30) of the reference time zone information.

[0084] In determining the risk, the degree of association shown in FIG. 7 acquired in advance is referred to. For example, when the acquired image is the same as or similar to the reference image information P12 and the time zone information is time zone V, the combination is associated with node 61c, and this node 61c is associated with a 30% risk with the degree of association w17 and a 70% risk with the degree of association w18. As a result of such a degree of association, based on w17 and w18, the risk at the time when the new image information and the proximity information are actually acquired is determined.

[0085] For the risk discrimination of the appearance of suspicious persons on public roads and the risk discrimination of the appearance of suspicious persons inside public transportation vehicles, the reference time zone information can also be learned in the same way, and it becomes possible to perform solution search via the newly acquired time zone information.

[0086] Figure 8 shows an example in which a three - level or higher correlation is set between a combination having the above - mentioned reference image information, reference audio information, and further reference person information, and the risk level for the combination.

[0087] The reference person information is information obtained by extracting the person depicted in the reference image information. The person information includes facial features (such as the outline of the face, the shape of the eyes and nose, hairstyle, presence or absence of glasses, wrinkles, scars, etc.), and the age and gender estimated from the facial features. As for the method of extracting facial features, a neural network may be used. In such a case, well - known methods may be utilized, such as: (1) extracting edges from the image data of the face part; (2) extracting the skin - colored area from the image data of the face part; (3) extracting the area corresponding to the skin - colored area extracted in (2) from the edge image generated in (1); (4) calculating the cumulative histograms in the vertical and horizontal directions from the edge image corresponding to the skin - colored area; (5) using the number of pixels in the cumulative histogram where the luminance is equal to or higher than a predetermined threshold as the input data of the NN (neural network). Also, (6) the age and gender of the visitor may be estimated by the NN based on the input data and the teacher data.

[0088] In such a case, as shown in Figure 8, the set of combinations having the reference image information, reference audio information, and reference person information will be represented as the intermediate nodes 61a to 61e in the same manner as described above.

[0089] For example, in FIG. 8, node 61c is associated with reference image information P12 with an association degree of w3, reference audio information "Structure G" with an association degree of w7, and "Person β (for example, if the person information is the wife living next door who often visits) as reference person information with an association degree of w11. Similarly, node 61e is associated with reference image information P13 with an association degree of w5, reference audio information G with an association degree of w8, and "Person α (for example, a 40-year-old man who has never been here before) as reference person information with an association degree of w10. Even if it is not registered in the reference person information, the fact that it is not registered may be treated as one piece of reference person information.

[0090] Similarly, when such association degrees are set, based on the newly acquired image information, audio information, and person information, the risk level at the time when the new image information is actually acquired is determined. The person information here is obtained by performing image analysis on the newly acquired image information by the same method as extracting the reference person information from the newly acquired image information, and extracting the person information.

[0091] When determining this risk level, refer to the association degrees shown in FIG. 8 acquired in advance. For example, when the acquired image is the same as or similar to the reference image information P12, and the reference audio information is G, and the extracted person information is Person β, this combination is associated with node 61c. This node 61c is associated with a risk level of 30% with an association degree of w17 and a risk level of 70% with an association degree of w18. As a result of such association degrees, based on w17 and w18, the risk level at the time when the new image information and the proximity information are actually acquired is determined.

[0092] If the case where it is not registered in the reference person information is assumed to be Person β, and the extracted person information is a person not registered in the reference person information, the solution search will be performed assuming that Person β has been input.

[0093] In addition, for a combination having reference image information and reference location information, a combination having reference image information and reference home pattern information, and a combination having reference image information and reference time information, the relevance may be defined by further including a combination having this reference person information. Alternatively, the relevance may be defined by a combination having reference image information and reference person information.

[0094] FIG. 9 shows an example in which a combination having the above-described reference image information, in addition to reference audio information, and further having reference security service information, and a relevance of three or more levels with respect to the combination are set.

[0095] The reference security service information relates to whether or not the building structure has contracted for the security service of a security company, or the contract situation regarding the specific contract content. This contract is, for example, that when a door is broken or a window glass is broken, a security guard comes to patrol the building structure from the security company.

[0096] In such a case, as shown in FIG. 9, the relevance is such that a set of combinations having reference image information, reference audio information, and reference security service information is expressed as intermediate nodes 61a to 61e in the same manner as described above.

[0097] For example, in FIG. 9, node 61c is associated with reference image information P12 with a relevance of w3, reference audio information G with a relevance of w7, and "not contracted" as reference security service information with a relevance of w11.

[0098] Similarly, when such a relevance is set, based on the newly acquired image information, audio information, and security service information, the risk level at the time when the new image information is actually acquired is determined. The security service information here relates to whether or not the building structure to be discriminated has actually contracted for the security service of a security company, or the contract situation regarding the specific contract content.

[0099] In determining this risk level, reference is made to the degree of correlation shown in FIG. 9 obtained in advance. For example, when the acquired image is identical or similar to the reference image information P12 and the reference audio information G, and the extracted security service information is "not contracted", the combination is associated with node 61c. This node 61c is associated with a risk level of 30% with a degree of correlation w17 and a risk level of 70% with a degree of correlation w18. Based on the results of such degrees of correlation, w17 and w18, the risk level at the time when the new image information and proximity information are actually acquired is determined.

[0100] FIG. 10 shows an example in which a three-level or higher degree of correlation is set between the above-described reference image information, a combination having reference weather information indicating the weather at the time of shooting the reference image information, and the risk level for the combination.

[0101] The reference weather information indicates information such as the weather (sunny, cloudy, rainy), disasters (typhoon, heavy rain, etc.), temperature, and humidity at the time of shooting. On rainy days, etc., the number of people going out is small, noise is difficult to detect, and there is a possibility that intruders can easily enter. Therefore, this is also added as an explanatory variable.

[0102] As input data, such reference image information and reference weather information are arranged. The intermediate node shown in FIG. 10 is a combination of the reference image information as such input data and the reference time zone information.

[0103] The discrimination device 2 acquires in advance a three-level or higher degree of correlation w13 to w22 shown in FIG. 10. That is, the discrimination device 2 accumulates data on the reference image information, the reference weather information, and the risk level in that case when actually discriminating the risk level, and creates the degree of correlation shown in FIG. 10 by analyzing and analyzing these.

[0104] In the example of the degree of association shown in FIG. 10, node 61b is a node of the combination of reference image information P11 and reference weather information S. The degree of association with a risk level of 60% is w15, and the degree of association with a risk level of 0% is w16. That is, there are weather conditions with few suspicious intrusions by criminals and those with many. In particular, on sunny days, there are few suspicious intrusions, and in the case of a lot of rain, for each reference weather information S~V,··, the risk level is analyzed in advance in the relationship with the reference image information, and it is associated and stored in the degree of association as the intermediate node 61.

[0105] Similarly, when such a degree of association is set, new image information is acquired, and weather information is acquired. The image information corresponds to the reference image information, and the weather information corresponds to the reference weather information.

[0106] For the acquisition of weather information, the weather at that time may be imported from the data of the Meteorological Agency, or the weather grasped by oneself may be input. Then, the extracted weather information is compared with the reference weather information for discrimination. In such a case, access is made to the database in which the reference weather information is recorded. For example, when the weather information is heavy rain, the risk level is obtained through the weather information (heavy rain) of the same reference time zone information.

[0107] When obtaining the risk level, refer to the degree of association shown in FIG. 10 acquired in advance. For example, when the acquired image is the same as or similar to the reference image information P12 and the weather information is V, the combination is associated with node 61c. This node 61c is associated with a risk level of 30% with a degree of association of w17 and a risk level of 70% with a degree of association of w18. As a result of such a degree of association, based on w17 and w18, the risk level at the time when the new image information and weather information are actually acquired is obtained.

[0108] Figure 11 shows an example of using reference traffic volume information as an alternative to the above-described reference weather information. The reference traffic volume information is information regarding the traffic volume of vehicles or pedestrians on the roads around a building structure. This traffic volume refers to the number of vehicles and pedestrians passing per unit time. Since it has already been reported that the likelihood of a suspicious person entering a building structure facing a road with a high traffic volume tends to be low, the traffic volume is included as an explanatory variable to determine the degree of risk. The same applies to the risk search for public roads. Since it has been reported that the likelihood of a suspicious person appearing varies according to the traffic volume of pedestrians and vehicles on the public road, the traffic volume is included as an explanatory variable to determine the degree of risk.

[0109] For example, assume that the reference traffic volume information S has a traffic volume of 100 people per unit time, and the reference traffic volume information T has a traffic volume of 1 person per unit time. In this case, when the degree of risk is low, such as in the case of the reference traffic volume information S, a process of reducing the weighting of the degree of risk is performed. In other words, a process of reducing the degree of risk itself is preset to be performed. On the other hand, when the degree of risk is high, such as in the case of the reference traffic volume information T, a process of increasing the weighting of the degree of risk is performed. In other words, a process of increasing the degree of risk itself is preset to be performed.

[0110] As input data, such reference image information and reference traffic volume information are arranged side by side. The combination of the reference traffic volume information with the reference image information as such input data is the intermediate node shown in Figure 11.

[0111] The discrimination device 2 acquires in advance such correlation degrees w13 to w22 at three or more levels shown in Figure 11. That is, the discrimination device 2 accumulates data on the reference image information, the reference traffic volume information, and the degree of risk in that case when actually discriminating the degree of risk, and creates the correlation degrees shown in Figure 11 by analyzing and analyzing these.

[0112] In the example of the degree of association shown in FIG. 11, node 61b is a node of the combination of reference traffic volume information S with respect to reference image information P11, and the degree of association with a risk level of 60% is w15, and the degree of association with a risk level of 0% is w16. That is, there are roads with a large traffic volume and roads with a small traffic volume. In particular, for a building structure facing a road with a large traffic volume, the intrusion of suspicious persons is small. Conversely, for a road with a small traffic volume, if there are many intruders, for each reference traffic volume information S to V, ···, the risk level is analyzed in advance in the relationship with the reference image information, and it is associated and stored in the degree of association as the intermediate node 61.

[0113] Similarly, even when such a degree of association is set, new image information is acquired and traffic volume information is acquired. The image information corresponds to the reference image information, and the traffic volume information corresponds to the reference traffic volume information.

[0114] Note that the above-mentioned traffic volume information and reference traffic volume information may directly use the data of traffic volume surveys conducted by municipalities, countries, or other institutions, or may be determined based on the images of buildings on the road taken per unit time. In such a case, the inspector may count the number of vehicles and pedestrians reflected in the image one by one, or use well-known deep learning techniques to extract and identify the vehicles and pedestrians, and count the number of the identified vehicles and pedestrians per unit time.

[0115] Then, the extracted traffic volume information is compared with the reference traffic volume information for discrimination. In such a case, access is made to the database in which the reference traffic volume information is recorded. For example, when the traffic volume information is the same as or similar to the reference traffic volume information, the risk level is obtained through the reference traffic volume information.

[0116] In determining the risk level, refer to the correlation shown in Fig. 11 obtained in advance. For example, when the acquired image is the same as or similar to the reference image information P12 and the traffic volume information is V, the combination is associated with node 61c. This node 61c is associated with a risk level of 30% with a correlation degree of w17 and a risk level of 70% with a correlation degree of w18. Based on the results of such correlation degrees, w17 and w18, the risk level at the time when the new image information and weather information are actually acquired is determined.

[0117] When performing risk search inside a public transportation vehicle, as an alternative to the reference traffic volume information, the solution search may be performed from the correlation between the risk level for a combination having reference passenger number information indicating the number of passengers inside the vehicle and the reference image information. Since the incidence of theft and sexual crimes changes as the number of passengers inside the vehicle increases, this is introduced as an explanatory variable. Similarly in such a case, the passenger number information regarding the number of passengers inside the vehicle to be newly discriminated is acquired. The method for acquiring the reference passenger number information and the passenger number information may be to image the inside of the vehicle with a camera and estimate the number of people from it, or to perform it via a sensor or the like, or to actually count by hand.

[0118] When newly acquiring the passenger number information, search for the risk level through the corresponding reference passenger number information.

[0119] FIG. 12 shows an example of using reference criminal history information as an alternative to the above-described reference weather information. The reference criminal history information is information regarding the past criminal history in the area of the building structure. The criminal history includes all crimes such as murder, housebreaking, burglary, abduction, robbery, sex crimes, etc. This criminal history may be quantified by the number of criminal cases per unit period, or may be weighted further according to the weight of the crime with respect to the numerical value. Adjustment is made such that the higher the criminal history in an area, the higher the degree of danger. The same applies to the search for the degree of danger of a public road, and the past criminal history in the area where the public road is laid may be included as an explanatory variable. The same applies to the search for the degree of danger inside a public transportation vehicle, and the past criminal history in the area where the public transportation vehicle is laid may be included as an explanatory variable.

[0120] The criminal history of each region may be obtained by using information publicly available on the Internet, or may be extracted and input from articles reported in TV, newspapers, etc.

[0121] As input data, such reference image information and reference criminal history information are arranged side by side. What is combined with the reference image information as such input data and the reference criminal history information is the intermediate node shown in FIG. 12.

[0122] The discrimination device 2 acquires in advance the relevance degrees w13 to w22 at three or more levels shown in FIG. 12. That is, the discrimination device 2 accumulates data on the reference image information, the reference criminal history information, and the degree of danger in that case when actually discriminating the degree of danger, and creates the relevance degree shown in FIG. 12 by analyzing and analyzing these.

[0123] In the example of the degree of association shown in FIG. 12, node 61b is a node of the combination of reference image information P11 and reference criminal history information S. The degree of association with a risk level of 60% is w15, and the degree of association with a risk level of 0% is w16. That is, there are areas with a large number of criminal histories and areas with a small number of criminal histories. In particular, for buildings facing areas with a large number of criminal histories, the intrusion of suspicious persons is small. Conversely, for roads with a small number of criminal histories, if there are many intruders, for each reference criminal history information S~V,··, the risk level is analyzed in advance in the relationship with the reference image information, and is associated and stored in the degree of association as the intermediate node 61.

[0124] Similarly, when such a degree of association is set, new image information is acquired, and criminal history information is acquired. The image information corresponds to the reference image information, and the criminal history information corresponds to the reference criminal history information.

[0125] Then, the extracted criminal history information is compared with the reference criminal history information for discrimination. In such a case, access is made to the database in which the reference criminal history information is recorded. For example, when the criminal history information is the same as or similar to the reference criminal history information, the risk level is obtained through the reference criminal history information.

[0126] In obtaining the risk level, the degree of association shown in FIG. 12 obtained in advance is referred to. For example, when the acquired image is the same as or similar to the reference image information P12 and the criminal history information is V, the combination is associated with node 61c. This node 61c is associated with a risk level of 30% with an association degree of w17 and a risk level of 70% with an association degree of w18. As a result of such a degree of association, based on w17 and w18, the risk level at the time when the new image information and criminal history information are actually acquired is obtained.

[0127] In addition, for a combination having reference image information and reference location information, a combination having reference image information and reference home pattern information, and a combination having reference image information and reference time information, the relevance may be defined by further including a combination having this reference security service information. Alternatively, the relevance may be defined by a combination having reference image information and reference security service information.

[0128] In the example of FIG. 13, an example of using a relevance of three or more levels between reference image information and a risk level is shown.

[0129] It is assumed that a relevance in which reference image information and a risk level are linked to each other is formed. In the example of FIG. 13, assume that the input data is, for example, reference image information P11 to P13. Such reference image information as input data is linked to the output. In this output, assume that it is a risk level as an output solution.

[0130] The reference image information is mutually related to the risk level as this output solution through a relevance of three or more levels. The reference image information is arranged on the left side through this relevance, and each risk level is arranged on the right side through the relevance.

[0131] The discrimination device 2 acquires in advance such relevances w13 to w19 of three or more levels shown in FIG. 13. That is, when actually discriminating a search solution, the discrimination device 2 accumulates past data on what risk level it was when acquiring reference image information captured in the past, and analyzes and interprets these to create the relevances shown in FIG. 13.

[0132] For example, assume that the reference image information is P11. Assume that the risk level for such P11 was mostly 90%. By collecting and analyzing such data sets, the relevance between the reference image information P11 and the risk level becomes stronger.

[0133] This analysis may be performed by artificial intelligence. Further, image discrimination may be performed using deep learning. Also, the degree of association shown in FIG. 13 may be composed of nodes of a neural network in artificial intelligence. That is, the weighting coefficient of this neural network node with respect to the output will correspond to the degree of association described above. Also, it may be composed of any decision-making factor constituting artificial intelligence, not limited to neural networks.

[0134] In addition, in the process of constructing a learned model based on such a degree of association, reference housing structure information including housing structure information regarding the building structure is also acquired. This reference housing structure information is not included in the degree of association described above.

[0135] Such a degree of association becomes learned data in the context of artificial intelligence. After creating such learned data, a solution will be searched for. In such a case, the captured image information is acquired, and the housing structure information is also acquired in the same manner.

[0136] First, based on the newly acquired image information, the risk level is searched for. In such a case, the degree of association shown in FIG. 13 acquired in advance is utilized. For example, when the newly acquired image information is the same as or similar to the reference image information P12, a risk level of 30% is associated with the degree of association w15 and a risk level of 60% is associated with the degree of association w16 through the degree of association. In such a case, the highest risk level of 30% in terms of the degree of association is selected as the optimal solution. However, it is not essential to select the one with the highest degree of association as the optimal solution. It is also possible to select the degree of association of 60% whose relevance is recognized although the degree of association is low as the solution. Also, of course, it is possible to select an output solution that is not connected by an arrow other than this. As long as it is based on the degree of association, it may be selected in any other priority order. Also, the output solution to be selected is not limited to one, and two or more may be selected. In such a case, two or more may be selected in order from the highest degree of association, but it is not limited to this, and it may be based on any other priority order of the degree of association.

[0137] The risk level obtained through the degree of association may be further corrected based on the housing structure information or the weighting may be changed.

[0138] For example, assume that the structure F of the reference housing structure information is a single-family house with an open exterior structure and no fence built around it, and the structure G of the reference housing structure information is a structure with a 2m-high fence surrounding it. At this time, in the case of a structure with a low risk level like the structure G of the reference housing structure information, a process of reducing the weighting of the risk level is performed, in other words, a process of reducing the risk level itself is preset. On the contrary, in the case of a structure with a high risk level like the structure F of the reference housing structure information, a process of increasing the weighting of the risk level is performed, in other words, a process of increasing the risk level itself is preset.

[0139] After such settings with the reference housing structure information, when the actually acquired housing structure information is the same as or similar to the structure G of the reference housing structure information, a process of reducing the weighting of the risk level is performed, in other words, a process of reducing the risk level itself is performed. On the contrary, when the actually acquired housing structure information is the same as or similar to the structure F of the reference housing structure information, a process of increasing the weighting of the risk level is performed, in other words, a process of increasing the risk level itself is performed.

[0140] In such a case, as a result of acquiring the housing structure information, when the actually acquired housing structure information is the same as or similar to the structure G and the safety is high, for example, as a result of performing the process of reducing the weighting of the risk level, the judgment itself using the above-mentioned degree of association may not be performed, and it may be judged as a risk level of 0%. When discriminating the risk level of the appearance of a suspicious person on a public road, as an alternative to the reference housing structure information, reference map structure information regarding the map or road structure on the photographed public road is used, and map structure information regarding the map or road structure that at least includes the range where the image information is photographed is acquired. The details of the map structure information are the same as those of the reference map structure information. Then, a solution search is performed based on the reference map structure information corresponding to the map structure information.

[0141] In the present invention, the solution search is not limited to the case where the above-described housing structure information is included. It is not essential to obtain the risk level only based on the degree of association and then correct it or change the weighting based on the housing structure information.

[0142] That is, obtain in advance the degree of association of three or more levels with the risk level for the reference image information. When newly determining the risk level, obtain image information by photographing the target, refer to the previously obtained degree of association, and based on the newly obtained image information, determine the risk level of a suspicious person's intrusion or the risk level of the appearance of a suspicious person in a vehicle or on a public road.

[0143] The same applies when using reference location information as an alternative to the reference housing structure information.

[0144] Further, FIG. 14 shows an example of using reference location information as an alternative to the reference housing structure information. In such a case, the same processing operations will be executed. Depending on the location of the building structure according to the reference location information, the weighting of the risk level will be changed according to whether the safety is high or not.

[0145] Of course, for the case shown in FIG. 13, reference occupancy pattern information, reference time zone information, reference security service information, and reference weather information may be applied as alternatives to the reference housing structure information.

[0146] Further, FIG. 15 shows an example of using reference traffic volume information as an alternative to the reference housing structure information. The reference traffic volume information is information regarding the traffic volume of vehicles or pedestrians on the roads around the building structure. This traffic volume is the number of vehicles or pedestrians passing per unit time. Since it has already been reported that the possibility of a suspicious person's intrusion tends to be lower for a building structure facing a road with a higher traffic volume, this is included as an explanatory variable to determine the risk level.

[0147] For example, assume that the reference traffic volume information F has a traffic volume of 100 people per unit time, and the reference traffic volume information G has a traffic volume of 1 person per unit time. At this time, when the risk level is low like the reference traffic volume information F, a process of reducing the weighting of the risk level is performed, in other words, a process of reducing the risk level itself is preset. On the contrary, when the risk level is high like the reference traffic volume information G, a process of increasing the weighting of the risk level is performed, in other words, a process of increasing the risk level itself is preset.

[0148] After the setting with the reference traffic volume information as described above, when the actually acquired traffic volume information is the same as or similar to the reference traffic volume information G, a process of increasing the weighting of the risk level is performed, in other words, a process of increasing the risk level itself is performed. On the contrary, when the actually acquired traffic volume information is the same as or similar to the reference traffic volume information F, a process of reducing the weighting of the risk level is performed, in other words, a process of reducing the risk level itself is performed.

[0149] In such a case, as a result of acquiring the traffic volume information, when the safety is high, for example, when the actually acquired traffic volume information is the same as or similar to the reference traffic volume information F, as a result of performing the process of reducing the weighting of the risk level, the determination itself using the above-described correlation degree may not be performed, and it may be determined as a risk level of 0%.

[0150] Note that the above-described traffic volume information and reference traffic volume information may directly use the data of the traffic volume survey conducted by municipalities, countries, or other organizations, or may be determined based on the images of the roads of the building structures captured per unit time. In such a case, the inspector may count the number of vehicles and pedestrians shown in the image one by one, or may use well-known deep learning technology to extract and identify the vehicles and pedestrians, and count the number of the identified vehicles and pedestrians per unit time.

[0151] Figure 16 shows an example of using reference criminal history information as an alternative to reference housing structure information. The reference criminal history information is information on the past criminal history in the area of the building structure. The criminal history includes all crimes such as murder, housebreaking, burglary, abduction, robbery, sex crimes, etc. This criminal history may be quantified by the number of criminal cases per unit period, or may be weighted further according to the weight of the crime for the numerical value. Adjustments are made such that the higher the criminal history of the area, the higher the risk level.

[0152] The criminal history of each area may be obtained by using information publicly available on the Internet, or may be extracted and input from articles reported in TV, newspapers, etc.

[0153] For example, assume that the reference criminal history information F has only 1 burglary in the past 10 years in that area. In this case, when the risk level is low like the reference criminal history information F, a process of reducing the weight of the risk level is performed, in other words, a process of reducing the risk level itself is preset. On the contrary, when the risk level is high, such as when the reference criminal history information G has 10 burglaries in the past 10 years, a process of increasing the weight of the risk level is performed, in other words, a process of increasing the risk level itself is preset.

[0154] After the setting with the reference criminal history information as described above, when the actually obtained criminal history information is the same as or similar to the reference criminal history information G, a process of increasing the weight of the risk level is performed, in other words, a process of increasing the risk level itself is performed. On the contrary, when the actually obtained criminal history information is the same as or similar to the reference criminal history information F, a process of reducing the weight of the risk level is performed, in other words, a process of reducing the risk level itself is performed.

[0155] In such a case, if the obtained criminal history information is the same as or similar to the reference criminal history information F as a result of obtaining the criminal history information, and the safety is high, for example, as a result of performing a process of reducing the weight of the risk level, the determination itself using the above-described relevance may not be performed, and it may be determined as a risk level of 0%.

[0156] Further, as an alternative to the reference residential structure information shown in FIG. 13, reference time zone information may be used. Then, a solution search is performed in the same manner as described above based on the reference time zone information corresponding to the newly obtained time zone information. Further, when searching for the risk level of the appearance of a suspicious person in a public transportation vehicle, as an alternative to the reference residential structure information shown in FIG. 13, reference passenger number information may be used. Then, a solution search is performed in the same manner as described above based on the reference passenger number information corresponding to the newly obtained passenger number information.

[0157] Further, as shown in FIGS. 13 to 16, the relevance is not limited to the case where it is formed in the relationship between the reference image information and the risk level. As shown in FIG. 17, a combination having the reference image information and the reference audio information recorded at the time of shooting the reference image information, and a relevance of three or more levels with respect to the risk level of the combination may be obtained in advance. For these, the reference information linked to any of the reference audio information, the reference location information, the reference at-home pattern information, the reference time zone information, the reference security service information, the reference weather information, the reference traffic volume information, and the reference criminal history information may be referred to, and the risk level of the intrusion of a suspicious person may be determined.

[0158] In the example of FIG. 18, the same processing operation is executed when forming the relevance in the relationship between the action information obtained by classifying a series of movements and poses of the person reflected in the reference image information and the risk level. That is, a relevance of three or more levels of the risk level for each of the above-described combinations including the reference action information obtained by extracting the action of the person by analyzing the reference image information is obtained in advance. Next, when newly determining the risk level, obtained by photographingFurther obtain action information by analyzing the image information to extract the actions of a person. Determine the degree of risk through the reference action information corresponding to the obtained action information.

[0159] The reference action information and the action information may utilize so-called pose estimation techniques. In pose estimation techniques, deep learning may be used. For example, feature points of the nose, eyes, ears, neck, shoulders, elbows, wrists, waist, knees, and ankles are detected by deep learning. Then, pose estimation is performed based on the dynamic changes of these feature points. As free software for pose estimation, for example, OpenPose (registered trademark) developed by Carnegie Mellon University may be utilized.

[0160] The correlation obtained by substituting the reference image information and the image information with such reference action information and action information is not limited to the reference correlation shown in FIG. 18, and is also applicable when constructing combinations with other reference housing structure information, reference location information, reference at-home pattern information, reference time zone information, reference security service information, reference weather information, reference traffic volume information, and reference criminal history information. Also, solution search may be performed only based on the correlation obtained by substituting the reference action information and the action information. The present invention also includes all embodiments embodied by replacing the reference image information and the image information with such reference action information and action information.

[0161] As an alternative to the reference image information, as shown in FIG. 19, reference sensing information obtained by sensing an intruder from the outside of a building structure with a sensor may be utilized. This reference sensing information includes infrared sensors, laser sensors capable of detecting intruders into the site of a building structure, passive sensors that detect the difference between the body temperature of a person and the surrounding temperature to detect an intrusion, and all information obtained by magnet switches, glass break sensors, etc. that detect the opening and closing, breakage, and vibration of the windows of a building structure.

[0162] Obtain in advance a correlation degree of three or more levels of risk for each of the above-described combinations including such reference sensing information. Next, when newly determining the risk degree, obtain sensing information by sensing an intruder from outside the building structure with a sensor. Determine the risk degree through the reference sensing information corresponding to such sensing information.

[0163] The correlation degree obtained by replacing the reference image information and the image information with such reference sensing information and sensing information is not limited to the reference correlation degree shown in FIG. 19, but other combinations of reference housing structure information, reference location information, reference at-home pattern information, reference time zone information, reference security service information, reference weather information, reference traffic volume information, and reference criminal history information are also applicable.

[0164] Note that in the present invention, as shown in FIG. 20, reference image information may be input as input data, a risk degree may be output as output data, at least one or more hidden layers may be provided between the input node and the output node, and machine learning may be performed. The above-described correlation degree is set in either one or both of the input node or the hidden layer node, which serves as the weighting of each node, and the output is selected based on this. Then, when this correlation degree exceeds a certain threshold value, the output may be selected.

[0165] According to the present invention having the above-described configuration, it is possible to easily determine the risk degree of an intruder in a building structure with little effort without requiring any particular skill. Also, according to the present invention, it is possible to determine this risk degree with higher accuracy than when a human does it. Furthermore, by configuring the above-described correlation degree with artificial intelligence (such as a neural network), it is possible to further improve the discrimination accuracy by learning this.

[0166] Further, according to the present invention, it is characterized in that an optimal physical property and generation mechanism are searched through a relevance set at three or more levels. The relevance can be described by numerical values from, for example, 0 to 100% in addition to the five levels described above, but is not limited thereto, and may be composed of any number of levels as long as it can be described by numerical values of three or more levels.

[0167] By searching for the most probable risk level based on the relevance represented by such numerical values of three or more levels, it is also possible to search and display in descending order of the relevance in a situation where multiple candidates with a high possibility of increasing the risk are considered. If it is possible to display to the user in descending order of the relevance in this way, it is also possible to preferentially display a more probable risk level, and it is possible to prompt attention due to an increase in the risk level.

[0168] When this risk level is high, it is possible to avoid danger by contacting a security service company or the police, or immediately prompting the occupants of the building structure to take notice. Particularly when the risk level is higher, it is possible to protect safety by arousing the attention of the residents through voice or the like, or generating a voice outside to discourage the intrusion intention of suspicious persons. Further, according to the present invention, since the detection accuracy of the risk level is high, it is only necessary to call the police only in truly necessary cases without calling the police unnecessarily.

[0169] In addition, according to the present invention, it is possible to make a judgment without overlooking a discrimination result with an extremely low output such as a relevance of 1%. It is possible to alert the user that even a discrimination result with an extremely low relevance is connected as a slight sign and may be useful as the discrimination result once in dozens or hundreds of times.

[0170] Further, the present invention makes a discrimination based on the relevance of a combination of two or more types of information, such as reference information U and reference information V, as shown in FIG. 21. This reference information U and V are each reference information including the above-described reference image information.

[0171] At this time, as shown in FIG. 21, the output obtained for the reference information U may be directly used as input data and associated with the output via the intermediate node 61 of the combination with the reference information V. For example, for the reference information U (reference image information), after obtaining the output solution as shown in FIG. 3, this may be directly used as input, and the degree of association with other reference information V may be utilized to search for the output.

[0172] Furthermore, according to the present invention, by performing the search based on such a degree of association of three or more levels, there is an advantage that the search policy can be determined by the way of setting the threshold value. If the threshold value is lowered, even if the degree of association described above is 1%, it can be picked up without omission, but on the other hand, the possibility of preferably detecting a more appropriate discrimination result is low, and there may be a case where a lot of noise is picked up. On the other hand, if the threshold value is raised, there is a high possibility of detecting the optimum risk level with a high probability, but on the other hand, although the degree of association is usually low and is passed through, there may be a case where a suitable solution that appears once in dozens or hundreds of times is overlooked. Which one to prioritize can be determined based on the ideas of the user side and the system side, but it is possible to increase the degree of freedom in selecting such a point of emphasis.

[0173] Furthermore, in the present invention, the above-described degree of association may be updated. This update may, for example, reflect information provided via a public communication network including the Internet. Also, based on camera images captured in a building structure, acquired reference image information, reference location information, reference home pattern information, reference time information, reference voice information, reference security service information, etc., when new findings are discovered about the relationship between the input parameters and the output solution (risk level), the degree of association is increased or decreased according to the findings.

[0174] That is, this update corresponds to learning in the sense of artificial intelligence. Since new data is acquired and reflected in the learned data, it can be said to be a learning behavior.

[0175] Also, in the process of initially creating the learned model and the above-described update, not only supervised learning but also unsupervised learning, deep learning, reinforcement learning, etc. may be used. In the case of unsupervised learning, instead of loading and training a dataset of input data and output data, information corresponding to the input data (reference image information, reference location information, reference home pattern information, reference time information, reference voice information, reference security service information, etc.) may be loaded and trained, and the degree of association related to the output data may be self-formed therefrom.

[0176] In addition to the case based on information obtainable from the public communication network, the update of this degree of association may be manually or automatically updated by the system side or the user side based on research data, papers, conference presentations by experts, and the content of newspaper articles, books, etc. Artificial intelligence may be utilized in these update processes.

[0177] Also, in the process of initially creating the learned model and the above-described update, not only supervised learning but also unsupervised learning, deep learning, reinforcement learning, etc. may be used. In the case of unsupervised learning, instead of loading and training a dataset of input data and output data, information corresponding to the input data may be loaded and trained, and the degree of association related to the output data may be self-formed therefrom.

[0178] Also, in the risk determination system 1 to which the present invention is applied, when the risk level is high, in addition to notifying the residents of the building structure, the management company, and further the security company, it is of course possible to generate an alarm or generate a voice such as "Please do not enter without permission".

[0179] In the above-described embodiment, the case of detecting the risk level of an intruder has been described as an example, but it is not limited thereto. The present invention can similarly detect the risk level of intrusion by animals other than humans, that is, beasts.

[0180] The term "wild animal" as used herein includes all animals that pose a threat to humans, such as bears, brown bears, wild boars, wild boars, raccoon dogs, stray dogs, cats, pigeons, etc. In addition, animals are not limited to mammals, but also include birds such as crows.

[0181] Similarly, when detecting the risk level of such wild animal intrusion, a correlation between the reference information and the risk level is formed in advance. At the actual intrusion detection, various information such as image information is obtained by a camera in the same way, and the risk level is determined with reference to the correlation.

[0182] When referring to each of the above-mentioned reference information, as an alternative to the reference criminal history information, a correlation is formed between the reference appearance history information indicating the history of wild animal appearances in each region previously and the risk level. At the actual discrimination, the appearance history information indicating the history of wild animal appearances in that region is obtained, and a solution search is performed with reference to the above-mentioned correlation based on the reference appearance history information corresponding to this appearance history information.

[0183] In this detection of wild animal intrusion degree, it can be similarly applied not only to within a building structure but also to cultivated lands such as paddy fields and fields.

[0184] In addition, in the risk level discrimination system 1 to which the present invention is applied, it is possible to discriminate a building structure, a public road, a public transportation facility, or the area with a high risk level. For a building structure, a public road, a public transportation facility, or the area where the risk level exceeds a certain threshold value, for example, an arrangement plan may be proposed such as displaying that more sensors should be arranged.

[0185] Hereinafter, embodiments applicable when discriminating the risk level of the appearance of a suspicious person on a public road and when discriminating the risk level of the appearance of a suspicious person inside a public transportation vehicle will be further described. FIG. 22 shows an example of using reference environmental information as an alternative to the above-described reference weather information. The reference environmental information is a parameter necessary for determining the quality of the actual public road, inside the vehicle, or the environment of the station, such as odor or the amount of garbage accumulation. The odor may be measured through an odor sensor, and the amount of garbage accumulation may be analyzed by image analysis using a camera or by sensors installed on the road, inside the vehicle, or in the trash can.

[0186] As input data, such reference image information and reference environmental information are arranged side by side. What is obtained by combining the reference environmental information with the reference image information as such input data is the intermediate node shown in FIG. 22.

[0187] The discrimination device 2 acquires in advance the relevance degrees w13 to w22 at three or more levels shown in FIG. 22. That is, the discrimination device 2 accumulates data on the reference image information, the reference environmental information, and the degree of risk in that case when actually discriminating the degree of risk, and creates the relevance degrees shown in FIG. 22 by analyzing and parsing these.

[0188] In the example of the relevance degree shown in FIG. 22, the node 61b is a node of a combination of the reference environmental information S with respect to the reference image information P11, and the relevance degree of a risk degree of 60% is w15, and the relevance degree of a risk degree of 0% is w16. That is, there are areas with many and few criminal histories. In particular, for each reference environmental information S to V,... in the case where the building structures facing areas with many criminal histories have few suspicious person intrusions, and conversely, roads with few criminal histories have many intruders, the risk degree is analyzed in advance in the relationship with the reference image information, and is associated and stored in the relevance degree as the intermediate node 61.

[0189] Similarly, when such a relevance degree is set, new image information is acquired and environmental information is acquired. The image information corresponds to the reference image information, and the environmental information corresponds to the reference environmental information.

[0190] Then, the extracted environmental information is compared with the reference environmental information for discrimination. In such a case, access is made to the database in which the reference environmental information is recorded. For example, when the environmental information is the same as or similar to the reference environmental information, the risk level is determined through the reference environmental information.

[0191] In determining the risk level, the relevance shown in FIG. 22, which has been acquired in advance, is referred to. For example, when the acquired image is the same as or similar to the reference image information P12 and the environmental information is V, the combination is associated with the node 61c. This node 61c is associated with a risk level of 30% with a relevance of w17 and a risk level of 70% with a relevance of w18. As a result of such relevance, based on w17 and w18, the risk level at the time when the new image information and environmental information are actually acquired is determined.

[0192] Also, needless to say, instead of discriminating a suspicious person on a public road, the risk level of an animal may be discriminated.

Explanation of Signs

[0193] 1 Risk Level Discrimination System 2 Discrimination Device 21 Internal Bus 23 Display Unit 24 Control Unit 25 Operation Unit 26 Communication Unit 27 Estimation Unit 28 Storage Unit 61 Node

Claims

1. In a risk determination program for determining the risk of the appearance of a suspicious person inside a vehicle of a public transportation agency, an information acquisition step of acquiring, when determining the risk, action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle, and voice information recorded at the time of the shooting; referring to a three-level or higher correlation between reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and the risk, and based on the reference action information corresponding to the action information obtained through the above information acquisition step, giving priority to those with a higher correlation, and determining the risk of the appearance of a suspicious person inside the vehicle based on the acquired voice information, and causing a computer to execute a determination step; A risk determination program characterized by the above.

2. In a risk determination program for determining the risk of the appearance of a suspicious person inside a vehicle of a public transportation agency, an information acquisition step of acquiring, when determining the risk, person information obtained by extracting a person by analyzing an image newly taken inside the vehicle, and action information obtained by extracting the actions of the person; referring to a three-level or higher correlation between reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and the risk, and based on the reference action information corresponding to the action information obtained through the above information acquisition step, giving priority to those with a higher correlation, and determining the risk of the appearance of a suspicious person inside the vehicle based on the acquired person information, and causing a computer to execute a determination step; A risk determination program characterized by the above.

3. In a risk determination program for determining the risk of the appearance of a suspicious person inside a vehicle of a public transportation agency, an information acquisition step of acquiring, when determining the risk, action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle, and time zone information regarding the time zone at the time of the shooting; referring to a three-level or higher correlation between reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and the risk, and based on the reference action information corresponding to the action information obtained through the above information acquisition step, giving priority to those with a higher correlation, and determining the risk of the appearance of a suspicious person inside the vehicle based on the acquired time zone information, and causing a computer to execute a determination step; A risk determination program characterized by the above.

4. In a risk determination program for determining the risk of the appearance of a suspicious person inside a public transportation vehicle, an information acquisition step of acquiring, when determining the risk, action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle, and sensing information obtained by sensing the person with a sensor; referring to a three-level or higher correlation between reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and the risk, prioritizing the higher correlation based on the reference action information corresponding to the action information obtained through the information acquisition step, and causing a computer to execute a determination step of determining the risk of the appearance of a suspicious person inside the vehicle based on the acquired sensing information; A risk determination program characterized by the above.

5. In the information acquisition step, acquiring the sensing information including information on the body temperature of the person The risk determination program according to claim 4, characterized by the above.

6. In a risk determination program for determining the risk of the appearance of a suspicious person inside a public transportation vehicle, an information acquisition step of acquiring, when determining the risk, action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle, and passenger number information indicating the number of passengers inside the vehicle; referring to a three-level or higher correlation between reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and the risk, prioritizing the higher correlation based on the reference action information corresponding to the action information obtained through the information acquisition step, and causing a computer to execute a determination step of determining the risk of the appearance of a suspicious person inside the vehicle based on the acquired passenger number information; A risk determination program characterized by the above.

7. In a risk determination program for determining the risk of the appearance of a suspicious person inside a public transportation vehicle, an information acquisition step of acquiring, when determining the risk, action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle, and criminal history information indicating the past criminal history in the route or area of the public transportation vehicle; Causing a computer to execute a discrimination step of referring to a three-stage or higher correlation degree between reference behavior information obtained by extracting a person's behavior by analyzing an image taken inside a vehicle and a risk level, and preferentially selecting a higher correlation degree, and discriminating the risk level of the appearance of a suspicious person inside the vehicle based on the obtained criminal history information, based on the reference behavior information corresponding to the behavior information obtained via the information acquisition step A risk level discrimination program characterized by the above.

8. In a risk level discrimination program for discriminating the risk level of the appearance of a suspicious person inside a public transportation vehicle, an information acquisition step of acquiring behavior information obtained by extracting a person's behavior by newly analyzing an image taken inside the vehicle and voice information recorded at the time of shooting when discriminating the risk level, and causing a computer to execute a discrimination step of referring to a three-stage or higher correlation degree between a combination having reference behavior information obtained by extracting a person's behavior by analyzing an image taken inside the vehicle and reference voice information recorded at the time of shooting and a risk level, and discriminating the risk level of the appearance of a suspicious person inside the vehicle based on the reference behavior information corresponding to the behavior information obtained via the information acquisition step and the reference voice information corresponding to the obtained voice information A risk level discrimination program characterized by the above.

9. In a risk level discrimination program for discriminating the risk level of the appearance of a suspicious person inside a public transportation vehicle, an information acquisition step of acquiring behavior information obtained by extracting a person's behavior by newly analyzing an image taken inside the vehicle and time zone information regarding the time zone at the time of shooting when discriminating the risk level, and causing a computer to execute a discrimination step of referring to a three-stage or higher correlation degree between a combination having reference behavior information obtained by extracting a person's behavior by analyzing an image taken inside the vehicle and reference time zone information regarding the time zone at the time of shooting and a risk level, and discriminating the risk level of the appearance of a suspicious person inside the vehicle based on the reference behavior information corresponding to the behavior information obtained via the information acquisition step and the reference time zone information corresponding to the obtained time zone information A risk level discrimination program characterized by the above.

10. In a risk level discrimination program for discriminating the risk level of the appearance of a suspicious person inside a public transportation vehicle, When determining the degree of danger, an information acquisition step of acquiring action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle and sensing information obtained by sensing the person with a sensor, referring to a three-level or higher correlation with the degree of danger for a combination having reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and reference sensing information obtained by sensing the person with a sensor, and based on the reference action information corresponding to the action information obtained through the above information acquisition step and the reference sensing information corresponding to the sensing information, causing a computer to execute a determination step of determining the degree of danger of the appearance of a suspicious person inside the vehicle A danger determination program characterized by the above.

11. In the above information acquisition step, sensing information including information on the body temperature of the person is acquired, In the above determination step, referring to a three-level or higher correlation with the degree of danger for a combination having the above reference sensing information including information on the body temperature of the person The danger determination program according to claim 10, characterized by the above.

12. In a danger determination program for determining the degree of danger of the appearance of a suspicious person inside a vehicle of a public transportation agency, When determining the degree of danger, an information acquisition step of acquiring action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle and passenger number information indicating the number of passengers inside the vehicle, referring to a three-level or higher correlation between a combination having reference action information obtained by extracting the actions of a person by analyzing an image taken inside the vehicle and reference passenger number information indicating the number of passengers inside the vehicle and the degree of danger, and based on the reference action information corresponding to the action information obtained through the above information acquisition step and the reference passenger number information corresponding to the acquired passenger number information, causing a computer to execute a determination step of determining the degree of danger of the appearance of a suspicious person inside the vehicle A danger determination program characterized by the above.

13. In a danger determination program for determining the degree of danger of the appearance of a suspicious person inside a vehicle of a public transportation agency, When determining the degree of danger, an information acquisition step of acquiring action information obtained by extracting the actions of a person by analyzing an image newly taken inside the vehicle and crime history information indicating the past crime history in the route or area of the public transportation agency, Causing a computer to execute a determination step of determining the risk of the appearance of a suspicious person in the vehicle based on reference behavior information obtained by analyzing an image taken inside the vehicle to extract the behavior of a person and reference criminal history information indicating the past criminal history in the route or area of the public transportation, with reference to a three-level or higher correlation degree between the combination having the above information and the risk. A risk determination program characterized by the above.

14. The above correlation degree is composed of nodes of a neural network in artificial intelligence. The risk determination program according to any one of claims 1 to 13, characterized by the above.

Images