Information processing method

Abstract

To provide an information processing method that notifies an elementary school or staff members of the elementary school of the fact that an elementary school student has deviated from a commute route.SOLUTION: An information processing method is executed in an information processing system 1 including a vehicle 10 which includes an imaging unit 11, a position information acquisition unit 12, and a communication unit 32, and a server 20. The information processing method includes: allowing the communication unit 32 of the vehicle 10 to transmit notification information which includes an ambient image of an elementary school student, and position information on the vehicle 10 when the ambient image is produced by the imaging unit 11, to the server 20; and allowing the server 20 to notify an information terminal 60 of notification information which includes an ambient image of an elementary school student existing away from an elementary school commute route and position information on the vehicle 10 when the ambient image is produced by the imaging unit 11, out of the notification information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing method. [Background technology]

[0002] Conventionally, there is known a device that sends an alert email when the location of a mobile terminal carried by an elementary school student who commutes to school via a pre-registered school route is away from the school route (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-066122 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the method disclosed in Patent Document 1, if each elementary school student does not carry a mobile terminal, it is not possible to determine whether the elementary school student has deviated from the school route. [Means for solving the problem]

[0005] An information processing method for solving the above problem is an information processing method executed in an information processing system including a vehicle having an imaging unit that generates a surrounding image by imaging the surrounding situation, a location information acquisition unit that acquires its own location information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, and configured so that the surrounding image and the location information are transmitted from the communication unit of the vehicle to the server device via a communication network, the information processing method including: the communication unit of the vehicle transmitting to the server device notification information including the surrounding image in which an elementary school student is shown and the location information of the vehicle when the surrounding image was generated by the imaging unit; and the server device notifying at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of the notification information including the surrounding image in which an elementary school student who is away from the elementary school's commuting route is shown, and the location information of the vehicle when the surrounding image was generated by the imaging unit. Effect of the Invention

[0006] According to the above configuration, even if the elementary school student does not have an information terminal, it is possible to notify the elementary school or elementary school staff that the elementary school student has left the school route. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram used to explain an information processing system. [Diagram 2] FIG. 2 is a diagram used to explain the server device and the control device. [Diagram 3] FIG. 3 is a flowchart illustrating an example of a process executed in the control device. [Figure 4] FIG. 4 is a flowchart illustrating an example of a process executed in the server device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] <Embodiment> Hereinafter, one embodiment of the information processing method will be described with reference to FIGS. <Overall configuration of information processing system 1> 1, an information processing system 1 that realizes the information processing method includes, for example, one or more vehicles 10 and one or more server devices 20. The vehicle 10 includes, for example, an imaging unit 11, a position information acquisition unit 12, a sound acquisition unit 13, and a control device 30. The control device 30 and the server device 20 are connected to each other via a communication network 40 so as to be able to transmit and receive information. The communication network 40 includes, for example, the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0009] The imaging unit 11 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). One example of the imaging unit 11 is a digital camera constituting a drive recorder. The imaging unit 11 captures images of the surroundings of the vehicle 10. Depending on the installation position of the imaging unit 11 on the vehicle 10, the imaging unit 11 can capture images of the front, rear, and sides of the vehicle 10. The imaging unit 11 generates a surrounding image by capturing images of the surroundings of the vehicle 10. When the vehicle 10 travels along a school route for an elementary school, an elementary school student 50 may appear in the surrounding image generated by the imaging unit 11. The school route is a route that the elementary school student 50 takes when commuting to school.

[0010] The position information acquisition unit 12 is, for example, a Global Navigation Satellite System (GNSS) receiver. The position information acquisition unit 12 identifies the position of the vehicle 10 based on a signal received from a GNSS satellite, and acquires position information indicating the position of the vehicle 10.

[0011] The sound acquisition unit 13 is, for example, an acoustic device that converts an input sound into an electrical signal and outputs the electrical signal, and is configured with a single microphone. The sound acquisition unit 13 is attached to an arbitrary location of the vehicle 10. The sound acquisition unit 13 acquires sounds around the vehicle 10 and generates sound information.

[0012] When the elementary school student 50 is captured in the peripheral image generated by the imaging unit 11 and the situation of the elementary school student 50 satisfies various conditions, the control device 30 transmits information about the elementary school student 50 to the server device 20. Details of the various conditions and the information transmitted by the control device 30 to the server device 20 will be described later.

[0013] The server device 20 outputs the information received from the control device 30 to, for example, an information terminal 60 provided in the elementary school and an information terminal 60 owned by the staff of the elementary school. In this way, the staff of the elementary school grasps information about the elementary school student 50. The information terminal 60 includes, for example, a smartphone, a tablet terminal, and a personal computer.

[0014] <Configuration of the control device 30> As shown in FIG. 2, the control device 30 includes, for example, a processing circuit 31, a communication unit 32, and a storage unit 33. The processing circuit 31 is configured with, for example, a hardware processor such as a CPU (Central Processing Unit). The hardware processor executes a program (software), which allows the processing circuit 31 to execute various processes. The program may be stored in the storage unit 33 in advance. The storage unit 33 is realized by, for example, a non-transitory storage medium such as a HDD (Hard Disk Drive) or a flash memory. The communication unit 32 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the server device 20 and other devices based on the control of the processing circuit 31.

[0015] The storage unit 33 stores, for example, a trained model 331, commuting route information 332, risk area information 333, and reference sound information 334. The trained model 331 is a model that performs machine learning to determine whether or not an elementary school student 50 appears in a peripheral image generated by the imaging unit 11. An example of the trained model 331 is a neural network. When a peripheral image is input to the trained model 331 as an input variable, the trained model 331 outputs an output variable indicating the probability that the elementary school student 50 appears in the peripheral image. For example, the trained model 331 outputs a larger value as the output variable as the probability that the elementary school student 50 appears in the peripheral image increases. In this case, when the output variable is equal to or greater than a threshold value, the processing circuit 31 of the control device 30 determines that the elementary school student 50 appears in the peripheral image. On the other hand, when the output variable is less than the threshold value, the processing circuit 31 determines that the elementary school student 50 does not appear in the peripheral image.

[0016] Here, when the elementary school student 50 goes to school, the elementary school student 50 carries a backpack or wears a hat or uniform designated by the elementary school. The trained model 331 is previously subjected to machine learning that can determine that the elementary school student 50 is in a surrounding image when a child who satisfies at least one of the following two conditions (A1) and (A2) is in the surrounding image.

[0017] (A1) Carrying a school bag. (A2) Students must wear clothing designated by the elementary school. The school route information 332 is information in which an elementary school is associated with location information indicating a school route set within the school district of the elementary school.

[0018] The risk area information 333 is information in which an elementary school is associated with location information indicating a risk area within the school district of the elementary school. A risk area is an area where accidents are likely to occur. Examples of risk areas include areas near flood control facilities such as rivers and reservoirs, roads with heavy traffic, and facilities where large vehicles frequently enter and exit.

[0019] The reference sound information 334 is information that indicates the sound of a general security buzzer. The processing circuit 31 determines whether or not the elementary school student 50 is located away from the school commuting route based on the surrounding image generated by the imaging unit 11, the position information acquired by the position information acquisition unit 12, and the information stored in the storage unit 33. Specifically, the processing circuit 31 determines whether or not the elementary school student 50 is included in the surrounding image based on the output variables of the trained model 331 when the surrounding image generated by the imaging unit 11 is input to the trained model 331. When the processing circuit 31 determines that the elementary school student 50 is included in the surrounding image, it acquires the position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11.

[0020] Next, the processing circuit 31 reads out from the storage unit 33 the commuting route information 332 of an elementary school having a school district including the current position of the vehicle 10. Then, based on the position information acquired from the position information acquisition unit 12 and the commuting route information 332 read out from the storage unit 33, the processing circuit 31 determines whether or not the position indicated by the position information is included in the commuting route of the elementary school. If the processing circuit 31 determines that the position indicated by the position information is not included in the commuting route of the elementary school, it determines that the elementary school student 50 is located away from the commuting route of the elementary school. On the other hand, if the processing circuit 31 determines that the position indicated by the position information is included in the commuting route of the elementary school, it determines that the elementary school student 50 is on the commuting route of the elementary school.

[0021] When the processing circuit 31 determines that the elementary school student 50 is away from the commuting route of the elementary school, it transmits notification information to the server device 20 of the elementary school via the communication network 40. The notification information transmitted to the server device 20 includes, for example, a surrounding image of the elementary school student 50 away from the commuting route, and location information acquired by the location information acquisition unit 12 when the surrounding image was generated by the imaging unit 11. At this time, the processing circuit 31 identifies an elementary school having a school district including the location indicated by the location information based on the location information included in the notification information transmitted to the server device 20. Then, the processing circuit 31 transmits notification information to the server device 20 of the identified elementary school.

[0022] In addition, even if the processing circuit 31 determines that there are no elementary school students away from the elementary school commuting route, if the elementary school student 50 is captured in the surrounding image generated by the imaging unit 11 during a time period other than the time of going to and from the elementary school, the processing circuit 31 transmits notification information to the server device 20. If the processing circuit 31 determines that the elementary school student 50 is captured in the surrounding image, it determines whether the time when the surrounding image was generated by the imaging unit 11 is during a time period other than the time of going to and from the elementary school. The time period of going to and from the elementary school includes the time period of going to the elementary school and the time period of going home from the elementary school. An example of the time period of going to school is between 7:00 and 9:00 in the morning. An example of the time period of going home from the elementary school is between 2:00 and 5:00 in the evening.

[0023] Here, during times other than when the elementary school student 50 is going to and from school, it is rare for the elementary school student 50 to go out with his / her backpack on. Also, during times other than when the elementary school student is going to and from school, it is rare for the elementary school student 50 to go out wearing clothes designated by the elementary school. Therefore, when it is determined that the elementary school student 50 is captured in the surrounding image captured by the imaging unit 11 during times other than when the elementary school student is going to and from school, it is preferable to notify the elementary school or the staff of the elementary school. Therefore, when the elementary school student 50 is captured in the surrounding image generated by the imaging unit 11 during times other than when the elementary school student is going to and from school, the processing circuit 31 transmits notification information to the server device 20 of the elementary school via the communication network 40.

[0024] Furthermore, when the processing circuit 31 determines that the elementary school student 50 is in the peripheral image, and when the elementary school student 50 is in a dangerous area even if the elementary school student 50 is not far from the school route, the processing circuit 31 transmits notification information to the server device 20. The processing circuit 31 determines whether the elementary school student 50 is in a dangerous area designated by the elementary school based on the peripheral image generated by the imaging unit 11 and the position information acquired by the position information acquisition unit 12. When the processing circuit 31 determines that the elementary school student 50 is in the peripheral image, it acquires the position information acquired by the position information acquisition unit 12 when the peripheral image was generated by the imaging unit 11.

[0025] Next, the processing circuit 31 reads out from the storage unit 33 risk area information 333 designated by an elementary school having a school district including the current location of the vehicle 10. Then, the processing circuit 31 determines whether or not the location indicated by the location information acquired by the location information acquisition unit 12 is a risk area based on the risk area information 333 read out from the storage unit 33. When the processing circuit 31 determines that the location indicated by the location information is a risk area, it determines that the elementary school student 50 is in the risk area.

[0026] When the processing circuit 31 determines that the elementary school student 50 is in a dangerous area, it transmits notification information to the server device 20 of the elementary school via the communication network 40. The notification information includes, for example, a surrounding image when the imaging unit 11 captures the elementary school student 50 in the dangerous area, and position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11.

[0027] The processing circuit 31 also determines whether or not the security buzzer carried by the elementary school student 50 is ringing based on the sound information generated by the sound acquisition unit 13. If the processing circuit 31 determines that the security buzzer is ringing, it transmits warning notification information to the server device 20. The processing circuit 31 determines whether or not the sound of the security buzzer carried by the elementary school student 50 has been acquired based on the sound information generated by the sound acquisition unit 13. If the reference sound information 334 and the sound information generated by the sound acquisition unit 13 match, the processing circuit 31 determines that the sound of the security buzzer carried by the elementary school student 50 has been acquired. For example, the processing circuit 31 may determine that the reference sound information 334 and the sound information generated by the sound acquisition unit 13 match when any one of the following conditions (B1) and (B2) is satisfied.

[0028] (B1) The frequency components of the sound in the reference sound information 334 match the frequency components of the sound in the sound information. (B2) The sound emission pattern of the reference sound information 334 matches the sound emission pattern of the sound information.

[0029] When the processing circuit 31 determines that it has acquired the sound of a security buzzer carried by the elementary school student 50, it transmits warning notification information to the server device 20 of the elementary school via the communication network 40. The warning notification information includes, for example, a surrounding image generated by the imaging unit 11 and position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11. The surrounding image included in the warning notification information is a surrounding image generated by the imaging unit 11 at the time when the sound of the security buzzer was acquired, and is a surrounding image in which the elementary school student 50 carrying the security buzzer is shown.

[0030] <Configuration of the server device 20> As shown in FIG. 2, the server device 20 includes, for example, a processing circuit 21, a communication unit 22, and a storage unit 23. The processing circuit 21 is configured with, for example, a hardware processor such as a CPU. The hardware processor executes a program (software), which allows the processing circuit 21 to execute various processes. The program may be stored in the storage unit 23 in advance. The storage unit 23 is realized by, for example, a non-transitory storage medium such as an HDD or a flash memory. The communication unit 22 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the control device 30 and other devices based on the control of the processing circuit 21.

[0031] Moreover, one or more information terminals 60 are connected to the server device 20. The communication unit 22 communicates with the information terminal 60 in a wired or wireless manner. At this time, the server device 20 and the information terminal 60 may communicate with each other via the above-mentioned communication network 40, or may communicate using another network. In the case of wireless communication, the communication mode between the communication unit 22 and the information terminal 60 may be short-distance wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). In addition, the communication mode between the communication unit 22 and the information terminal 60 may be communication via a network such as the Internet, a WAN, a LAN, a provider terminal, a wireless communication network, a wireless base station, or a dedicated line. The information terminal 60 is realized, for example, by a smartphone, a tablet, or the like.

[0032] When the processing circuit 21 receives notification information or caution notification information from the control device 30, it notifies the information terminal 60 of the received information. Specifically, the processing circuit 21 notifies the information terminal 60 that the notification information or caution notification information has been uploaded to the server device 20. When a staff member who knows that a notification from the server device 20 has arrived at the information terminal 60 operates the operation unit of the information terminal 60, the notification information or caution notification information is downloaded from the server device 20 to the information terminal 60. Then, in the information terminal 60, the downloaded information is displayed on the display unit of the information terminal 60.

[0033] <Series of processes executed in the control device 30> A series of processes executed in the control device 30 will be described with reference to FIG. 3. The processes shown in FIG. 3 are repeatedly executed at predetermined time intervals. First, the processing circuit 31 acquires a surrounding image generated by the imaging unit 11 (step S100). Next, the processing circuit 31 acquires position information from the position information acquisition unit 12 (step S102). The position information acquired in step S102 is the position information of the vehicle 10 when the surrounding image acquired in step S100 was generated by the imaging unit 11. Next, the processing circuit 31 acquires sound information from the sound acquisition unit 13 (step S104).

[0034] Next, the processing circuit 31 judges whether the sound acquisition unit 13 has acquired the sound of the security buzzer carried by the elementary school student 50 based on the sound information acquired from the sound acquisition unit 13 (step S106). When the reference sound information indicating the reference sound of the security buzzer matches the sound information generated by the sound acquisition unit 13, the processing circuit 31 judges that the sound of the security buzzer has been acquired. When the processing circuit 31 judges that the sound of the security buzzer has been acquired (step S106; YES), it judges whether the elementary school student 50 is included in the peripheral image acquired in step S100 (step S108). The processing circuit 31 judges whether the elementary school student 50 is included in the peripheral image based on the output variable of the trained model 331 when the peripheral image acquired in step S100 is input to the trained model 331. When the processing circuit 31 judges that the elementary school student 50 is not included in the peripheral image (step S108; NO), it ends the series of processes.

[0035] When the processing circuit 31 determines that an elementary school student 50 is included in the peripheral image (step S108; YES), it causes the communication unit 22 to transmit warning notification information to the server device 20 of the elementary school (step S110). The warning notification information transmitted to the server device 20 in step S110 includes the peripheral image acquired in step S100 during the current execution of the series of processes shown in Fig. 3, and the position information acquired in step S102 during the current execution of the series of processes. Thereafter, the processing circuit 31 ends the series of processes.

[0036] When the processing circuit 31 determines that the sound of the security buzzer carried by the elementary school student 50 has not been acquired (step S106; NO), the processing circuit 31 shifts the processing to step S112. As in step S108, the processing circuit 31 determines whether the elementary school student 50 is captured in the peripheral image acquired in step S100 (step S112). When the processing circuit 31 determines that the elementary school student 50 is not captured in the peripheral image (step S112; NO), there is no information to notify the server device 20, and therefore the series of processing is terminated.

[0037] When the processing circuit 31 determines that the elementary school student 50 is included in the peripheral image (step S112; YES), it determines whether or not the time when the peripheral image was generated by the imaging unit 11 was a time period other than the time when the student was going to or coming from school (step S114). When the processing circuit 31 determines that the time when the peripheral image was generated by the imaging unit 11 was a time period other than the time when the student was going to or coming from school (step S114; YES), it causes the communication unit 22 to transmit notification information to the server device 20 of the elementary school (step S116). The notification information transmitted to the server device 20 in step S116 includes the peripheral image acquired in step S100 during the current execution of the series of processes shown in FIG. 3 and the position information acquired in step S102 during the current execution of the series of processes. After that, the processing circuit 31 ends the series of processes.

[0038] When the processing circuit 31 determines that the time generated by the imaging unit 11 is the time period during which the elementary school student 50 is going to and from school (step S114; NO), the processing circuit 31 determines whether or not the elementary school student 50 is away from the school route (step S118). The processing circuit 31 determines whether or not the elementary school student 50 is away from the school route based on the position information acquired in step S102 and the elementary school commuting route information 332. When the processing circuit 31 determines that the elementary school student 50 is away from the school route (step S118; YES), the processing circuit 31 advances the processing to step S116.

[0039] When the processing circuit 31 determines that the elementary school student 50 is on the commuting route (step S118; NO), it determines whether or not the elementary school student 50 is in a dangerous area (step S120). The processing circuit 31 determines whether or not the elementary school student 50 is in a dangerous area based on the position information acquired in step S102 and the dangerous area information 333. When the processing circuit 31 determines that the elementary school student 50 is in a dangerous area (step S120; YES), it advances the processing to step S116. When the processing circuit 31 determines that the elementary school student 50 is in a place that is not a dangerous area (step S120; NO), it ends the series of processing.

[0040] In other words, even if it is determined that an elementary school student 50 is included in the surrounding image acquired in step S100, if any of the following three conditions (C1), (C2), and (C3) are satisfied, the processing circuit 31 terminates the series of processes without sending notification information to the server device 20.

[0041] (C1) The time when the surrounding image was generated is during school commute hours. (C2) There are 50 elementary school students on the route to school. (C3) There are 50 elementary school students on the school route that are not in dangerous areas.

[0042] <Series of processes executed in the server device 20> A series of processes executed in the server device 20 will be described with reference to FIG. 4. The processes shown in FIG. 4 are repeatedly executed at predetermined time intervals. The processing circuit 21 judges whether or not notification information or attention notification information has been received from the control device 30 (step S200). If the processing circuit 21 has not received either notification information or attention notification information from the control device 30 (step S200; NO), the processing circuit 21 ends the series of processes. If the processing circuit 21 has received notification information or attention notification information from the control device 30 (step S200; YES), the processing circuit 21 notifies the information terminal 60 of the received information (step S202). Then, the processing circuit 21 ends the series of processes.

[0043] <Actions and Effects of the Embodiments> According to the above embodiment, the following actions and effects can be obtained. (1) When the vehicle 10 is traveling, the imaging unit 11 generates a surrounding image by capturing an image of the surroundings of the vehicle 10. At this time, the position information acquisition unit 12 acquires position information of the vehicle 10 at the time when the imaging unit 11 captured the surrounding image.

[0044] The processing circuit 31 of the control device 30 judges whether or not the elementary school student 50 is captured in the peripheral image generated by the imaging unit 11. The processing circuit 31 transmits notification information to the server device 20 on the condition that the elementary school student 50 captured in the peripheral image is located away from the school route.

[0045] When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60. That is, the server device 20 transmits to the information terminal 60 the notification information including a surrounding image showing the elementary school student 50 who is away from the school route and the position information when the surrounding image was generated by the imaging unit 11.

[0046] As a result, according to the information processing method, even if the elementary school student 50 does not have an information terminal, it is possible to notify the elementary school or elementary school staff that the elementary school student 50 has left the school route. (2) If the elementary school student 50 is in a dangerous area within the school district of the elementary school, there is a possibility that the elementary school student 50 will be involved in an accident. Therefore, in the information processing method, when the imaging unit 11 captures an image of the elementary school student 50 in a dangerous area while the vehicle 10 is traveling, notification information including a surrounding image in which the elementary school student 50 is captured is transmitted from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60. In other words, the server device 20 transmits to the information terminal 60 notification information including a surrounding image when the imaging unit 11 captures the elementary school student in the dangerous area.

[0047] As a result, according to the information processing method, it is possible to make the staff of the elementary school aware that the elementary school student 50 is in a dangerous area before the elementary school student 50 actually gets involved in an accident. (3) When an elementary school student 50 operates his / her own security buzzer, there is a possibility that the elementary school student 50 may be involved in an incident or accident. According to the information processing method, when the sound acquisition unit 13 acquires the sound of a security buzzer carried by the elementary school student 50 while the vehicle 10 is traveling, warning notification information including a surrounding image showing the elementary school student 50 carrying the security buzzer is transmitted from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the warning notification information from the vehicle 10, the server device 20 transmits the warning notification information to the information terminal 60.

[0048] This allows elementary school staff to know if an elementary school student may have been involved in an incident or accident. Staff can also identify the location where the incident or accident may have occurred. (4) When the vehicle 10 is traveling during a time other than the time when the elementary school student 50 is going to and from school, the imaging unit 11 of the vehicle 10 may capture the image of the elementary school student 50. The elementary school student 50, dressed in a manner that allows the control device 30 to determine that the elementary school student 50 is captured in the peripheral image, rarely goes out during a time other than the time when the elementary school student 50 is going to and from school. If the elementary school student 50 goes out during such a time, the elementary school student 50 may be involved in an incident. Therefore, according to the information processing method, when it is determined that the elementary school student 50 is captured in the peripheral image generated by the imaging unit 11 of the vehicle 10 during a time other than the time when the elementary school student is going to and from school, notification information including the peripheral image is transmitted from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60.

[0049] This allows elementary school staff to know when 50 elementary school students are out and about outside of school hours. <Example of change> The above embodiment may be modified as follows: The above embodiment and each of the following modifications may be combined with each other within a range where no technical contradiction occurs.

[0050] 3, some of the processes may not be executed. For example, if the determination process related to the sound of the security buzzer is not executed, the process of step S104 and the processes of steps S106, S108, and S110 may be omitted. In this case, the process of step S112 is executed after the process of step S102.

[0051] Also, for example, if the process of determining whether the elementary school student 50 is out during the time period other than the time when the elementary school student 50 is going to and from school is not performed, the process of step S114 may be omitted. In this case, if the process circuit 31 determines in the process of step S112 that the elementary school student 50 is captured in the peripheral image (step S112; YES), the process proceeds to step S118.

[0052] Also, for example, when the process of determining whether or not the elementary school student 50 is in a risky area is not performed, the process of step S120 may be omitted. In this case, when it is determined in the process of step S118 that the elementary school student 50 is on the school route (step S118; NO), the series of processes shown in FIG. 3 is ended.

[0053] In the series of processes shown in Fig. 3, the process of determining whether the elementary school student 50 is in a position away from the school route and the process of determining whether the elementary school student 50 is in a dangerous area are prioritized over the process of determining whether the elementary school student 50 is in a time period other than the time to and from school, but this is not limited to this. Any of these processes may be prioritized. In other words, the order of steps S114, S118, and S120 may be interchanged.

[0054] In the process of determining whether the elementary school student 50 is away from the school route, if an exceptional condition is satisfied, the processing circuit 31 may not notify the information terminal 60 of the notification information even if the elementary school student 50 is away from the school route. The exceptional condition includes, for example, being on a route used when traveling from home to the after-school care center. In this case, the storage unit 33 stores after-school care route information in which an elementary school is associated with position information indicating a route used when traveling from home to the after-school care center by the elementary school student 50 who attends the elementary school. When the processing circuit 31 determines that the elementary school student 50 is included in the surrounding image, it acquires the position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11. Next, the processing circuit 31 reads out from the storage unit 33 after-school care route information corresponding to an elementary school having a school district including the current position of the vehicle 10. Then, based on the location information acquired by the location information acquisition unit 12 and the after-school route information read from the storage unit 33, the processing circuit 31 determines whether or not the location indicated by the location information is included in the route used by the elementary school student 50 when traveling from his / her home to the after-school program. When the processing circuit 31 determines that the location indicated by the location information is on the route used by the elementary school student 50 when traveling from his / her home to the after-school program, it does not transmit notification information to the server device 20 even if the elementary school student 50 is away from the school route. With this configuration, it is possible to prevent the information terminal 60 from notifying unnecessary notification information when the elementary school student 50 is attending the after-school program after school hours.

[0055] The processing circuit 31 has been described as determining whether or not the security buzzer carried by the elementary school student 50 is sounding based on the reference sound information 334 and the sound information generated by the sound acquisition unit 13, but this is not limited to the above. The processing circuit 31 may determine whether or not the security buzzer carried by the elementary school student 50 is sounding using a trained model that has been machine-learned using the sound information of the security buzzer carried by the elementary school student 50. For example, when sound information is input, the trained model outputs whether or not the security buzzer is sounding. The processing circuit 31 inputs the sound information generated by the sound acquisition unit 13 into the trained model, and determines whether or not the security buzzer carried by the elementary school student 50 is sounding based on the output of the trained model.

[0056] Instead of using the trained model 331 to determine whether or not an elementary school student 50 is in the surrounding image, the processing circuit 31 may determine whether or not an elementary school student 50 is in the surrounding image based on the results obtained by general image analysis.

[0057] The imaging unit 11 may be a stereo camera. The imaging unit 11 may be provided with multiple cameras (for example, a first camera and a second camera), may capture images in the same direction using multiple cameras, or may capture images using the first camera under normal circumstances and the second camera or both the first and second cameras when a predetermined condition is satisfied. The first camera may capture images close to the imaging unit 11, and the second camera may capture images far from the imaging unit 11.

[0058] The imaging unit 11 may be attached to any location of the vehicle 10 as long as it can capture an image of the surroundings of the vehicle 10 and generate a surrounding image. For example, the imaging unit 11 may be attached to the top of the rear windshield or the back door when capturing an image of the rear of the vehicle 10. The imaging unit 11 may be attached to a door mirror when capturing an image of the side and rear side of the vehicle 10.

[0059] The location information acquired by the location information acquisition unit 12 may be identified or supplemented by an INS (Inertial Navigation System) that uses the outputs of various sensors equipped in the vehicle 10. The commuting route information 332 and the risk area information 333 may be updated sequentially according to the traveling position of the vehicle 10. For example, the control device 30 transmits the location information acquired by the location information acquisition unit 12 to an information providing server included in the information processing system 1. The information providing server identifies the school district of the received location information, and transmits the commuting route information 332 and the risk area information 333 related to the identified elementary school to the control device 30. The control device 30 transmits notification information and caution notification information to the server device 20 of the corresponding elementary school while executing various determination processes based on the received commuting route information 332 and the risk area information 333.

[0060] The manner in which the information terminal 60 notifies the notification information or the warning notification information is not limited to displaying it on a display unit. The information terminal 60 may be realized by, for example, a control device that controls an in-school broadcast at an elementary school. In this case, the information terminal 60 may notify the staff of the elementary school by broadcasting a sound indicating that the elementary school student 50 is at the location indicated by the location information included in the notification information or the warning notification information.

[0061] The server device 20 may be installed at a location other than an elementary school as long as it is installed outside the vehicle 10. For example, the server device 20 may be installed at the business premises of a business entity that provides the information processing system 1, or may be installed at a facility that manages elementary schools, such as a city hall. The server device 20 does not have to be installed at each elementary school. In this case, the server device 20 identifies the corresponding elementary school based on the location information included in the notification information or warning notification information received from the control device 30. The server device 20 notifies the notification information or warning notification information to an information terminal 60 installed at the identified elementary school or to an information terminal 60 of an employee belonging to the elementary school.

[0062] In the above description, the processing circuit 31 performs various determinations based on the surrounding image generated by the imaging unit 11, but this is not limited to the above. Some of the various determinations performed by the processing circuit 31 may be performed by the processing circuit 21 of the server device 20. In this case, the processing circuit 31 transmits to the server device 20 a surrounding image generated by the imaging unit 11, which includes the elementary school student 50, in association with the position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11. The processing circuit 21 performs various determinations based on the surrounding image and the position information received from the control device 30. For example, the processing circuit 21 determines whether or not the elementary school student in the surrounding image is located away from the school route based on the received position information. Then, when the processing circuit 21 determines that the elementary school student 50 in the surrounding image is located away from the school route, it transmits notification information including the surrounding image and the position information associated with the surrounding image to the information terminal 60. In this case, unlike the above embodiment, the server device 20 does not need to provide the school route information 332 to the vehicle 10.

[0063] For example, the processing circuit 21 determines whether or not the elementary school student 50 shown in the peripheral image is in a dangerous area in a school district based on the peripheral image and the position information received from the control device 30. When the processing circuit 21 determines that the elementary school student 50 shown in the peripheral image is in a dangerous area, the processing circuit 21 transmits notification information including the peripheral image showing the elementary school student 50 and the position information associated with the peripheral image to the information terminal 60. In this case, unlike the above embodiment, the server device 20 does not need to provide the dangerous area information 333 to the vehicle 10.

[0064] Further, for example, when it can be determined that a human being, not limited to an elementary school student, is included in the peripheral image, the control device 30 of the vehicle 10 may transmit the peripheral image and the position information of the vehicle 10 at the time when the peripheral image was generated to the server device 20 in association with each other. In this case, the processing circuit 21 of the server device 20 may input the received peripheral image to the trained model 331, or the like, to determine whether or not the elementary school student 50 is included in the peripheral image. When it is determined that the elementary school student 50 is included in the peripheral image, the processing circuit 21 may sequentially execute the processes from step S112 shown in FIG. 3 onward.

[0065] When the security alarm carried by the elementary school student 50 is sounding, the processing circuit 31 may transmit notification information instead of the process of transmitting the caution notification information to the server device 20. In this case, the notification information includes, for example, at least a surrounding image in which the elementary school student 50 carrying the security alarm is captured, and position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11.

[0066] Some or all of the components of the processing circuit 21 and the processing circuit 31 may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. [Explanation of symbols]

[0067] 1...information processing system, 10...vehicle, 11...imaging unit, 12...location information acquisition unit, 13...sound acquisition unit, 20...server device, 21, 31...processing circuit, 22, 32...communication unit, 23, 33...memory unit, 30...control device, 40...communication network, 50...elementary school student, 60...information terminal, 331...trained model, 332...school route information, 333...dangerous area information, 334...reference sound information.

Claims

1. An information processing method executed in an information processing system including a vehicle including an imaging unit that generates a surrounding image by imaging a surrounding situation, a position information acquisition unit that acquires its own position information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, wherein the surrounding image and the position information are transmitted from the communication unit of the vehicle to the server device via a communication network, The communication unit of the vehicle transmits to the server device notification information including the surrounding image in which the elementary school student is captured and position information of the vehicle when the surrounding image was generated by the imaging unit; The server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including the surrounding image showing an elementary school student who is away from a school commuting route of the elementary school and location information of the vehicle when the surrounding image was generated by the imaging unit; An information processing method comprising:

2. The server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including the surrounding image showing an elementary school student in a dangerous area within the school district of the elementary school and location information of the vehicle when the surrounding image was generated by the imaging unit. The information processing method according to claim 1 .

3. The vehicle includes a sound acquisition unit that acquires surrounding sounds, When the sound acquisition unit acquires a sound of a security alarm carried by an elementary school student, the communication unit of the vehicle transmits to the server device warning notification information including the surrounding image in which the elementary school student is captured and position information of the vehicle when the surrounding image was generated by the imaging unit; When the server device receives the warning notification information, the server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of the warning notification information. The information processing method according to claim 1 .

4. The notification information further includes a processing circuit of the server device notifying at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including the surrounding image generated by the imaging unit during a time period other than when the elementary school is going to or from school and including elementary school children, and location information of the vehicle when the surrounding image was generated by the imaging unit. The information processing method according to claim 1 .

Images