Information processing method

Abstract

The invention provides an information processing method. The information processing method is executed by an information processing system provided with a vehicle and a server device. The vehicle is provided with an imaging unit, a position information acquisition unit, and a communication unit. The information processing method includes: transmitting notification information to a server device through a communication unit of a vehicle, the notification information including a peripheral image of a pupil and position information of the vehicle when the peripheral image is generated by an imaging unit; the server device notifies, to the information terminal, notification information including, among the notification information, a peripheral image of a primary school student at a place away from the general school route of the primary school, and position information of the vehicle when the peripheral image is generated by the imaging unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing method. Background Art

[0002] As disclosed in Japanese Patent Application Laid-Open No. 2016-066122, there has been known a device that sends an alert email when the location of a mobile terminal held by an elementary school student commuting to school on a pre-registered commuting route deviates from the commuting route. Summary of the invention

[0003] In the method disclosed in the above document, if each elementary school student does not have a portable terminal, it is impossible to determine whether the elementary school student has left the commuting route.

[0004] In one embodiment of the present disclosure, an information processing method performed by an information processing system is provided. The information processing system includes a vehicle and a server device provided outside the vehicle. The vehicle includes: a photographing unit that generates a surrounding image by photographing surrounding conditions; a position information acquisition unit that acquires its own position information; and a communication unit that communicates with the outside. The information processing system is configured to send the surrounding image and the position information from the communication unit of the vehicle to the server device via a communication network. The information processing method includes the following processing: sending notification information including the surrounding image of the elementary school student and the position information of the vehicle when the surrounding image is generated by the photographing unit to the server device through the communication unit of the vehicle; and notifying the notification information including the surrounding image of the elementary school student at a place away from the commuting route of the elementary school and the position information of the vehicle when the surrounding image is generated by the photographing unit in the notification information to at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school through the server device. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A diagram for explaining an information processing system.

[0006] Figure 2 A diagram for explaining a server device and a control device.

[0007] Figure 3 The following is a flowchart showing an example of processing executed in the control device.

[0008] Figure 4 The following is a flowchart showing an example of processing executed in the server device. DETAILED DESCRIPTION

[0009] <Implementation Method>

[0010] Below, according to Figures 1 to 4 An embodiment of the information processing method will be described.

[0011] <Overall Configuration of Information Processing System 1>

[0012] like Figure 1 As shown, an information processing system 1 for implementing the information processing method includes, for example, one or more vehicles 10 and one or more server devices 20. The vehicle 10 includes, for example, a photographing unit 11, a position information acquisition unit 12, a sound acquisition unit 13, and a control device 30. The control device 30 and the server device 20 are connected via a communication network 40 in a manner capable of receiving and sending information. The communication network 40 includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0013] The photographing unit 11 is a digital camera that utilizes a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). An example of the photographing unit 11 is a digital camera that constitutes a driving recorder. The photographing unit 11 photographs the surrounding conditions of the vehicle 10. Depending on the installation position of the photographing unit 11 on the vehicle 10, the photographing unit 11 can photograph the front, rear, and sides of the vehicle 10. The photographing unit 11 generates a surrounding image by photographing the surrounding conditions of the vehicle 10. When the vehicle 10 is traveling on a commuting route to an elementary school, an elementary school student 50 is sometimes photographed in the surrounding image generated by the photographing unit 11. The commuting route refers to the path that the elementary school student 50 takes when going to school.

[0014] The position information acquisition unit 12 is, for example, a GNSS (Global Navigation Satellite System) receiver. The position information acquisition unit 12 identifies the position of the vehicle 10 based on a signal received from a GNSS satellite, and acquires position information indicating the position of the vehicle 10 .

[0015] The sound acquisition unit 13 is, for example, an audio device that converts input sound into an electrical signal and outputs the signal, and is composed of a single microphone. The sound acquisition unit 13 is installed at any location of the vehicle 10. The sound acquisition unit 13 acquires sounds around the vehicle 10 and generates sound information.

[0016] When the pupil 50 is captured in the surrounding image generated by the imaging unit 11 and the pupil 50 satisfies various conditions, the control device 30 sends information related to the pupil 50 to the server device 20. Details of the various conditions and the information sent by the control device 30 to the server device 20 will be described below.

[0017] The server device 20 outputs the information received from the control device 30 to, for example, an information terminal 60 installed at an elementary school and an information terminal 60 owned by an elementary school staff. Thus, the elementary school staff can grasp information related to the elementary school students 50. The information terminal 60 includes, for example, a smartphone, a tablet terminal, and a personal computer.

[0018] <Configuration of Control Device 30>

[0019] like Figure 2 As shown, the control device 30 includes, for example, a processing circuit 31, a communication unit 32, and a storage unit 33. The processing circuit 31 is composed of a hardware processor such as a CPU (Central Processing Unit). The processing circuit 31 can perform various processes by executing a program (software) through the hardware processor. The program can be pre-stored in the storage unit 33. The storage unit 33 is implemented by a non-volatile storage medium such as an HDD (Hard Disk Drive) or a flash memory. The communication unit 32 includes, for example, a communication circuit or a communication module, and receives and sends various data with the server device 20 and other devices based on the control of the processing circuit 31.

[0020] The storage unit 33 stores, for example, a learned model 331, commuting route information 332, dangerous area information 333, and standard sound information 334. The learned model 331 is a model that performs machine learning to determine whether the elementary school student 50 is photographed in the surrounding image generated by the imaging unit 11. An example of the learned model 331 is a neural network. When the surrounding image is input as an input variable to the learned model 331, the learned model 331 outputs an output variable indicating the probability that the elementary school student 50 is photographed in the surrounding image. For example, if the probability of photographing the elementary school student 50 in the surrounding image is higher, the learned model 331 outputs a larger value as the output variable. In this case, the processing circuit 31 of the control device 30 determines that the elementary school student 50 is photographed in the surrounding image when the output variable is greater than the threshold. On the other hand, the processing circuit 31 determines that the elementary school student 50 is not photographed in the surrounding image when the output variable is less than the threshold.

[0021] Here, when the elementary school student 50 is going to school, the elementary school student 50 is carrying a school bag or wearing a hat or school uniform designated by the elementary school. The following machine learning is pre-implemented in the learned model 331, that is, when a child who satisfies at least one of the following two conditions (A1) and (A2) is photographed in the surrounding image, it can be determined that the elementary school student 50 is photographed in the surrounding image.

[0022] (A1) Carrying a school bag.

[0023] (A2) Wear the clothing specified by the elementary school.

[0024] The commuting route information 332 is information in which an elementary school and location information indicating a commuting route set within the school district of the elementary school are associated with each other.

[0025] Dangerous area information 333 is information that establishes a correspondence between a primary school and location information of a dangerous area within the school district of the primary school. A dangerous area is an area where accidents are prone to occur. Examples of dangerous areas include water conservancy facilities such as rivers and reservoirs, roads with heavy traffic, and facilities where large vehicles frequently enter and exit.

[0026] The standard sound information 334 is information indicating the sound of a general alarm buzzer.

[0027] The processing circuit 31 determines whether there is an elementary school student 50 away from the commuting route of the elementary school based on the surrounding image generated by the imaging unit 11, the position information obtained by the position information obtaining unit 12, and the information stored in the storage unit 33. Specifically, the processing circuit 31 determines whether the elementary school student 50 is photographed in the surrounding image based on the output variable of the learned model 331 when the surrounding image generated by the imaging unit 11 is input to the learned model 331. When the processing circuit 31 determines that the elementary school student 50 is photographed in the surrounding image, it obtains the position information obtained by the position information obtaining unit 12 when the surrounding image is generated by the imaging unit 11.

[0028] Next, the processing circuit 31 reads out the school route information 332 of the elementary school having the school zone including the current position of the vehicle 10 from the storage unit 33. Then, based on the position information obtained from the position information acquisition unit 12 and the school route information 332 read out from the storage unit 33, the processing circuit 31 determines whether the position indicated by the position information is included in the school route of the elementary school. If the processing circuit 31 determines that the position indicated by the position information is not included in the school route of the elementary school, it determines that the elementary school student 50 is at a place away from the school route of the elementary school. On the other hand, if the processing circuit 31 determines that the position indicated by the position information is included in the school route of the elementary school, it determines that the elementary school student 50 is on the school route of the elementary school.

[0029] When the processing circuit 31 determines that the elementary school student 50 is at a place away from the elementary school's commuting route, the processing circuit 31 sends notification information to the elementary school's server device 20 via the communication network 40. The notification information sent to the server device 20 includes, for example, a surrounding image when the elementary school student 50 at a place away from the commuting route is photographed, and position information acquired by the position information acquisition unit 12 when the surrounding image is generated by the photographing unit 11. At this time, the processing circuit 31 identifies an elementary school having a school district including a position indicated by the position information based on the position information included in the notification information sent to the server device 20. Then, the processing circuit 31 sends notification information to the server device 20 of the identified elementary school.

[0030] Furthermore, even when it is determined that the elementary school student 50 is not away from the commuting route to the elementary school, if the elementary school student 50 is captured in the surrounding image generated by the shooting unit 11 during a time period other than the time of going to and from school, the processing circuit 31 will send notification information to the server device 20. When the processing circuit 31 determines that the elementary school student 50 is captured in the surrounding image, it determines whether the moment when the surrounding image is generated by the shooting unit 11 is a time period other than the time of going to and from school. The time period for going to and from school includes the time period for going to and from school. An example of the time period for going to and from school is between 7 am and 9 am. An example of the time period for going to and from school is between 2 pm and 5 pm.

[0031] Here, it is rare for the elementary school student 50 to walk outside with a school bag on his back during the time period other than going to and from school. In addition, it is rare for the elementary school student 50 to walk outside in the clothes designated by the elementary school during the time period other than going to and from school. Therefore, when it is determined that the elementary school student 50 is photographed in the surrounding image captured by the shooting unit 11 during the time period other than going to and from school, it is preferable to notify the elementary school and the staff of the elementary school. Therefore, when the elementary school student 50 is photographed in the surrounding image generated by the shooting unit 11 during the time period other than going to and from school, the processing circuit 3 sends notification information to the server device 20 of the elementary school via the communication network 40.

[0032] In addition, when the processing circuit 31 determines that the elementary school student 50 is captured in the surrounding image, even if the elementary school student 50 has not left the school route but is in a dangerous area, the notification information is also sent to the server device 20. The processing circuit 31 determines whether the elementary school student 50 is in the dangerous area designated by the elementary school based on the surrounding image generated by the imaging unit 11 and the position information obtained by the position information acquisition unit 12. When the processing circuit 31 determines that the elementary school student 50 is captured in the surrounding image, the processing circuit 31 obtains the position information obtained by the position information acquisition unit 12 when the surrounding image is generated by the imaging unit 11.

[0033] Next, the processing circuit 31 reads the dangerous area information 333 specified by the elementary school having the school zone including the current position of the vehicle 10 from the storage unit 33. Then, the processing circuit 31 determines whether the position indicated by the position information is a dangerous area based on the position information obtained by the position information obtaining unit 12 and the dangerous area information 333 read from the storage unit 33. If the processing circuit 31 determines that the position indicated by the position information is a dangerous area, it determines that the elementary school student 50 is in the dangerous area.

[0034] When the processing circuit 31 determines that the elementary school student 50 is in a dangerous area, it sends notification information to the elementary school server device 20 via the communication network 40. The notification information includes, for example, the surrounding image when the imaging unit 11 captures the elementary school student 50 in the dangerous area, and the position information obtained by the position information acquisition unit 12 when the imaging unit 11 generates the surrounding image.

[0035] In addition, the processing circuit 31 determines whether the alarm buzzer held by the primary school student 50 is ringing based on the sound information generated by the sound acquisition unit 13. When the processing circuit 31 determines that the alarm buzzer is ringing, it sends attention notification information to the server device 20. The processing circuit 31 determines whether the sound of the alarm buzzer held by the primary school student 50 is obtained based on the sound information generated by the sound acquisition unit 13. When the standard sound information 334 and the sound information generated by the sound acquisition unit 13 are consistent, the processing circuit 31 determines that the sound of the alarm buzzer held by the primary school student 50 is obtained. For example, when any one of the following conditions (B1) and (B2) is satisfied, the processing circuit 31 can regard that the standard sound information 334 and the sound information generated by the sound acquisition unit 13 are consistent.

[0036] (B1) The frequency components of the sound of the standard sound information 334 match the frequency components of the sound of the sound information.

[0037] (B2) The sound pattern of the standard sound information 334 matches the sound pattern of the sound information.

[0038] When the processing circuit 31 determines that the sound of the alarm buzzer held by the elementary school student 50 is obtained, the processing circuit 31 sends the attention notification information to the server device 20 of the elementary school via the communication network 40. The attention notification information includes, for example, the surrounding image generated by the imaging unit 11 and the position information obtained by the position information obtaining unit 12 when the surrounding image is generated by the imaging unit 11. The surrounding image included in the attention notification information is the surrounding image generated by the imaging unit 11 at the moment when the sound of the alarm buzzer is obtained, and is the surrounding image of the elementary school student 50 holding the alarm buzzer.

[0039] <Configuration of Server Device 20>

[0040] like Figure 2 As shown, the server device 20 includes, for example, a processing circuit 21, a communication unit 22, and a storage unit 23. The processing circuit 21 is composed of a hardware processor such as a CPU. The processing circuit 21 can perform various processes by executing a program (software) by the hardware processor. The program can be pre-stored in the storage unit 23. The storage unit 23 is implemented by a non-volatile storage medium such as an HDD or a flash memory. The communication unit 22 includes, for example, a communication circuit or a communication module, and receives and sends various data with the control device 30 and other devices based on the control of the processing circuit 21.

[0041] In addition, one or more information terminals 60 are connected to the server device 20. The communication unit 22 communicates with the information terminal 60 via wired or wireless communication. At this time, the server device 20 and the information terminal 60 can communicate via the above-mentioned communication network 40, or can use other networks for communication. In the case of wireless communication, the communication method between the communication unit 22 and the information terminal 60 can be short-range wireless communication such as Wi-Fi (registered trademark) and Bluetooth (registered trademark). In addition, the communication method between the communication unit 22 and the information terminal 60 can also be communication via the Internet, WAN, LAN, provider terminal, wireless communication network, wireless base station, dedicated line and other networks. The information terminal 60 is implemented by, for example, a smart phone, a tablet, etc.

[0042] When the processing circuit 21 receives the notification information or the attention notification information from the control device 30, it notifies the information terminal 60 of the received information. Specifically, the processing circuit 21 notifies the information terminal 60 that the notification information or the attention notification information has been uploaded to the server device 20. When the staff who knows that the notification from the server device 20 has been delivered to the information terminal 60 operates the operation unit of the information terminal 60, the notification information or the attention notification information is downloaded from the server device 20 to the information terminal 60. Then, in the information terminal 60, the downloaded information is displayed on the display unit of the information terminal 60.

[0043]

[0044] Reference Figure 3 , a series of processing executed in the control device 30 is described. Figure 3 The processing shown is repeatedly performed at predetermined time intervals. First, the processing circuit 31 obtains the surrounding image generated by the imaging unit 11 (step S100). Next, the processing circuit 31 obtains the position information from the position information acquisition unit 12 (step S102). The position information obtained in step S102 is the position information of the vehicle 10 when the surrounding image obtained in step S100 is generated by the imaging unit 11. Next, the processing circuit 31 obtains the sound information from the sound acquisition unit 13 (step S104).

[0045] Next, the processing circuit 31 determines whether the sound acquisition unit 13 has acquired the sound of the alarm buzzer held by the elementary school student 50 based on the sound information acquired from the sound acquisition unit 13 (step S106). When the standard sound information indicating the standard sound of the alarm buzzer is consistent with the sound information generated by the sound acquisition unit 13, the processing circuit 31 determines that the sound of the alarm buzzer has been acquired. When the processing circuit 31 determines that the sound of the alarm buzzer has been acquired (step S106: Yes), it determines whether the elementary school student 50 is photographed in the surrounding image acquired in step S100 (step S108). The processing circuit 31 determines whether the elementary school student 50 is photographed in the surrounding image based on the output variable of the learned model 331 when the surrounding image acquired in step S100 is input to the learned model 331. When the processing circuit 31 determines that the elementary school student 50 is not photographed in the surrounding image (step S108: No), the series of processing ends.

[0046] When the processing circuit 31 determines that the elementary school student 50 is captured in the surrounding image (step S108: Yes), the communication unit 32 sends the attention notification information to the elementary school server device 20 (step S110). The attention notification information sent to the server device 20 in step S110 includes Figure 3 The peripheral image acquired in step S100 during the current execution of the series of processes shown in the figure and the position information acquired in step S102 during the current execution of the series of processes shown in the figure. Thereafter, the processing circuit 31 ends the series of processes.

[0047] When the processing circuit 31 determines that the sound of the alarm buzzer held by the pupil 50 is not obtained (step S106: No), the processing proceeds to step S112. The processing circuit 31 performs the same processing as the above-mentioned step S108 to determine whether the pupil 50 is captured in the surrounding image obtained in step S100 (step S112). When the processing circuit 31 determines that the pupil 50 is not captured in the surrounding image (step S112: No), since there is no information to be notified to the server device 20, a series of processing ends.

[0048] When the processing circuit 31 determines that the primary school student 50 is captured in the surrounding image (step S112: Yes), it determines whether the time when the surrounding image is generated by the imaging unit 11 is a time period other than the time when the student goes to and from school (step S114). When the processing circuit 31 determines that the time when the surrounding image is generated by the imaging unit 11 is a time period other than the time when the student goes to and from school (step S114: Yes), it causes the communication unit 32 to send notification information to the server device 20 of the primary school (step S116). The notification information sent to the server device 20 in step S116 includes Figure 3 The peripheral image acquired in step S100 during the current execution of the series of processes shown, and the position information acquired in step S102 during the current execution of the series of processes are displayed. Thereafter, the processing circuit 31 ends the series of processes.

[0049] When the processing circuit 31 determines that the time when the surrounding image is generated by the imaging unit 11 is the time period for going to and from school (step S114: No), it determines whether the elementary school student 50 is away from the commuting route (step S118). The processing circuit 31 determines whether the elementary school student 50 is away from the commuting route based on the position information obtained in step S102 and the elementary school commuting route information 332. When the processing circuit 31 determines that the elementary school student 50 is away from the commuting route (step S118: Yes), the processing proceeds to step S116.

[0050] When the processing circuit 31 determines that the elementary school student 50 is on the commuting route (step S118: No), it determines whether the elementary school student 50 is in a dangerous area (step S120). The processing circuit 31 determines whether the elementary school student 50 is in a dangerous area based on the position information obtained in step S102 and the dangerous area information 333. When the processing circuit 31 determines that the elementary school student 50 is in a dangerous area (step S120: Yes), the processing proceeds to step S116. When the processing circuit 31 determines that the elementary school student 50 is in a place that is not a dangerous area (step S120: No), the series of processing ends.

[0051] That is, even if it is determined that the elementary school student 50 is captured in the surrounding image obtained in step S100, if each of the following three conditions (C1), (C2) and (C3) is met, the processing circuit 31 will not send notification information to the server device 20, but will end a series of processing.

[0052] (C1) The time when the surrounding image is generated is the time when students go to and from school.

[0053] (C2) An elementary school student 50 is on the commuting route.

[0054] (C3) The elementary school student 50 is in a place other than the dangerous area on the commuting route.

[0055]

[0056] Reference Figure 4 , a series of processing executed in the server device 20 is described. Figure 4 The processing shown is repeatedly executed at predetermined time intervals. The processing circuit 21 determines whether the notification information or the attention notification information is received from the control device 30 (step S200). When the processing circuit 21 does not receive the notification information and the attention notification information from the control device 30 (step S200: No), the series of processing is terminated. When the processing circuit 21 receives the notification information or the attention notification information from the control device 30 (step S200: Yes), the information terminal 60 is notified of the received information (step S202). Thereafter, the processing circuit 21 terminates the series of processing.

[0057] <Functions and effects of implementation>

[0058] According to the above-described embodiment, the following operations and effects can be obtained.

[0059] (1) When the vehicle 10 is traveling, the imaging unit 11 generates a surrounding image by imaging the surroundings of the vehicle 10. At this time, the position information acquisition unit 12 acquires the position information of the vehicle 10 when the imaging unit 11 captures the surrounding image.

[0060] The processing circuit 31 of the control device 30 determines whether the pupil 50 is captured in the surrounding image generated by the imaging unit 11. The processing circuit 31 transmits notification information to the server device 20 if the pupil 50 captured in the surrounding image is away from the commuting route.

[0061] When receiving the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60. That is, the server device 20 transmits the notification information including the surrounding image of the elementary school student 50 at a location away from the commuting route and the position information when the surrounding image was generated by the imaging unit 11 to the information terminal 60.

[0062] Thus, according to the information processing method, even if the elementary school student 50 does not have an information terminal, it is possible to notify the elementary school or the staff of the elementary school that the elementary school student 50 has left the commuting route.

[0063] (2) When the elementary school student 50 is in a dangerous area within the school district of the elementary school, the elementary school student 50 may be involved in an accident. Therefore, according to the information processing method, when the camera unit 11 captures the elementary school student 50 in the dangerous area while the vehicle 10 is traveling, notification information including the surrounding image of the elementary school student 50 is sent from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 sends the notification information to the information terminal 60. That is, the server device 20 sends the notification information including the surrounding image of the elementary school student 50 when the camera unit 11 captures the elementary school student 50 in the dangerous area to the information terminal 60.

[0064] Thus, according to the information processing method, it is possible to enable the staff of the elementary school to understand that the elementary school student 50 is in a dangerous area before the elementary school student 50 is actually involved in an accident.

[0065] (3) When the pupil 50 operates his own alarm buzzer, the pupil 50 may be involved in an incident or accident. Therefore, according to the information processing method, when the sound acquisition unit 13 acquires the sound of the alarm buzzer held by the pupil 50 while the vehicle 10 is traveling, attention notification information including a surrounding image of the pupil 50 holding the alarm buzzer is sent from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the attention notification information from the vehicle 10, the server device 20 sends the attention notification information to the information terminal 60.

[0066] Thereby, the staff of the elementary school can understand that the elementary school student 50 may be involved in an incident or accident. In addition, the staff can identify the place where the incident or accident may have occurred.

[0067] (4) When the vehicle 10 is traveling at a time other than the time when the vehicle 10 is going to and from school, the camera unit 11 of the vehicle 10 may capture the pupil 50. The control device 30 can determine that the pupil 50 dressed like the pupil 50 captured in the peripheral image rarely goes out at a time other than the time when the vehicle 10 is going to and from school. If the pupil 50 goes out at such a time, the pupil 50 may be involved in an incident. Therefore, according to the information processing method, when it is determined that the pupil 50 is captured in the peripheral image generated by the camera unit 11 of the vehicle 10 at a time other than the time when the vehicle 10 is going to and from school, notification information including the peripheral image is sent from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 sends the notification information to the information terminal 60.

[0068] This allows the staff of the elementary school to understand that the elementary school student 50 is out at a time other than the time of going to and from school.

[0069] <Change Example>

[0070] The above-mentioned embodiment can also be modified as follows. In addition, the above-mentioned embodiment and the following individual examples can be combined within the range of no technical contradiction.

[0071] · Figure 3 Some of the processes shown may not be performed. For example, if the judgment process related to the sound of the alarm buzzer is not performed, the processes of step S104, step S106, step S108, and step S110 may be omitted. In such a case, the process of step S112 is performed after the process of step S102.

[0072] In addition, for example, if the processing of determining whether the pupil 50 is out at a time other than going to and from school is not performed, the processing of step S114 may be omitted. In such a case, when the processing circuit 31 determines in the processing of step S112 that the pupil 50 is captured in the peripheral image (step S112: Yes), the processing proceeds to step S118.

[0073] In addition, for example, if the process of determining whether the pupil 50 is in a dangerous area is not performed, the process of step S120 may be omitted. In such a case, if it is determined in the process of step S118 that the pupil 50 is on the commuting route (step S118: No), the process ends. Figure 3 A series of processing is shown.

[0074] Although Figure 3 In the series of processes shown, the determination process of whether the pupil 50 is in a time period other than the time of going to and from school is prioritized over the determination process of whether the pupil 50 is away from the commuting route and the determination process of whether the pupil 50 is in a dangerous area, but the present invention is not limited thereto. Any of these processes may be prioritized. That is, the processing order of step S114, step S118, and step S120 may be interchanged for execution.

[0075] It is also possible to set that the processing circuit 31 does not notify the information terminal 60 of the notification information when the exception condition is satisfied in the judgment process of whether the elementary school student 50 is in a place away from the commuting route, even if the elementary school student 50 is in a place away from the commuting route. The exception condition includes, for example, a case where the elementary school student 50 is on the path used when moving from his home to the after-school ... Then, the processing circuit 31 determines whether the position indicated by the position information is included in the route used by the primary school student 50 when moving from his home to the after-school ...

[0076] Although the processing circuit 31 has been described as judging whether the alarm buzzer held by the pupil 50 is ringing based on the standard sound information 334 and the sound information generated by the sound acquisition unit 13, the present invention is not limited to this. Alternatively, the processing circuit 31 may judge whether the alarm buzzer held by the pupil 50 is ringing by using a learned model that has been machine-learned using the sound information of the alarm buzzer held by the pupil 50. Regarding the learned model, for example, when sound information is input, it outputs whether the alarm buzzer is ringing. The processing circuit 31 inputs the sound information generated by the sound acquisition unit 13 into the learned model, and judges whether the alarm buzzer held by the pupil 50 is ringing based on the output of the learned model.

[0077] Instead of using the learned model 331 to determine whether the elementary school student 50 is captured in the surrounding images, the processing circuit 31 may determine whether the elementary school student 50 is captured in the surrounding images based on the results obtained by ordinary image analysis.

[0078] The photographing unit 11 may also be a stereo camera. In addition, the photographing unit 11 may be provided with a plurality of cameras (for example, a first camera and a second camera), and may use a plurality of cameras to photograph in the same direction. It may also be configured such that the first camera is used to photograph normally, and the second camera or both the first camera and the second camera are used to photograph when a predetermined condition is met. In addition, it may be configured such that the first camera is used to photograph the near side of the photographing unit 11, and the second camera is used to photograph the far side of the photographing unit 11.

[0079] The camera unit 11 may be installed at any position of the vehicle 10 as long as it can capture the surroundings of the vehicle 10 and generate a surrounding image. For example, when the camera unit 11 captures the rear of the vehicle 10, it may be installed on the upper part of the rear windshield, the rear tailgate, etc. In addition, when the camera unit 11 captures the side and rear side of the vehicle 10, it may be installed on a door mirror, etc.

[0080] The position information acquired by the position information acquisition unit 12 may be determined or supplemented by an INS (Inertial Navigation System) using outputs of various sensors included in the vehicle 10 .

[0081] The school route information 332 and the dangerous area information 333 may also be updated sequentially according to the driving position of the vehicle 10. For example, the control device 30 sends the location information obtained by the location information obtaining unit 12 to the information providing server provided in the information processing system 1. The information providing server determines the school district of the received location information, and sends the school route information 332 and the dangerous area information 333 related to the determined elementary school to the control device 30. Based on the received school route information 332 and the dangerous area information 333, the control device 30 performs various judgment processes and sends the notification information and the caution notification information to the server device 20 of the corresponding elementary school.

[0082] The notification method of the notification information and attention notification information of the information terminal 60 is not limited to the display by the display unit. The information terminal 60 can also be implemented by, for example, a control device that controls the school broadcast of the elementary school. In this case, the information terminal 60 can also notify the staff of the elementary school, etc., through the school broadcast, of a sound indicating that the elementary school student 50 is at the location of the location information included in the notification information and attention notification information.

[0083] The server device 20 may be set at a location other than an elementary school as long as it is set outside the vehicle 10. The server device 20 may be set, for example, at the business location of the operator that provides the information processing system 1, or at a facility that manages an elementary school such as a municipal government agency. In addition, the server device 20 may not be set up in every elementary school. In such a case, the server device 20 determines the corresponding elementary school based on the location information contained in the notification information and attention notification information received from the control device 30. The server device 20 notifies the information terminal 60 set up in the determined elementary school and the information terminal 60 of the staff of the elementary school of the notification information and attention notification information.

[0084] In the above, although the processing circuit 31 performs various judgments based on the surrounding image generated by the shooting unit 11, it is not limited to this. It can also be set that the processing circuit 21 of the server device 20 performs a part of the various judgments implemented in the processing circuit 31. In this case, the processing circuit 31 associates the surrounding image generated by the shooting unit 11 and the surrounding image in which the elementary school student 50 is photographed with the position information obtained by the position information acquisition unit 12 when the surrounding image is generated by the shooting unit 11, and sends it to the server device 20. The processing circuit 21 performs various judgments based on the surrounding image and position information received from the control device 30. For example, the processing circuit 21 judges whether the elementary school student 50 photographed in the surrounding image is at a place away from the commuting route based on the received position information. Then, when the processing circuit 21 judges that the elementary school student 50 photographed in the surrounding image is at a place away from the commuting route, the notification information including the surrounding image and the position information associated with the surrounding image is sent to the information terminal 60. In such a case, unlike the above-described embodiment, the server device 20 may not provide the commuting route information 332 to the vehicle 10 .

[0085] In addition, for example, the processing circuit 21 determines whether the elementary school student 50 captured in the surrounding image is in a dangerous area within the school district based on the surrounding image and position information received from the control device 30. When the processing circuit 21 determines that the elementary school student 50 captured in the surrounding image is in a dangerous area, it sends notification information including the surrounding image in which the elementary school student 50 is captured and the position information associated with the surrounding image to the information terminal 60. In such a case, unlike the above-mentioned embodiment, the server device 20 may not provide the dangerous area information 333 to the vehicle 10.

[0086] Furthermore, for example, the control device 30 of the vehicle 10 may be configured to, when it is determined that a person other than an elementary school student is captured in the surrounding image, associate the surrounding image with the position information of the vehicle 10 when the surrounding image was generated, and transmit the association to the server device 20. In such a case, the processing circuit 21 of the server device 20 determines whether the elementary school student 50 is captured in the surrounding image by inputting the received surrounding image into the learned model 331. The processing circuit 21 may also be configured to, when it is determined that the elementary school student 50 is captured in the surrounding image, sequentially execute the following steps: Figure 3 The processing after step S112 is shown.

[0087] When the alarm buzzer held by the pupil 50 sounds, the processing circuit 31 may send notification information instead of sending the attention notification information to the server device 20. In such a case, the notification information includes at least a surrounding image of the pupil 50 holding the alarm buzzer and position information acquired by the position information acquisition unit 12 when the surrounding image is generated by the imaging unit 11.

[0088] Part or all of the structural elements of the processing circuit 21 and the processing circuit 31 may be implemented by hardware (circuit portion; including circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or may be implemented by the collaboration of software and hardware.

Claims

1. An information processing method, which is performed by an information processing system, The information processing system comprises: vehicle; a server device disposed outside the vehicle, The vehicle has: a photographing unit that generates a surrounding image by photographing surrounding conditions; a position information acquisition unit that acquires its own position information; and A communication unit that communicates with the outside, The information processing system is configured to transmit the surrounding image and the position information from the communication unit of the vehicle to the server device via a communication network. The information processing method includes the following processing, namely: transmitting, via the communication unit of the vehicle, notification information including the surrounding image of the elementary school student and the position information of the vehicle when the surrounding image was generated by the imaging unit to the server device; The server device is used to notify at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school, including the surrounding image of the elementary school students captured at a location away from the elementary school's commuting route and the position information of the vehicle when the surrounding image was generated by the capturing unit.

2. The information processing method according to claim 1, wherein: It also includes the following processing, namely, The server device is used to notify at least one of an information terminal owned by the primary school and an information terminal owned by a staff member of the primary school, including the surrounding image of the primary school students captured in a dangerous area within the school district of the primary school and the position information of the vehicle when the surrounding image was generated by the shooting unit.

3. The information processing method according to claim 1, wherein: The vehicle further includes a sound acquisition unit that acquires surrounding sounds. The information processing method further includes the following processing, namely: When the sound of the alarm buzzer held by the elementary school student is acquired by the sound acquisition unit, the attention notification information including the surrounding image of the elementary school student and the position information of the vehicle when the surrounding image is generated by the photography unit is sent to the server device through the communication unit of the vehicle; When the server device receives the attention notification information, the server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of the attention notification information.

4. The information processing method according to claim 1, wherein: It also includes the following processing, namely, The processing circuit of the server device is used to notify at least one of an information terminal owned by the primary school and an information terminal owned by a staff member of the primary school of the notification information including the surrounding image generated by the shooting unit and capturing primary school students during a time period other than the time when the primary school goes to and from school, and the position information of the vehicle when the surrounding image is generated by the shooting unit.

Images