Information processing method, program, and information processing system

Abstract

To make it possible to notify an elementary school as well as staff members of the elementary school of a state where elementary school children need to be notified thereof.SOLUTION: An information processing method is a method executed with an information processing system 1 comprising a vehicle 10 provided with an imaging part 11, a position information acquisition part 12, and a communication part 32; and a server device 20. The information processing method comprises the steps of: transmitting, to the server device 20, notification information including a peripheral image on which elementary school children are photographed, and position information of the vehicle 10 when the peripheral image is produced by the imaging part 11, with the communication part 32 in the vehicle 10; and notifying an information terminal 60 of notification information including a lost article image as a peripheral image on which no elementary school child is photographed, though personal belongings are photographed, and the position information of the vehicle 10 when the lost article image is produced by the imaging part 11, with the server device 20, among a plurality pieces of notification information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing method, a program, and an information processing system. [Background technology]

[0002] Conventionally, there is known a device that notifies an elementary school student by sending an alarm email if the location of a mobile terminal carried by the student, who commutes to school via a pre-registered route, is away from the route (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-066122 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the method disclosed in Patent Document 1, if each elementary school student does not have a mobile terminal, it is not possible to determine whether or not the elementary school student is in a state requiring notification. [Means for solving the problem]

[0005] An information processing method for solving the above problem is an information processing method executed in an information processing system including a vehicle having an imaging unit that generates a surrounding image by imaging the surrounding situation, a location information acquisition unit that acquires its own location information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, and configured so that the surrounding image and the location information are transmitted from the communication unit of the vehicle to the server device via a communication network, the information processing method including: the communication unit of the vehicle transmitting to the server device notification information including the surrounding image in which the elementary school student's belongings are captured and the location information of the vehicle at the time the surrounding image was generated by the imaging unit; and a processing circuit of the server device notifying at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including a lost item image, which is the surrounding image in which the elementary school student's belongings are captured but the elementary school student is not captured, and the location information of the vehicle at the time the lost item image was generated by the imaging unit.

[0006] The program for solving the above problem is a program executed on a vehicle control device that has an imaging unit that generates a surrounding image by imaging the surrounding situation, a location information acquisition unit that acquires the vehicle's own location information, and a communication unit that communicates with a server device installed outside the vehicle, wherein the server device is configured to communicate with at least one of an information terminal owned by an elementary school and an information terminal owned by a staff member of the elementary school, and causes the control device to acquire notification information including a lost item image, which is a surrounding image that shows an elementary school student's belongings but does not show the elementary school student around the belongings, and the location information of the vehicle at the time the lost item image was generated by the imaging unit, and to transmit the notification information from the communication unit to the server device.

[0007] An information processing system for solving the above problem is an information processing system comprising a vehicle equipped with an imaging unit that generates a surrounding image by imaging the surrounding situation, a location information acquisition unit that acquires its own location information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, wherein the communication unit of the vehicle transmits notification information including the surrounding image in which the elementary school student's belongings are captured and the location information of the vehicle at the time the surrounding image was generated by the imaging unit to the server device via a communication network, and the server device transmits notification information including a lost item image, which is the surrounding image in which the elementary school student's belongings are captured but the elementary school student is not captured, and the location information of the vehicle at the time the lost item image was generated by the imaging unit, to at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school. Effect of the Invention

[0008] According to the above configuration, even if the elementary school student does not have an information terminal, it is possible to notify the elementary school or elementary school staff that the elementary school student is in a state requiring notification. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram used to explain an information processing system. [Diagram 2] FIG. 2 is a diagram used to explain the server device and the control device. [Diagram 3] FIG. 3 is a flowchart illustrating an example of a process executed in the control device. [Figure 4] FIG. 4 is a flowchart illustrating an example of a process executed in the server device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Embodiment> Hereinafter, one embodiment of the information processing method will be described with reference to FIGS. <Overall configuration of information processing system 1> 1, an information processing system 1 that realizes the information processing method includes, for example, one or more vehicles 10 and one or more server devices 20. The vehicle 10 includes, for example, an imaging unit 11, a position information acquisition unit 12, and a control device 30. The control device 30 and the server device 20 are connected to each other via a communication network 40 so as to be able to transmit and receive information. The communication network 40 includes, for example, the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0011] The imaging unit 11 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). One example of the imaging unit 11 is a digital camera constituting a drive recorder. The imaging unit 11 captures the surroundings of the vehicle 10. Depending on the installation position of the imaging unit 11 on the vehicle 10, the imaging unit 11 can capture the front, rear, and sides of the vehicle 10. The imaging unit 11 generates a surrounding image by capturing the surroundings of the vehicle 10. When the vehicle 10 runs near an elementary school, the surrounding image generated by the imaging unit 11 may include elementary school students and their belongings 51. The belongings 51 are, for example, items that elementary school students are required to wear or carry when going to school, or items that elementary school students use in class. Specifically, the belongings 51 are a school bag, a hat, a uniform, a handbag, a water bottle, a recorder, and the like.

[0012] The position information acquisition unit 12 is, for example, a Global Navigation Satellite System (GNSS) receiver. The position information acquisition unit 12 identifies the position of the vehicle 10 based on a signal received from a GNSS satellite, and acquires position information indicating the position of the vehicle 10.

[0013] When belongings 51 are captured in the surrounding image generated by the imaging unit 11 but the elementary school student is not captured, and the situation in which belongings 51 are captured satisfies various conditions, the control device 30 transmits information about the elementary school student to the server device 20. Details of the various conditions and the information transmitted by the control device 30 to the server device 20 will be described later.

[0014] The server device 20 outputs the information received from the control device 30 to, for example, an information terminal 60 provided in the elementary school and an information terminal 60 owned by the elementary school staff. In this way, the elementary school staff can grasp information about the elementary school students. The information terminal 60 includes, for example, a smartphone, a tablet terminal, and a personal computer.

[0015] <Configuration of the control device 30> As shown in FIG. 2, the control device 30 includes, for example, a processing circuit 31, a communication unit 32, and a storage unit 33. The processing circuit 31 is configured with, for example, a hardware processor such as a CPU (Central Processing Unit). The hardware processor executes a program (software), which allows the processing circuit 31 to execute various processes. The program may be stored in the storage unit 33 in advance. The storage unit 33 is realized by, for example, a non-transitory storage medium such as a HDD (Hard Disk Drive) or a flash memory. The communication unit 32 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the server device 20 and other devices based on the control of the processing circuit 31.

[0016] The storage unit 33 stores, for example, a first trained model 331, a second trained model 332, and risk area information 333. The first trained model 331 is a model that performs machine learning to determine whether or not an elementary school student is included in a peripheral image generated by the imaging unit 11. An example of the first trained model 331 is a neural network. When a peripheral image is input to the first trained model 331 as an input variable, the first trained model 331 outputs an output variable indicating the probability that an elementary school student is included in the peripheral image. For example, the first trained model 331 outputs a larger value as the output variable as the probability that an elementary school student is included in the peripheral image increases. In this case, the processing circuit 31 of the control device 30 determines that an elementary school student is included in the peripheral image when the output variable is equal to or greater than a threshold value. On the other hand, the processing circuit 31 determines that an elementary school student is not included in the peripheral image when the output variable is less than the threshold value.

[0017] Here, when an elementary school student goes to school, the elementary school student carries a backpack, wears a hat and a uniform designated by the elementary school, etc. The first trained model 331 is previously subjected to machine learning that can determine that an elementary school student is included in a surrounding image when a child who satisfies at least one of the following two conditions (A1) and (A2) is included in the surrounding image.

[0018] (A1) Carrying a school bag. (A2) Students must wear clothing designated by the elementary school. The second trained model 332 is a model that has undergone machine learning to determine whether or not the belongings 51 are captured in the peripheral image generated by the imaging unit 11. An example of the second trained model 332 is a neural network. When the peripheral image is input to the second trained model 332 as an input variable, the second trained model 332 outputs an output variable indicating the probability that the belongings 51 are captured in the peripheral image. For example, the second trained model 332 outputs a larger value as the output variable as the probability that the belongings 51 are captured in the peripheral image increases. In this case, when the output variable is equal to or greater than a threshold value, the processing circuit 31 of the control device 30 determines that the belongings 51 are captured in the peripheral image. On the other hand, when the output variable is less than the threshold value, the processing circuit 31 determines that the belongings 51 are not captured in the peripheral image.

[0019] As described above, the belongings 51 are, for example, items that an elementary school student is required to wear or carry when going to school, or items that an elementary school student uses in class. Specifically, the belongings 51 are a school bag, a hat, a uniform, a handbag, a water bottle, a recorder, etc. The second trained model 332 is previously subjected to machine learning that can determine that the belongings 51 are included in a peripheral image when these items are included in the peripheral image.

[0020] The risk area information 333 is information in which an elementary school is associated with location information indicating a risk area within the school district of the elementary school. A risk area is an area where accidents are likely to occur. Examples of risk areas include areas near flood control facilities such as rivers and reservoirs, roads with heavy traffic, and facilities where large vehicles frequently enter and exit.

[0021] The processing circuit 31 determines whether or not personal belongings 51 are captured in the peripheral image based on the peripheral image generated by the imaging unit 11. Specifically, the processing circuit 31 determines whether or not personal belongings 51 are captured in the peripheral image based on the output variables of the second trained model 332 when the peripheral image generated by the imaging unit 11 is input to the second trained model 332.

[0022] The processing circuit 31 determines whether or not an elementary school student is captured in the peripheral image based on the peripheral image determined to include the belongings 51. Specifically, the processing circuit 31 determines whether or not an elementary school student is captured in the peripheral image based on the output variables of the first trained model 331 when the peripheral image generated by the imaging unit 11 is input to the first trained model 331.

[0023] When the processing circuit 31 determines that the surrounding image shows belongings 51 but does not show an elementary school student, it transmits notification information to the server device 20 of the elementary school via the communication network 40. In the following description, the surrounding image showing belongings 51 but not showing an elementary school student is also referred to as a lost item image. Accordingly, when the processing circuit 31 determines that the surrounding image shows belongings 51 but not a elementary school student, it is also referred to as determining that the surrounding image is a lost item image.

[0024] The notification information transmitted to the server device 20 includes, for example, a lost item image and location information acquired by the location information acquisition unit 12 when the lost item image was generated by the imaging unit 11. At this time, the processing circuit 31 identifies an elementary school having a school district including the location indicated by the location information based on the location information included in the notification information transmitted to the server device 20. Then, the processing circuit 31 transmits the notification information to the server device 20 of the identified elementary school.

[0025] Furthermore, when the processing circuit 31 determines that the peripheral image generated by the imaging unit 11 is a lost item image and the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area, the processing circuit 31 transmits notification information to the server device 20. The processing circuit 31 determines whether or not the belongings 51 shown in the lost item image are in a dangerous area based on the peripheral image generated by the imaging unit 11 and the position information acquired by the position information acquisition unit 12. When the processing circuit 31 determines that the peripheral image generated by the imaging unit 11 is a lost item image, the processing circuit 31 acquires the position information acquired by the position information acquisition unit 12 when the lost item image was generated by the imaging unit 11.

[0026] Next, the processing circuit 31 reads out from the storage unit 33 risk area information 333 designated by an elementary school having a school district including the current location of the vehicle 10. Then, the processing circuit 31 judges whether or not the location indicated by the location information acquired by the location information acquisition unit 12 is a risk area based on the risk area information 333 read out from the storage unit 33. When the processing circuit 31 judges that the location indicated by the location information is a risk area, it judges that the lost item image is a surrounding image generated by the imaging unit 11 capturing an image of the risk area.

[0027] When the processing circuit 31 determines that the lost item image is a surrounding image generated by the imaging unit 11 capturing an image of a dangerous area, it transmits notification information to the server device 20 of the elementary school via the communication network 40. The notification information includes, for example, the lost item image when the imaging unit 11 captures the belongings 51 in the dangerous area, and the location information acquired by the location information acquisition unit 12 when the lost item image was generated by the imaging unit 11.

[0028] Furthermore, when the processing circuit 31 determines that the peripheral image generated by the imaging unit 11 during the time period when children are going to and from elementary school is a lost item image, the processing circuit 31 transmits notification information to the server device 20. When the processing circuit 31 determines that the peripheral image is a lost item image, the processing circuit 31 determines whether the time when the lost item image was generated by the imaging unit 11 is during the time period when children are going to and from school. The time period when children are going to and from school includes the time period when children are going to and from elementary school and the time period when children are going home from elementary school. An example of the time period when children are going to school is between 7:00 and 9:00 in the morning. An example of the time period when children are going home is between 2:00 and 5:00 in the evening.

[0029] During the school commute time, it is rare that there are belongings 51 on the street without elementary school students nearby. Therefore, if the surrounding image captured by the imaging unit 11 during the school commute time is determined to be a lost item image, it is preferable to notify the elementary school or its staff. Therefore, if the surrounding image generated by the imaging unit 11 during the school commute time is a lost item image, the processing circuit 31 transmits notification information to the server device 20 of the elementary school via the communication network 40.

[0030] <Configuration of the server device 20> As shown in FIG. 2, the server device 20 includes, for example, a processing circuit 21, a communication unit 22, and a storage unit 23. The processing circuit 21 is configured with, for example, a hardware processor such as a CPU. The hardware processor executes a program (software), which allows the processing circuit 21 to execute various processes. The program may be stored in the storage unit 23 in advance. The storage unit 23 is realized by, for example, a non-transitory storage medium such as an HDD or a flash memory. The communication unit 22 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the control device 30 and other devices based on the control of the processing circuit 21.

[0031] Moreover, one or more information terminals 60 are connected to the server device 20. The communication unit 22 communicates with the information terminal 60 in a wired or wireless manner. At this time, the server device 20 and the information terminal 60 may communicate with each other via the above-mentioned communication network 40, or may communicate using another network. In the case of wireless communication, the communication mode between the communication unit 22 and the information terminal 60 may be short-distance wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). In addition, the communication mode between the communication unit 22 and the information terminal 60 may be communication via a network such as the Internet, a WAN, a LAN, a provider terminal, a wireless communication network, a wireless base station, or a dedicated line. The information terminal 60 is realized, for example, by a smartphone, a tablet, or the like.

[0032] When the processing circuit 21 receives notification information from the control device 30, it notifies the information terminal 60 of the received information. Specifically, the processing circuit 21 notifies the information terminal 60 that the notification information has been uploaded to the server device 20. When the staff member who knows that the notification from the server device 20 has arrived at the information terminal 60 operates the operation unit of the information terminal 60, the notification information is downloaded from the server device 20 to the information terminal 60. Then, in the information terminal 60, the downloaded information is displayed on the display unit of the information terminal 60.

[0033] <Series of processes executed in the control device 30> A series of processes executed in the control device 30 will be described with reference to Fig. 3. The processes shown in Fig. 3 are repeatedly executed at predetermined time intervals. First, the processing circuit 31 acquires a surrounding image generated by the imaging unit 11 (step S100). Next, the processing circuit 31 acquires position information from the position information acquisition unit 12 (step S102). The position information acquired in step S102 is the position information of the vehicle 10 when the surrounding image acquired in step S100 was generated by the imaging unit 11.

[0034] The processing circuit 31 judges whether the acquired surrounding image is a lost item image (step S104). That is, the processing circuit 31 judges whether the acquired surrounding image is a surrounding image in which the elementary school student's belongings 51 are shown but the elementary school student is not shown. When the processing circuit 31 judges that the surrounding image is not a lost item image (step S104; NO), there is no information to notify the server device 20, and therefore the series of processes is terminated.

[0035] When the processing circuit 31 determines that the peripheral image is a lost item image (step S104; YES), it determines whether or not the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area (step S120). Based on the position information acquired in step S102 and the dangerous area information 333, the processing circuit 31 determines whether or not the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area. When the processing circuit 31 determines that the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area (step S106; YES), it causes the communication unit 22 to transmit notification information to the server device 20 of the elementary school (step S108). The notification information transmitted to the server device 20 in step S108 includes the peripheral image acquired in step S100 during the current execution of the series of processes shown in FIG. 3 and the position information acquired in step S102 during the current execution of the series of processes. After that, the processing circuit 31 ends the series of processes.

[0036] When the processing circuit 31 determines that the lost item image is not a surrounding image generated by the imaging unit 11 capturing an image of a dangerous area (step S106; NO), it determines whether or not the time when the lost item image was generated by the imaging unit 11 was during the time period when children go to and from school (step S110). When the processing circuit 31 determines that the time when the lost item image was generated by the imaging unit 11 was during the time period when children go to and from school (step S110; YES), it advances the process to step S108. When the processing circuit 31 determines that the time when the lost item image was generated by the imaging unit 11 was not during the time period when children go to and from school (step S110; NO), it ends the series of processes.

[0037] In other words, even if it is determined that the surrounding image acquired in step S100 is a lost item image, if either of the following two conditions (C1) and (C2) is satisfied, the processing circuit 31 terminates the series of processes without sending notification information to the server device 20.

[0038] (C1) The image of the lost item is not a surrounding image generated by the imaging unit 11 capturing an image of a dangerous area. (C2) The time when the lost item image was generated is outside the time when students are going to or from school.

[0039] <Series of processes executed in the server device 20> A series of processes executed in the server device 20 will be described with reference to FIG. 4. The processes shown in FIG. 4 are repeatedly executed at predetermined time intervals. The processing circuit 21 judges whether or not notification information has been received from the control device 30 (step S200). If the processing circuit 21 has not received notification information from the control device 30 (step S200; NO), the processing circuit 21 ends the series of processes. If the processing circuit 21 has received notification information from the control device 30 (step S200; YES), the processing circuit 21 notifies the information terminal 60 of the received information (step S202). Then, the processing circuit 21 ends the series of processes.

[0040] <Actions and Effects of the Embodiments> According to the above embodiment, the following actions and effects can be obtained. (1) When the vehicle 10 is traveling, the imaging unit 11 generates a surrounding image by capturing an image of the surroundings of the vehicle 10. At this time, the position information acquisition unit 12 acquires position information of the vehicle 10 at the time when the imaging unit 11 captured the surrounding image.

[0041] The processing circuit 31 of the control device 30 determines whether the surrounding image generated by the imaging unit 11 is a lost item image in which belongings 51 are captured but no elementary school student is captured. The processing circuit 31 transmits notification information to the server device 20 on the condition that the surrounding image is a lost item image.

[0042] When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60. That is, the server device 20 transmits to the information terminal 60 the notification information including a lost item image in which the belongings 51 are captured but the elementary school student is not captured, and location information at the time when the lost item image was generated by the imaging unit 11.

[0043] When a vehicle 10 is traveling near an elementary school, the imaging unit 11 of the vehicle 10 may capture only the belongings 51 of the elementary school student without the elementary school student being present. In such a case, the elementary school student may be involved in an incident in which the elementary school student is kidnapped, leaving the belongings 51 behind. The elementary school student may also be involved in an accident in which the elementary school student falls from the road where the belongings 51 are located into a river below or under an overpass. Therefore, according to the information processing method, even if the elementary school student does not have the information terminal 60, it is possible to notify the elementary school or the staff of the elementary school that the elementary school student is in a state requiring notification.

[0044] (2) If an elementary school student is in a dangerous area within the school district of the elementary school, the elementary school student may be involved in an accident. In this regard, in the information processing method, when the imaging unit 11 generates a lost item image in which the belongings 51 are captured in a dangerous area while the vehicle 10 is traveling, notification information including the lost item image is transmitted from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60. In other words, the server device 20 transmits to the information terminal 60 notification information including the lost item image when the imaging unit 11 captures the belongings 51 in the dangerous area.

[0045] As a result, the information processing method can make elementary school staff aware that an elementary school student may have been involved in an accident in a dangerous area. (3) When the vehicle 10 is traveling during the school commute time, the imaging unit 11 of the vehicle 10 may capture an elementary school student or belongings 51. As described above, during the school commute time, it is rare that there are belongings 51 on the road but no elementary school student in the vicinity. In such a case, the elementary school student may be involved in an incident. According to the information processing method, when the surrounding image generated by the imaging unit 11 of the vehicle 10 during the school commute time shows the belongings 51 of the elementary school student but is a lost item image that does not show the elementary school student, notification information including the lost item image is transmitted from the control device 30 of the vehicle 10 to the server device 20. When the server device 20 receives the notification information from the vehicle 10, the server device 20 transmits the notification information to the information terminal 60.

[0046] This allows elementary school staff to be aware that elementary school students may be involved in incidents while they are arriving or leaving school. <Example of change> The above embodiment may be modified as follows: The above embodiment and each of the following modifications may be combined with each other within a range where no technical contradiction occurs.

[0047] Some of the processes shown in Fig. 3 may not be executed. For example, if the process of determining that the surrounding image is a lost item image is not performed during the school commute, the process of step S110 may be omitted. In this case, if the processing circuit 31 determines in the process of step S106 that the belongings 51 shown in the lost item image are not in a dangerous area (step S106; NO), the series of processes shown in Fig. 3 is terminated.

[0048] Also, for example, if the process of determining whether or not the belongings 51 shown in the lost item image are in a dangerous area is not performed, the process of step S106 may be omitted. In this case, if the process of step S104 determines that the surrounding image is a lost item image (step S104; YES), the process proceeds to step S110.

[0049] 3, the process of determining whether the belongings 51 shown in the lost item image are in a dangerous area is performed with priority over the process of determining whether the time when the lost item image was generated by the imaging unit 11 is during the school commute, but this is not limited to this. Either of these processes may be performed with priority. In other words, the order of steps S106 and S110 may be switched.

[0050] When the processing circuit 31 determines that the surrounding image acquired in step S100 is a lost item image, the processing circuit 31 may be configured to transmit notification information to the server device 20 even in the above-mentioned case, even if at least one of the above-mentioned two conditions (C1) and (C2) is not satisfied.

[0051] The processing circuit 31 may determine whether or not an elementary school student is shown in the surrounding image based on the results obtained by general image analysis, instead of the process of determining whether or not an elementary school student is shown in the surrounding image using the first trained model 331. Also, the processing circuit 31 may determine whether or not personal belongings 51 are shown in the surrounding image based on the results obtained by general image analysis, instead of the process of determining whether or not personal belongings 51 are shown in the surrounding image using the second trained model 332.

[0052] The imaging unit 11 may be a stereo camera. The imaging unit 11 may be provided with multiple cameras (for example, a first camera and a second camera), may capture images in the same direction using multiple cameras, or may capture images using the first camera under normal circumstances and the second camera or both the first and second cameras when a predetermined condition is satisfied. The first camera may capture images close to the imaging unit 11, and the second camera may capture images far from the imaging unit 11.

[0053] The imaging unit 11 may be attached to any location of the vehicle 10 as long as it can capture an image of the surroundings of the vehicle 10 and generate a surrounding image. For example, the imaging unit 11 may be attached to the top of the rear windshield or the back door when capturing an image of the rear of the vehicle 10. The imaging unit 11 may be attached to a door mirror when capturing an image of the side and rear side of the vehicle 10.

[0054] The location information acquired by the location information acquisition unit 12 may be identified or supplemented by an INS (Inertial Navigation System) that uses the outputs of various sensors equipped in the vehicle 10. The risk area information 333 may be updated successively according to the traveling position of the vehicle 10. For example, the control device 30 transmits the location information acquired by the location information acquisition unit 12 to an information providing server included in the information processing system 1. The information providing server identifies the school district of the received location information, and transmits risk area information 333 related to the identified elementary school to the control device 30. The control device 30 performs various determination processes based on the received risk area information 333, and transmits notification information to the server device 20 of the corresponding elementary school.

[0055] The manner in which the information terminal 60 notifies the notification information is not limited to displaying it on the display unit. The information terminal 60 may be realized by, for example, a control device that controls an in-school broadcast at an elementary school. In this case, the information terminal 60 may notify the staff of the elementary school by broadcasting a sound indicating that an elementary school student is at the location indicated by the location information included in the notification information.

[0056] The server device 20 may be installed at a location other than an elementary school as long as it is installed outside the vehicle 10. For example, the server device 20 may be installed at the business premises of a business entity that provides the information processing system 1, or may be installed at a facility that manages elementary schools, such as a city hall. Also, the server device 20 does not have to be installed at each elementary school. In this case, the server device 20 identifies the corresponding elementary school based on the location information included in the notification information received from the control device 30. The server device 20 notifies the notification information to an information terminal 60 installed at the identified elementary school and an information terminal 60 of a staff member belonging to the elementary school.

[0057] In the above description, the processing circuit 31 performs various determinations based on the surrounding image generated by the imaging unit 11, but this is not limited thereto. Some of the various determinations performed by the processing circuit 31 may be performed by the processing circuit 21 of the server device 20. In this case, the processing circuit 31 transmits to the server device 20 a surrounding image generated by the imaging unit 11, which includes the belongings 51, in association with the position information acquired by the position information acquisition unit 12 when the surrounding image was generated by the imaging unit 11. The processing circuit 21 performs various determinations based on the surrounding image and the position information received from the control device 30. For example, the processing circuit 21 determines whether the surrounding image is a lost item image in which the belongings 51 of an elementary school student are captured but the elementary school student is not captured, based on the received position information. Then, when the processing circuit 21 determines that the surrounding image is a lost item image, it transmits notification information including the lost item image and the position information associated with the lost item image to the information terminal 60. In this case, unlike the above embodiment, the server device 20 does not need to provide the school route information 332 to the vehicle 10.

[0058] For example, the processing circuit 21 determines whether or not the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area, based on the peripheral image and the position information received from the control device 30. When the processing circuit 21 determines that the lost item image is a peripheral image generated by the imaging unit 11 capturing an image of a dangerous area, the processing circuit 21 transmits notification information including the lost item image and the position information associated with the lost item image to the information terminal 60. In this case, unlike the above embodiment, the server device 20 does not need to provide the dangerous area information 333 to the vehicle 10.

[0059] Furthermore, for example, when the control device 30 of the vehicle 10 can determine that a human being, not limited to an elementary school student, is captured in the peripheral image, the peripheral image and the position information of the vehicle 10 at the time the peripheral image was generated may be associated with each other and transmitted to the server device 20. In this case, the processing circuit 21 of the server device 20 determines whether or not an elementary school student is captured in the peripheral image by inputting the received peripheral image to the first trained model 331, for example.

[0060] Also, for example, when it can be determined that any object, not limited to the elementary school student's belongings 51, is left on the road, the control device 30 of the vehicle 10 may transmit the peripheral image and the position information of the vehicle 10 at the time when the peripheral image was generated to the server device 20 in association with each other. In this case, the processing circuit 21 of the server device 20 may input the received peripheral image to the second trained model 332, or the like, to determine whether or not the belongings 51 are included in the peripheral image. When the processing circuit 21 determines that the peripheral image is an image in which the belongings 51 are included but the elementary school student is not included, the processing circuit 21 may sequentially execute the processes from step S106 shown in FIG. 3 onward.

[0061] Some or all of the components of the processing circuit 21 and the processing circuit 31 may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. [Explanation of symbols]

[0062] 1...information processing system, 10...vehicle, 11...imaging unit, 12...location information acquisition unit, 20...server device, 21, 31...processing circuit, 22, 32...communication unit, 23, 33...memory unit, 30...control device, 40...communication network, 51...personal belongings, 60...information terminal, 331...first trained model, 332...second trained model, 333...dangerous area information.

Claims

1. An information processing method executed in an information processing system including a vehicle including an imaging unit that generates a surrounding image by imaging a surrounding situation, a position information acquisition unit that acquires its own position information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, wherein the surrounding image and the position information are transmitted from the communication unit of the vehicle to the server device via a communication network, The communication unit of the vehicle transmits to the server device notification information including the surrounding image showing belongings of the elementary school student and position information of the vehicle when the surrounding image was generated by the imaging unit; The processing circuit of the server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including a lost item image, which is the surrounding image that shows the elementary school student's belongings but does not show the elementary school student, and location information of the vehicle when the lost item image was generated by the imaging unit; An information processing method comprising:

2. In the case where the notification information is notified to at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school, a processing circuit of the server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including a lost item image, which is the surrounding image generated by the imaging unit by imaging a dangerous area within the school district of the elementary school and in which the elementary school student's belongings are captured but the elementary school student is not captured, and location information of the vehicle at the time the lost item image was generated by the imaging unit; The information processing method according to claim 1 .

3. In the case where the notification information is notified to at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school, a processing circuit of the server device notifies at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school of notification information including a lost item image, which is the peripheral image generated by the imaging unit during the time period when the elementary school is going to and from school and in which the elementary school student's belongings are captured but the elementary school student is not captured, and location information of the vehicle at the time the lost item image was generated by the imaging unit; The information processing method according to claim 1 .

4. A program executed on a vehicle control device including an imaging unit that generates a surrounding image by imaging a surrounding situation, a position information acquisition unit that acquires its own position information, and a communication unit that communicates with a server device installed outside the vehicle, the server device is configured to be able to communicate with at least one of an information terminal owned by an elementary school and an information terminal owned by a staff member of the elementary school; The control device includes: acquiring notification information including a lost item image, which is a peripheral image that shows belongings of an elementary school student but does not show the elementary school student around the belongings, and location information of the vehicle at the time the lost item image was generated by the imaging unit, from among the peripheral images; and transmitting the notification information from the communication unit to the server device. program.

5. An information processing system including a vehicle including an imaging unit that generates a surrounding image by imaging a surrounding situation, a position information acquisition unit that acquires its own position information, and a communication unit that communicates with the outside, and a server device installed outside the vehicle, the communication unit of the vehicle transmits, to the server device via a communication network, notification information including the surrounding image in which the elementary school student's belongings are captured and location information of the vehicle when the surrounding image was generated by the imaging unit; The server device transmits notification information including a lost item image, which is the surrounding image in which the elementary school student's belongings are captured but the elementary school student is not captured, and location information of the vehicle when the lost item image was generated by the imaging unit, to at least one of an information terminal owned by the elementary school and an information terminal owned by a staff member of the elementary school. Information processing system.

Images