Information processing device

Abstract

To provide an information processing device that collects a peripheral image in which an event is appropriately captured.SOLUTION: A server device 20 requests a vehicle 10 to provide a peripheral image that is generated by imaging a periphery of the vehicle 10 by an imaging unit 11 mounted on the vehicle 10; and receives the peripheral image from the vehicle 10 via a communication network 40. When at least either a weather-related event or a natural disaster event occurs, the server device 20 requests the vehicle 10 located at a place where the event has occurred to provide a peripheral image satisfying a predetermined condition; and changes the predetermined condition according to contents of the event.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device. [Background technology]

[0002] A vehicle is provided with an imaging unit that captures an image of the surroundings of the vehicle and generates a surrounding image. The surrounding image shows the surrounding environment in which the vehicle is located at the time of capturing the image. Patent Document 1 discloses a technology that collects surrounding images generated by the imaging unit of the vehicle in a user's desired position and surrounding environment, and provides the collected surrounding images to the user. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-83034 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when a weather-related event that occurs rarely, such as a tornado or a meteorite fall, or a natural disaster event that occurs rarely, such as an earthquake or a flood, occurs, a user may desire a surrounding image showing the event. However, depending on the vehicle's position, the specifications, settings, and imaging direction of the imaging unit, the surrounding image may not adequately capture the event. The technology disclosed in Patent Document 1 collects surrounding images based on the vehicle's position, but it is difficult to collect surrounding images that adequately capture the event. [Means for solving the problem]

[0005] An information processing device that achieves the above-mentioned objective is an information processing device that receives a surrounding image from a vehicle via a communication network by requesting the vehicle to provide a surrounding image, which is an image generated by an imaging unit mounted on the vehicle capturing an image of the surroundings of the vehicle, and when at least one of a weather-related event and a natural disaster event occurs, it requests the vehicle located at the location where the event occurred to provide the surrounding image that satisfies specified conditions, and changes the specified conditions depending on the content of the event. Effect of the Invention

[0006] According to the present invention, a suitable image of the surroundings of the event is collected. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram used to explain an information processing system. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the server device and the control device. [Diagram 3] FIG. 3 is a diagram showing an example of the contents of the first predetermined condition information. [Figure 4] FIG. 4 is a diagram showing an example of the contents of second predetermined condition information. [Diagram 5] FIG. 5 is a flowchart showing an example of processing by the control device. [Figure 6] FIG. 6 is a flowchart illustrating an example of processing of the server device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] <Embodiment> Hereinafter, an embodiment of an information processing device will be described with reference to FIGS. <Overall configuration of information processing system 1> As shown in Fig. 1, the information processing system 1 includes, for example, one or more vehicles 10 and one or more server devices 20. The server device 20 is an example of an information processing device. The vehicle 10 includes, for example, an imaging unit 11, a position information acquisition unit 12, and a control device 30. The control device 30 and the server device 20 are connected to each other via a communication network 40 so as to be able to transmit and receive information. The communication network 40 includes, for example, the Internet, a Wide Area Network (WAN), a Local Area Network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like.

[0009] The imaging unit 11 is, for example, a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). One example of the imaging unit 11 is a digital camera constituting a drive recorder. The imaging unit 11 captures images of the surrounding conditions of the vehicle 10 in which it is mounted. Depending on the installation position of the imaging unit 11 on the vehicle 10, the imaging unit 11 can capture images of the front, rear, and sides of the vehicle 10. The imaging unit 11 generates a surrounding image by capturing images of the surrounding conditions of the vehicle 10. The imaging unit 11 also generates a surrounding video composed of multiple surrounding images that are successive in time series.

[0010] The position information acquisition unit 12 is, for example, a Global Navigation Satellite System (GNSS) receiver. The position information acquisition unit 12 identifies the position of the vehicle 10 based on a signal received from a GNSS satellite, and acquires position information indicating the position of the vehicle 10.

[0011] The control device 30 transmits position information to the server device 20 at predetermined time intervals, so that the server device 20 can identify the position of the vehicle 10. Furthermore, when there is a request from the server device 20 and when the predetermined conditions presented by the server device 20 are met, the control device 30 transmits a peripheral image or a peripheral video to the server device 20. Specifically, when the control device 30 can only provide a peripheral image due to the specifications of the imaging unit 11 or the drive recorder, the control device 30 transmits a peripheral image that matches the predetermined conditions to the server device 20. Furthermore, when the control device 30 can only provide a peripheral video due to the specifications of the imaging unit 11 or the drive recorder, the control device 30 transmits a peripheral video that matches the predetermined conditions to the server device 20. On the other hand, when the control device 30 can provide both a peripheral image and a peripheral video, the control device 30 provides only the peripheral video.

[0012] The server device 20 is, for example, a device that aggregates information provided from the vehicle 10. The server device 20 collects, from the vehicle 10, surrounding images that appropriately capture, for example, weather-related events such as tornadoes and meteorite falls, and natural disaster events such as earthquakes and floods. In the following description, when there is no need to distinguish between weather-related events and natural disaster events, they will simply be referred to as events.

[0013] <Configuration of the server device 20> As shown in FIG. 2, the server device 20 includes, for example, a processing circuit 21, a communication unit 22, and a storage unit 23. The processing circuit 21 is configured with, for example, a hardware processor such as a CPU (Central Processing Unit). The hardware processor executes a program, which allows the processing circuit 21 to execute various processes. The program may be stored in the storage unit 23 in advance. The storage unit 23 is realized by, for example, a non-transitory storage medium such as an HDD or a flash memory. The communication unit 22 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the control device 30 and other devices based on the control of the processing circuit 21.

[0014] The storage unit 23 stores predetermined condition information 231 in advance. The predetermined condition information 231 is, for example, information in which an event and a predetermined condition suitable for capturing an image of the event are associated with each other. The predetermined condition information 231 includes first predetermined condition information 231-1 and second predetermined condition information 231-2.

[0015] As shown in FIG. 3, the first predetermined condition information 231-1 includes information indicating predetermined conditions of an imaging position, an imaging direction, and video quality for weather-related events and natural disaster events, for example. The predetermined condition of the imaging position is whether the position of the vehicle 10 is a general road or an expressway. The predetermined condition of the imaging direction is whether the angle of view of the imaging unit 11 is lower than the horizontal direction or higher than the horizontal direction. The predetermined condition of the video quality includes a condition of the resolution of the surrounding image generated by the imaging unit 11 and a condition of the frame rate of the surrounding video. The predetermined condition of the resolution is whether the surrounding image has a lower resolution than a predetermined resolution or a higher resolution than the predetermined resolution. The predetermined condition of the frame rate is whether the surrounding video has a lower frame rate than a predetermined frame rate or a higher frame rate than the predetermined frame rate. The predetermined resolution is, for example, full high definition. The predetermined frame rate is, for example, 60 [fps]. Note that the predetermined resolution and the predetermined frame rate are only examples and are not limited thereto.

[0016] Specifically, the first predetermined condition information 231-1 indicates the following predetermined conditions (A1) to (A5) for weather-related events. (A1) It is better for the imaging position to be on a highway rather than on an ordinary road.

[0017] (A2) The imaging direction should only have an upward angle of view. (A3) As for the provision mode, between the peripheral images and the peripheral video, it is preferable to provide only the peripheral video. (A4) Higher resolution is better than lower resolution.

[0018] (A5) Only high frame rates are acceptable. Moreover, the first predetermined condition information 231-1 indicates the following predetermined conditions (B1) to (B5) regarding natural disaster events.

[0019] (B1) It is better for the imaging position to be on an ordinary road rather than on an expressway. (B2) The imaging direction should only have a downward angle of view. In the (B3) provision mode, between a peripheral image and a peripheral video, the peripheral image has a higher priority.

[0020] (B4) Higher resolution is better than lower resolution. (B5) Only low frame rates are acceptable. 4, the second predetermined condition information 231-2 indicates, for example, conditions for natural disaster events, particularly earthquakes. Specifically, in the second predetermined condition information 231-2, a predetermined condition (C1) is associated with an earthquake with a high seismic intensity equal to or greater than a predetermined seismic intensity. In addition, in the second predetermined condition information 231-2, a predetermined condition (C2) is associated with an earthquake with a low seismic intensity less than the predetermined seismic intensity.

[0021] (C1) The video must include footage of the earthquake either during the period in which the earthquake occurred or after the shaking has subsided. (C2) The video contains footage of the earthquake occurring during the period.

[0022] The predetermined conditions included in the predetermined condition information 231 are merely examples and are not limited thereto. Some of the predetermined conditions included in the predetermined condition information 231 may not be included, and other predetermined conditions may be included.

[0023] Returning to FIG. 2, the server device 20 receives input of information indicating the occurrence of an event and the location of the event from, for example, a user or administrator of the server device 20. The processing circuit 21 refers to the predetermined condition information 231 and identifies an event that matches the event indicated by the input information. The processing circuit 21 identifies information indicating the predetermined condition associated with the identified event. That is, the server device 20 changes the predetermined condition according to the content of the event by selecting a predetermined condition from the predetermined condition information 231. The processing circuit 21 identifies the vehicle 10 that exists within a range of a predetermined distance from the location of the event based on the information indicating the location of the event and the position information received from the vehicle 10. The processing circuit 21 requests the vehicle 10 to transmit a surrounding image or a surrounding video by having the identified vehicle 10 transmit information indicating the identified predetermined condition via the communication unit 22. The server device 20 collects the surrounding image or the surrounding video transmitted by the vehicle 10 in response to the request.

[0024] <Configuration of the control device 30> As shown in FIG. 2, the control device 30 includes, for example, a processing circuit 31, a communication unit 32, and a storage unit 33. The processing circuit 31 is configured with, for example, a hardware processor such as a CPU. The hardware processor executes a program, which allows the processing circuit 31 to execute various processes. The program may be stored in the storage unit 33 in advance. The storage unit 33 is realized by, for example, a non-transitory storage medium such as an HDD (Hard Disk Drive) or a flash memory. The communication unit 32 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the server device 20 and other devices based on the control of the processing circuit 31.

[0025] When information indicating a predetermined condition is received from the server device 20 by the communication unit 32, the processing circuit 31 determines whether or not the peripheral image or the peripheral video of the determination target satisfies the predetermined condition. The peripheral image of the determination target is, for example, a peripheral image generated by the imaging unit 11 at the timing when the information indicating the predetermined condition is received. Also, the peripheral video of the determination target is, for example, a video that includes a period from the timing when the information indicating the predetermined condition is received to a timing a predetermined time before.

[0026] The processing circuit 31 specifies the imaging direction, resolution, and frame rate based on the specifications of the imaging unit 11 or the parameter settings of the imaging unit 11. The processing circuit 31 also specifies the angle of view based on the positions of the vanishing points of the peripheral image of the determination target or each peripheral video constituting the peripheral video of the determination target. The processing circuit 31 also specifies whether the road shown in the peripheral image is a general road or an expressway by performing general image processing on the peripheral image of the determination target or each peripheral image constituting the peripheral video of the determination target.

[0027] The processing circuit 31 determines whether the peripheral video of the judgment target or the peripheral video of the judgment target satisfies a predetermined condition based on each piece of identified information. If the processing circuit 31 determines that the predetermined condition is not satisfied, it does not provide the peripheral video or the peripheral video to the server device 20. If the processing circuit 31 determines that the predetermined condition is satisfied, it provides the peripheral video or the peripheral video to the server device 20.

[0028] <Series of processes executed in the control device 30> The process executed in the control device 30 will be described with reference to Fig. 5. In addition to the process shown in Fig. 5, the processing circuit 31 executes a process of transmitting position information indicating the position of the vehicle 10 to the server device 20 via the communication unit 32 at predetermined time intervals.

[0029] As shown in FIG. 5, first, the processing circuit 31 judges whether or not information indicating a predetermined condition is received from the server device 20 by the communication unit 32 (step S100). The processing circuit 31 waits until information indicating the predetermined condition is received from the server device 20 (step S100; NO). When the processing circuit 31 receives information indicating the predetermined condition, it judges whether or not the peripheral image or the peripheral video of the judgment target satisfies the predetermined condition (step S102). When the received predetermined condition is the predetermined conditions (A1) to (A5) related to a weather-related event and the predetermined conditions (C1) to (C2) related to an earthquake, the processing circuit 31 judges whether or not the peripheral video of the judgment target satisfies the predetermined condition. When the received predetermined condition is the predetermined conditions (B1) to (B5) related to a natural disaster event other than an earthquake, the processing circuit 31 judges whether or not the peripheral video of the judgment target satisfies the predetermined condition.

[0030] When the processing circuit 31 determines that the predetermined condition is not satisfied (step S102; NO), it terminates the series of processes assuming that there is no peripheral video or peripheral video to be provided to the server device 20. When the processing circuit 31 determines that the predetermined condition is satisfied (step S102; YES), it transmits the peripheral video to be judged or the peripheral video to be judged to the server device 20 via the communication unit 22 (step S104). After that, the processing circuit 31 terminates the series of processes.

[0031] <Series of processes executed in the server device 20> The process executed in the server device 20 will be described with reference to Fig. 6. In addition to the process shown in Fig. 6, the processing circuit 21 executes a process of receiving position information indicating the position of the vehicle 10 from the control device 30 via the communication unit 22 at predetermined time intervals.

[0032] As shown in FIG. 5, first, the processing circuit 21 judges whether or not an event has occurred (step S200). The processing circuit 21 waits until an event occurs (step S200; NO). The processing circuit 21 judges that an event has occurred when information indicating the occurrence of an event and the location of the event is input. When the processing circuit 21 judges that an event has occurred (step S200; YES), the processing circuit 21 refers to the predetermined condition information 231 and specifies a predetermined condition corresponding to the event (step S202). The processing circuit 21 causes the communication unit 22 to transmit information indicating the specified predetermined condition to the vehicle 10 that exists within a range of a predetermined distance from the location of the event (step S204). The processing circuit 21 receives, via the communication unit 22, a surrounding image or a surrounding video transmitted by the vehicle 10 that matches the predetermined condition (step S206). After that, the processing circuit 21 ends the series of processes.

[0033] <Actions and Effects of the Embodiments> According to the above embodiment, the following actions and effects can be obtained. (1) The imaging unit 11 generates a surrounding image by capturing an image of the surroundings of the vehicle 10. In addition, the position information acquisition unit 12 acquires position information of the vehicle 10 and transmits the acquired position information to the server device 20.

[0034] The processing circuit 21 identifies information indicating a predetermined condition associated with the event that has occurred. The processing circuit 21 causes the communication unit 22 to transmit information indicating the predetermined condition of the identified surrounding image to the vehicle 10, thereby requesting a surrounding image from the vehicle 10. The processing circuit 21 also causes the communication unit 22 to transmit information indicating the predetermined condition of the identified surrounding video to the vehicle 10, thereby requesting a surrounding video from the vehicle 10. Then, the control device 30 of the vehicle 10 searches for a surrounding image generated at the timing when the event occurred. Then, it is determined whether or not this surrounding image satisfies the predetermined condition. If it is determined that the surrounding image satisfies the predetermined condition, the surrounding image is transmitted to the server device 20. On the other hand, if it is determined that the surrounding image does not satisfy the predetermined condition, the surrounding image is not transmitted to the server device 20.

[0035] The server device 20 collects the surrounding images and surrounding videos transmitted by the vehicle 10 in response to a request. With this configuration, the server device 20 can collect surrounding images and surrounding videos that appropriately capture the event. Specifically, the server device 20 does not need to collect images that do not satisfy a predetermined condition.

[0036] (2) The predetermined conditions include at least one of the position of the vehicle 10 and the imaging direction of the peripheral image. Here, in order to appropriately capture weather-related events such as tornadoes and meteorite falls in the surrounding video, it is preferable that the vehicle 10 is located on an expressway with better visibility than general roads. According to this configuration, the first predetermined condition information 231-1 for the weather-related event includes the above-mentioned predetermined condition (A1). Therefore, the server device 20 can collect surrounding videos with good visibility that are generated when the vehicle 10 is on the expressway.

[0037] In addition, in order to appropriately capture a weather-related event in the surrounding image, it is preferable that the angle of view of the surrounding image is in the upward direction. With this configuration, the first predetermined condition information 231-1 of the weather-related event includes the above-mentioned predetermined condition (A2). Therefore, the server device 20 can collect surrounding images and surrounding videos with an angle of view in the upward direction that appropriately captures the sky.

[0038] Furthermore, in order to properly capture natural disaster events such as earthquakes and floods in surrounding images and surrounding videos, it is preferable to be able to confirm cracks in the ground and water flows, so it is preferable for the vehicle 10 to be located on a general road closer to the ground surface than on a highway. According to this configuration, the first predetermined condition information 231-1 for natural disaster events includes the above-mentioned predetermined condition (B1). Therefore, the server device 20 can collect surrounding images and surrounding videos that clearly show the state of the ground surface when the vehicle 10 is on a general road.

[0039] In addition, in order to appropriately transfer the natural disaster event to the surrounding image, the angle of view of the surrounding image is preferably downward. According to this configuration, the first predetermined condition information 231-1 of the natural disaster event includes the above-mentioned predetermined condition (B2). Therefore, the server device 20 can collect surrounding images and surrounding videos with an angle of view downward that appropriately capture the ground surface.

[0040] (3) The predetermined condition includes a frame rate for the surrounding images. Here, in order to properly capture a weather-related event, such as a tornado or a meteorite falling, that occurs within a short period of time in the surrounding video, it is preferable for the imaging unit 11 to capture the event at a high frame rate. With this configuration, the first predetermined condition information 231-1 for the weather-related event includes the above-mentioned predetermined condition (A5). Therefore, the server device 20 can collect surrounding images or surrounding videos captured at a frame rate according to the event.

[0041] (4) As shown by the second predetermined condition information 231-2, the predetermined condition when the seismic intensity of the earthquake is equal to or greater than the predetermined seismic intensity includes that the surrounding video includes both the surrounding images generated during the period when the earthquake is occurring and the surrounding images generated during the period after the shaking caused by the earthquake has subsided. Also, the predetermined condition when the seismic intensity of the earthquake is less than the predetermined seismic intensity includes that, of the surrounding images generated during the period when the earthquake is occurring and the surrounding images generated during the period after the shaking caused by the earthquake has subsided, only the surrounding images generated during the period when the earthquake is occurring are included in the video.

[0042] Here, if the seismic intensity of the earthquake is equal to or greater than a predetermined seismic intensity, it is preferable that the video shows the state of the earthquake during the earthquake and the state of the surroundings of the vehicle 10 after the shaking caused by the earthquake has subsided. This is because if the seismic intensity is high, there is a high possibility that damage will occur due to the earthquake. On the other hand, if the seismic intensity of the earthquake is less than the predetermined seismic intensity, it is preferable that the video shows how much the vehicle was shaking during the earthquake. This is because if the seismic intensity is low, there is a low possibility that damage will occur due to the earthquake. With this configuration, the predetermined condition is changed depending on whether the seismic intensity is equal to or greater than a predetermined seismic intensity. Therefore, the server device 20 can collect surrounding videos that appropriately show the earthquake according to the seismic intensity of the earthquake.

[0043] <Modification> The above embodiment may be modified as follows: The above embodiment and each of the following modifications may be combined with each other within a range that does not cause technical contradiction.

[0044] The predetermined condition information 231 may include a condition related to the length of the surrounding video. For example, the predetermined condition information 231 may include a condition that the surrounding video is equal to or longer than a predetermined length. With this configuration, the server device 20 can obtain a surrounding video with a length that allows the event to be sufficiently confirmed.

[0045] In the above description, a uniform predetermined condition is set for the frame rate of the weather-related event in the first predetermined condition information 231-1, but this is not limited thereto. Here, between a tornado and a meteorite fall, the meteorite fall occurs in a shorter time. Accordingly, a predetermined condition indicating a lower frame rate than a meteorite fall may be associated with the weather-related event of a tornado. On the other hand, a predetermined condition indicating a higher frame rate than a meteorite fall may be associated with the weather-related event of a tornado.

[0046] In the above, the control device 30 provides only the peripheral video when both the peripheral image and the peripheral video are available, but this is not limited to the above. When both the peripheral image and the peripheral video are available, the control device 30 may transmit both the peripheral image and the peripheral video to the server device 20 if a predetermined condition is met. Alternatively, the control device 30 may transmit an appropriate one of the peripheral image and the peripheral video to the server device 20 according to the event based on the predetermined condition.

[0047] In the above description, the peripheral image to be judged is, for example, a peripheral image generated by the imaging unit 11 at the timing when the information indicating the predetermined condition is received. However, this is not limited to the above. The server device 20 may transmit, for example, information indicating the date and time when an event occurred to the control device 30 together with the information indicating the predetermined condition. In this case, the processing circuit 31 performs subsequent processing with the peripheral image captured and generated by the imaging unit 11 at the date and time when the event indicated in the received information occurred as the peripheral image to be judged. In addition, the processing circuit 31 performs subsequent processing with the peripheral video including the peripheral image captured and generated by the imaging unit 11 at the date and time when the event indicated in the received information occurred as the peripheral video to be judged.

[0048] Some or all of the components of the processing circuit 21 and the processing circuit 31 may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. [Explanation of symbols]

[0049] 1...information processing system, 10...vehicle, 11...imaging unit, 12...location information acquisition unit, 20...server device, 21, 31...processing circuit, 22, 32...communication unit, 23, 33...memory unit, 30...control device, 40...communication network, 231...predetermined condition information, 231-1...first predetermined condition information, 231-2...second predetermined condition information.

Claims

1. An information processing device that receives a surrounding image from a vehicle via a communication network by requesting the vehicle to provide a surrounding image, the surrounding image being an image generated by an imaging unit mounted on the vehicle, the information processing device comprising: When at least one of a weather-related event and a natural disaster event occurs, a request is made to the vehicle located at a location where the event occurs to provide the surrounding image that satisfies a predetermined condition; and changing the predetermined condition depending on the content of the event. Information processing device.

2. When the event occurs, a request is made to the vehicle located at a location where the event occurs to provide a surrounding video that satisfies the predetermined condition and is composed of a plurality of surrounding images that are consecutive in time series; The predetermined condition includes at least one of a length of the surrounding video, a position of the vehicle, and an imaging direction of the surrounding image. The information processing device according to claim 1 .

3. When the event occurs, a request is made to the vehicle located at a location where the event occurs to provide a surrounding video that satisfies the predetermined condition and is composed of a plurality of surrounding images that are consecutive in time series; The predetermined condition includes a frame rate related to the peripheral video. The information processing device according to claim 1 .

4. The weather-related events include tornadoes and meteorites; When the tornado occurs, the frame rate is lower than when the meteorite falls. The information processing device according to claim 3 .

5. When the event occurs, a request is made to the vehicle located at a location where the event occurs to provide a surrounding video that satisfies the predetermined condition and is composed of a plurality of surrounding images that are consecutive in time series; the natural disaster event includes an earthquake occurring; When the earthquake occurs, The predetermined condition when the seismic intensity of the earthquake is equal to or greater than a predetermined seismic intensity includes that the surrounding video includes both the surrounding images generated during the period when the earthquake occurred and the surrounding images generated during the period after the shaking caused by the earthquake subsided, When the seismic intensity of the earthquake is less than the predetermined seismic intensity, the predetermined condition includes that, of the peripheral images generated during the period during which the earthquake occurs and the peripheral images generated during a period after the shaking caused by the earthquake has subsided, only the peripheral images generated during the period during which the earthquake occurs are included in the peripheral video. The information processing device according to claim 1 .

Images