Information collection device, information collection server, and information collection system

Abstract

To provide an information collection device, information collection server, and information collection system for collecting information related to an event that is to occur at an unspecified place of a route such as a road on which a mobile body moves, so as to provide the information to the mobile body.SOLUTION: An information collection device comprises: a passenger state information collection section for collecting a state of a passenger of a mobile body as passenger state information; a state change detection section for detecting a state change in the passenger based on the passenger state information so as to generate state change detection information; an outside image capturing section for capturing the outside of the mobile body so as to generate a captured image; and an outside state information transmission section for generating outside state information including the captured image based on the state change detection information so as to transmit the outside state information via a communication channel.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information collecting device, an information collecting server, and an information collecting system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, traffic information providing systems have been considered that collect information from cameras installed at intersections and transmit the conditions around the intersections to vehicles in the vicinity of the intersections (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-258068 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the traffic information system disclosed in Patent Document 1, it is only possible to obtain information on specific locations on the road, such as intersections where cameras are installed. Therefore, for example, even if some kind of event occurs in a location other than the specific location, it is not possible to collect the information and provide information based on the collected information.

[0005] The present invention has been made in consideration of the above-mentioned points, and one of its objectives is to provide an information collection device, an information collection server, and an information collection system that are capable of collecting information about events occurring at unspecified locations on a route, such as a road, along which a mobile body travels, and providing the information to the mobile body. [Means for solving the problem]

[0006] The invention described in claim 1 of the present application is an information collection device which includes a passenger status information collection unit which collects the status of a passenger of a moving body as passenger status information, a status change detection unit which detects a change in the status of the passenger based on the passenger status information and generates status change detection information, an external photographing unit which photographs the outside of the moving body and generates a photographed image, and an external situation information transmission unit which generates external situation information including the photographed image in accordance with the status change detection information and transmits the external situation information via a communication path.

[0007] In addition, the invention described in claim 7 of the present application is an information collection system, comprising: a passenger status information collection unit provided in a moving body for collecting the status of passengers of the moving body as passenger status information; a status change detection unit for detecting a change in the status of the passengers based on the passenger status information and generating status change detection information; an external photographing unit provided in the moving body for photographing the outside of the moving body and generating a photographed image; and an external situation information generation unit for generating external situation information including the photographed image in accordance with the status change detection information.

[0008] The invention described in claim 10 of the present application is an information collecting server comprising an external situation information receiving unit that receives, via a communication path, external situation information including a captured image of the outside of a moving body generated in response to detection of a change in the status of a passenger on board the moving body, a determination unit that analyzes the captured image included in the external situation information and determines the usefulness of the external situation information, and an external situation information distribution unit that distributes, via the communication path, the external situation information that has been determined to be useful. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall configuration diagram of an information collecting system according to the present invention; [Diagram 2] FIG. 1 is a diagram showing an example of how a terminal device is attached; [Diagram 3] FIG. 2 is a functional block diagram of a terminal device and a server. [Figure 4] 4 is an example of an operation routine of a passenger monitoring unit. [Diagram 5]4 is an example of an operation routine of an image extraction unit. [Figure 6] 13 is an example of an operation routine of a self-location identifying unit. [Figure 7] 4 is an example of an operation routine of a content selection unit. [Figure 8] 4 is an example of an operation routine of an information receiving unit. [Figure 9] 4 is an example of an operation routine of a content selection unit. [Figure 10] FIG. 2 is a functional block diagram of a terminal device and a server. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the following, preferred embodiments of the present invention will be described. However, these may be modified and combined as appropriate. In the following description and accompanying drawings, substantially the same or equivalent parts are denoted by the same reference numerals. In the following description, images are described as including moving images and still images. In the following description, transmission or reception of information includes transmission or reception of information signals. EXAMPLES

[0011] An information collecting system 100 according to a first embodiment of the present invention will be described below with reference to the accompanying drawings.

[0012] Fig. 1 is an overall configuration diagram of an information collection system 100 according to a first embodiment. As shown in Fig. 1, the information collection system 100 is configured by connecting a terminal device 10 as an information collection device mounted on an automobile M as a moving body and a server 50 via a network NW. Note that, although an automobile M is shown in Fig. 1 as an example of a moving body, the moving body may be a moving body other than an automobile, such as a bicycle, a motorcycle, an airplane, a ship, or a moving person.

[0013] Fig. 2 is a diagram showing an example of mounting the terminal device 10 to an automobile M. Fig. 2 shows the terminal device 10 mounted in the front seat portion of the automobile M as an example of mounting.

[0014] The exterior photographing camera 11, which serves as an exterior photographing unit that photographs the situation outside (surroundings) of the automobile M, i.e., the outside (external world) of the automobile M, is disposed, for example, on the dashboard DB. The exterior photographing camera 11 may be, for example, a wide-angle camera that can photograph a wide area in front of the automobile M through the windshield.

[0015] The exterior camera 11 may be installed anywhere as long as it can capture the situation outside the vehicle. For example, the exterior camera 11 may be installed behind the rearview mirror RM, that is, on the surface facing the windshield FG. The exterior camera 11 may also be installed on the outside of the automobile M, for example, on the front bumper. A plurality of exterior cameras 11 may also be installed so that they can capture images of the outside of the vehicle in multiple directions.

[0016] The in-vehicle camera 13 as a passenger photographing unit is a camera that photographs the passenger's actions and other conditions as passenger status information, which is passenger status information. The interior camera 13 may be provided anywhere as long as it can capture images of the interior of the vehicle and the passengers inside, for example, on the dashboard DB or on the rearview mirror RM.

[0017] The microphone 15 that receives sounds inside the vehicle as passenger status information is disposed, for example, on the dashboard DB. The microphone 15 may be disposed anywhere, such as the rearview mirror RM or the steering wheel, as long as it can receive sounds inside the vehicle.

[0018] The in-vehicle camera 13 and microphone 15 serve as a passenger status information collection unit that collects passenger status information.

[0019] The GPS receiver 17 that receives signals (GPS signals) from GPS (Global Positioning System) satellites is disposed, for example, on the dashboard DB. Note that the GPS receiver 17 may be disposed anywhere as long as it can receive GPS signals.

[0020] The display 19, which serves as a notification means for displaying, i.e., notifying, information received from the outside, etc., is disposed, for example, on the dashboard DB. The display 19 may be disposed in a location that is visible to the passengers. For example, the display 19 may be attached to the center console CC in the dashboard DB.

[0021] The main body 21 is connected to the exterior camera 11, the interior camera 13, the microphone 15, the GPS receiver 17, and the display 19, and includes a control unit (see FIG. 3) that controls these. The main body 21 is mounted, for example, in the center console CC portion in the dashboard DB.

[0022] 3 shows a functional block diagram of the control unit 23 of the terminal device 10 and the server 50. First, the control unit 23 will be described.

[0023] The passenger monitoring unit 25, which serves as a status change detection unit, can receive information from the interior camera 13 and microphone 15. The passenger monitoring unit 25 monitors the status of the passengers aboard the automobile M, i.e., the interior of the passenger compartment, based on the images or audio input transmitted from the interior camera 13 and microphone 15, and determines whether a change in the status of the passengers has been detected, for example, whether an abnormality or incident has occurred. When the passenger monitoring unit 25 determines that a change in the status of the passengers has been detected, it transmits status change detection information to the image extraction unit 27, the vehicle position identification unit 29, and the information transmission unit 31.

[0024] The image extraction unit 27 is connected to the outside-vehicle shooting camera 11 and receives images captured by the outside-vehicle shooting camera 11. The image extraction unit 27 extracts a part of the image received from the outside-vehicle shooting camera 11 based on the state change detection information from the passenger monitoring unit 25 to generate an extracted image. When the image extraction unit 27 generates the extracted image, it transmits the extracted image to the information transmission unit 31 as an image at the time when the state change is detected, that is, an event image capturing an event occurring outside the mobile body at the time when the state change is detected.

[0025] Alternatively, the outside-vehicle camera 11 may not always take pictures, but the passenger monitoring unit 25 may transmit status change detection information to the outside-vehicle camera 11, and the outside-vehicle camera 11 may take pictures in response to receiving the information. For example, the outside-vehicle camera 11 may take pictures when the status change detection information is received, or may take pictures for a certain period of time after receiving the status change detection information. In this case, a part of the taken image may be further extracted by the image extraction unit, or the taken image may be transmitted to the information transmission unit 31 as an event image without going through the image extraction unit.

[0026] The own position identifying unit 29 is connected to the GPS receiver 17, and can output position identifying information indicating the current position of the automobile M in which the terminal device 10 is mounted. When the own position identifying unit 29 receives state change detection information from the passenger monitoring unit 25, it identifies the position of the automobile M at the time when the state change detection information was received by the GPS signal from the GPS receiver 17, and transmits position identifying information, which is information on the position, to the information transmitting unit 31. In addition, the own position identifying unit 29 can transmit the position identifying information to other parts such as the information receiving unit 33 as necessary.

[0027] The self-position identifying unit 29 may identify the position of the automobile M based on the attitude information of the automobile M from a gyro device (not shown) or the vehicle speed information obtained from the vehicle speed pulse of the automobile M. The self-position identifying unit 29 may also hold map information or be able to acquire it from an external source. That is, the self-position identifying unit 29 may be able to identify the position of the automobile M by combining at least one of the GPS information from the GPS receiver 17, the attitude information of the automobile M from the gyro device, and the vehicle speed information of the automobile M with the map information.

[0028] The information transmitting section 31, which serves as an external situation information generating section or an external situation information transmitting section, is connected to a transmitter (not shown) capable of transmitting an information signal to the server 50 via the network NW (see FIG. 1).

[0029] When the information transmission unit 31 receives the status change detection information from the passenger monitoring unit 25, it generates information content as external situation information. The information transmission unit 31 receives event images from the image extraction unit 27 or the exterior vehicle shooting camera 11, and also receives position identification information from the vehicle's own position identification unit 29. Then, based on this received information, it generates information content as external situation information including information on the time when the status change detection information was received (for example, the time when the event image was captured), the event image, and the position identification information. That is, the information transmission unit 31 generates information content according to the status change detection information.

[0030] When the information transmission unit 31 generates the information content, it transmits the information content to the server 50. Note that even if the information transmission unit 31 does not directly receive the status change detection information from the passenger monitoring unit 25, the information transmission unit 31 may be deemed to have received the status change detection information from the passenger monitoring unit 25 upon receipt of an event image or location identification information.

[0031] The information receiving unit 33 is connected to a receiver (not shown) capable of receiving information from an external source such as the server 50 via the network NW. When the information receiving unit 33 receives information from the server 50, it displays information on the display 19 based on the information, and is capable of notifying passengers of information based on the information.

[0032] Next, the server 50 will be described. The information receiving unit 51, which serves as an external situation information receiving unit, is capable of receiving information content from the information transmitting unit 31 via the network NW. Upon receiving the information content, the information receiving unit 51 transmits it to the content selecting unit 53. Although not shown in the figure, the information receiving unit 51 receives information content from a plurality of terminal devices 10 each mounted on another automobile M.

[0033] The content selection unit 53, which serves as a selection unit or a determination unit, is a functional unit that receives information content and selects from the information content information content to be transmitted to the automobile M. The content selection unit 53 includes a content usefulness determination unit 55 that determines whether or not the information content is useful or significant from the viewpoint of contributing to the selection of a travel route for the automobile M from among the information content information.

[0034] For example, when a plurality of information contents are received and stored, the content usefulness determination unit 55 determines whether or not there are a large number (or a predetermined number or more) of information contents (hereinafter also referred to as neighboring contents) having location specifying information indicating the same or neighboring locations among the stored information contents. If there are a predetermined number or more of neighboring contents, the content usefulness determination unit 55 determines that the neighboring contents are useful or significant, and selects and transmits them to the representative content selection unit 57.

[0035] The nearby contents may be information contents that have location specifying information indicating the same or nearby locations and have event images taken in the same time period (for example, within 30 minutes before or after).

[0036] The content selection unit 53 includes a representative content selection unit 57 that selects information content that holds the most useful or significant or high-quality event image from the nearby content group as the representative content. The representative content selection unit 57 may, for example, analyze the event images held by the nearby content group and select information content that holds the clearest event image as the representative content. Also, information content that holds the most recently captured event image from the nearby content group may be selected as the representative content.

[0037] After selecting the representative content, the representative content selection unit 57 transmits it to the information transmission unit 59 .

[0038] The information transmitting unit 59, which serves as an external situation information distributing unit, is capable of transmitting an information signal to the terminal device 10 via the network NW. Upon receiving the representative content from the representative content selecting unit 57, the information transmitting unit 59 transmits it to the terminal device 10.

[0039] As described above, when the information receiving unit 33 of the terminal device 10 receives information from the server 50, it is possible to display and notify the user on the display 19 based on the information. This display is performed, for example, by displaying on a map the position of the automobile M and the position specifying information included in the information content, i.e., the position where the event occurred, and also by displaying an event image included in the information content.

[0040] Here, after receiving the representative content, the information receiving unit 33 may determine whether to display information based on the representative content on the display 19 based on the location identification information held by the representative content and the shooting time of the event image.

[0041] This determination may be made, for example, based on whether or not the current position of the automobile M in which the terminal device 10 is mounted and the position indicated by the position specifying information possessed by the representative content are within a predetermined distance.

[0042] In addition, if the automobile M is equipped with a navigation system, the route set in the navigation system may be determined based on whether it passes through or passes near the position indicated by the position identification information of the representative content.

[0043] [Operation routines for each part] The operation routines of each of the above-mentioned parts will be described below. Among the operation routines described below, the operation routines of each part in the control unit 23 of the terminal device 10 are started, for example, when a prime mover such as an engine or a motor of the automobile M is started, and are repeatedly executed until the prime mover is stopped. Also, an individual switch may be provided in the terminal device 10, and the operation routine may be started when the switch is turned on, and repeatedly executed until the switch is turned off.

[0044] 4 shows an example of an operation routine of the passenger monitoring unit 25. First, the passenger monitoring unit 25 judges whether or not a change in state, for example, an abnormality, has occurred in the passenger (step S1). In this judgment, for example, it may be judged whether or not a change in state has occurred in response to a change in the gaze state of the pilot or the passenger, specifically, based on whether or not a change in the gaze state has occurred. The change in the gaze state includes, for example, a change from a state of not gazing to a state of gazing, that is, a change in the degree of gaze, or a change in the direction of gaze, etc.

[0045] For example, when it is detected by gaze analysis using images transmitted from the in-vehicle camera 13 that the driver has been gazing in one direction for a certain period of time or longer, it may be determined that a state change has occurred (been detected).

[0046] Also, when it is detected that a majority of passengers, including the pilot, gaze in one direction for a certain period of time or more, it may be determined that a state change has occurred. Note that, even without performing gaze analysis, when the face direction of passengers, including the pilot, turns in one direction for a certain period of time or more, it may be determined that the pilot or passengers gaze in that one direction.

[0047] In addition, it may be determined that a state change has occurred when it is detected that the movement of the pilot's gaze direction (position) or line of sight direction exceeds a threshold value, or when the gaze direction (position) or line of sight direction moves at a speed that exceeds a threshold value.

[0048] Also, voice transmitted from microphone 15 may be recognized, and when specific keywords such as "accident," "construction," "serious," or "what's that?" are recognized, it may be determined that an abnormality has occurred. Note that the status change detection information may include gaze direction information, which is information on the direction in which the pilot or passenger gazed when the status change occurred. The status change detection information may also include the results of voice recognition.

[0049] In step S1, if it is determined that a state change has occurred (step S1: YES), the passenger monitoring unit 25 transmits state change detection information to the image extraction unit 27, the vehicle position identification unit 29, and the information transmission unit 31 (step S2), and the operation routine ends. In step S1, if it is determined that a state change has not occurred (step S1: NO), then the operation routine ends.

[0050] 5 shows an example of an operation routine of the image extraction unit 27. First, the image extraction unit 27 determines whether or not state change detection information has arrived from the passenger monitoring unit 25 (step S1). If it is determined in step S1 that state change detection information has not arrived (step S1: NO), the operation routine then ends.

[0051] When it is determined in step S1 that the status change detection information has arrived, the image extraction unit 27 extracts an image from the outside-vehicle shooting camera 11 to generate an extracted image (step S2). This extracted image may be generated by extracting a still image at the time when the status change detection information was received from the image from the outside-vehicle shooting camera 11.

[0052] Alternatively, this may be performed by extracting one or more still images taken around the time when the state change detection information was received, or a video taken within a predetermined time range around the time, from the image taken by the outside-vehicle camera 11. If the state change detection information includes gaze direction information of the passenger as described above, a partial area may be further cut out (trimmed) from the still image or video extracted based on the time when the state change detection information was received, based on the gaze direction information. In this case, the image extraction unit functions as an image cutout unit.

[0053] In this case, an area portion that coincides with the gaze direction of the passenger may be cut out and extracted based on the gaze direction information from the image or video extracted based on the time when the state change detection information was received.

[0054] After step S2 is completed, the image extraction unit 27 transmits the extracted image to the information transmission unit 31 as an image when a state change is detected, i.e., an event image capturing an event occurring outside the moving body when a state change is detected (step S3), and then the operation routine is terminated.

[0055] As described above, the exterior camera may not always take pictures, but the passenger monitoring unit 25 may transmit status change detection information to the exterior camera 11, and the exterior camera 11 may take pictures in response to receiving the information. For example, the exterior camera 11 may take pictures when it receives the status change detection information, or may take pictures for a certain period of time after it receives the status change detection information.

[0056] In this case, a part of the captured image may be further extracted in time by the image extraction unit 27 and transmitted as an event image to the information transmission unit 31. Also, a part of the captured image may be cut out by the image extraction unit 27 functioning as an image cutout unit and transmitted as an event image to the information transmission unit 31. Note that the captured image may be directly transmitted as an event image to the information transmission unit 31 without going through the image extraction unit 27.

[0057] 6 shows an example of an operation routine of the self-location identifying unit 29. First, the self-location identifying unit 29 determines whether or not state change detection information has arrived from the passenger monitoring unit 25 (step S1). If it is determined in step S1 that state change detection information has not arrived (step S1: NO), the operation routine is then terminated.

[0058] When it is determined in step S1 that the state change detection information has arrived, the self-position identifying unit 29 generates position identifying information for identifying the position of the automobile M in which the terminal device 10 is mounted (step S2). This position identifying information may be information of a position identified by a GPS signal from the GPS receiver 17. Note that the self-position identifying unit 29 may generate position identifying information for identifying the position of the automobile M based on attitude information of the automobile M from a gyro device (not shown) or vehicle speed information based on the vehicle speed pulse of the automobile M.

[0059] In addition, the self-position identifying unit 29 may hold map information or may be able to acquire it from an external source. That is, the self-position identifying unit 29 may generate position identifying information for identifying the position of the automobile M by combining the map information with at least one of the GPS signal from the GPS receiver 17, the attitude information of the automobile M from the gyro device, and the vehicle speed information of the automobile M.

[0060] After step S2 is completed, the own position identification unit 29 transmits the generated position identification information to the information transmission unit 31 (step S3), and then the operation routine is terminated.

[0061] The event image and position identification information transmitted to the information transmitting unit 31 are transmitted as information content including these to the information receiving unit 51 of the server 50 by the information transmitting unit 31. That is, in the information transmitting unit 31, information content including the event image and the position identification information is generated and transmitted to the information receiving unit 51 of the server 50. Upon receiving the information content, the information receiving unit 51 transmits it to the content selecting unit 53.

[0062] 7 shows an example of an operation routine of the content selection unit 53 consisting of the content usefulness determination unit 55 and the representative content selection unit 57. In this example, the content selection unit 53 determines whether or not a certain number of information contents including event images taken at positions within a certain range have arrived in order to determine whether the information contents are significant or useful. As described above, the information receiving unit 51 receives information contents from a plurality of terminal devices 10 mounted on a plurality of automobiles M, respectively.

[0063] First, the content selection unit 53 judges whether or not a predetermined number of information contents having location specifying information indicating the same or nearby locations (hereinafter also referred to as nearby contents) have arrived from the information receiving unit 51 (step S1). If it is judged in step S1 that the predetermined number of nearby contents have not arrived (step S1: NO), the operation routine is then terminated.

[0064] In step S1, when it is determined that a predetermined number or more of neighboring contents have arrived (step S1: YES), the content selection unit 53 selects one information content from the neighboring contents as a representative content (step S2).

[0065] This selection of representative content may be performed, for example, by analyzing event images held by each of the neighboring contents, determining the clearest event image as the representative image, and selecting the information content holding the representative image as the representative content. Alternatively, the most recently captured event image among the event images held by each of the neighboring contents may be determined as the representative image, and the information content holding the representative image may be selected as the representative content.

[0066] The information content may include the vehicle speed pulse of the automobile M when the event image was captured or information on the vehicle speed based thereon, in which case the vehicle speed pulse or the vehicle speed information may be used to select the clearest event image. For example, the vehicle speed pulse when the event image was captured may be analyzed, the event image captured at the slowest speed may be set as the representative image, and the information content holding the representative image may be set as the representative content.

[0067] The information content may also include information on the direction (position) or line of sight of the occupant of the automobile M when the state change detection information is transmitted from the occupant monitoring unit 25 (i.e., when the event image is captured). In this case, for example, it may be determined from the event image which lane the automobile M was traveling in (in which lane it was stopped) on the road on which the automobile M was traveling when the event image was captured, and the representative image may be determined.

[0068] For example, if the passenger is gazing to the left of the automobile M when the state change detection information is transmitted, the event image captured in the leftmost lane may be used as the representative image. Also, if the passenger is gazing to the right of the automobile M, the event image captured in the rightmost lane may be used as the representative image.

[0069] When step S2 is completed, the content selection unit 53 transmits the representative content, which is the information content including the representative image, to the information transmission unit 59 (step S3), and then the operation routine ends. When the information transmission unit 59 receives the information content from the content selection unit 53, it transmits it to the information reception unit 33 of the terminal device 10.

[0070] 8 shows an example of an operation routine of the information receiving unit 33. First, the information receiving unit 33 determines whether information content has arrived from the server (step S1). If it is determined that information content has not arrived (step S1: NO), the operation routine ends.

[0071] In step S1, when it is determined that information content has arrived from the server (step S1: YES), the information receiving unit 33 determines whether the arrived information content is useful to a passenger in the automobile M in which the terminal device 10 is installed (step S2).

[0072] In this determination, for example, when the position indicated by the position specifying information possessed by the arrived information content is in the vicinity of the current position of the automobile M (within a predetermined distance), the information content may be determined to be useful.

[0073] Also, for example, when a route is set in a navigation system mounted on the automobile M, it may be determined whether the arrived information content is useful or not based on the route. That is, for example, the information content may be determined to be useful by checking that the position indicated by the position specifying information held by the arrived information content is on the set route or in the vicinity of the route.

[0074] In step S2, when it is determined that the received information content is useful (step S2: YES), the information receiving unit displays information based on the information content on the display 19 (step S3). This display may be, for example, simply displaying the event image and position specifying information held by the information content. In addition, the location where the event image was captured may be displayed on a map based on the position specifying information.

[0075] In step S2, if it is determined that the received information content is not useful (step S2: NO), the operation routine ends. Also, when step S3 ends, the operation routine ends.

[0076] The above-mentioned information system 100, terminal device 10, and server 50 make it possible to collect information, including images, related to events occurring at unspecified positions on the moving path of a moving object, such as narrow roads, roads with little traffic, and other locations including routes where no information collection device is installed. Furthermore, it is possible to distribute the collected information to other moving objects.

[0077] In addition, the information system 100, the terminal device 10, and the server 50 can generate information including images taken at the time of a change in the condition of a passenger in a moving body, based on a change in the condition such as an abnormality, and analyze the information including the images to select and distribute useful or significant information, thereby making it possible to efficiently distribute only the necessary information. EXAMPLES

[0078] A system 200 according to a second embodiment of the present invention will be described below with reference to the drawings. The system 200 according to the second embodiment has the same configuration as the system 100 according to the first embodiment, except for the operation routine of the content selection unit 53. Therefore, the description of the operation routine of the content selection unit 53 will be omitted.

[0079] In the second embodiment, a case will be described in which the content selection unit 53 of the system 200 performs image analysis on an event image in information content transmitted from the terminal device 10, analyzes what event the event image captures, and determines the significance or usefulness of each piece of information content. That is, a case will be described in which the content selection unit 53 analyzes what event is occurring when the event image is captured, and determines the significance or usefulness.

[0080] 9 shows an example of an operation routine of the content selection unit 53 of the system 200 of the second embodiment. First, the content selection unit 53 determines whether information content has arrived from the terminal device 10 (step S1). If it is determined in step S1 that information content has not arrived (step S1: NO), the operation routine is then terminated. As described above, the information content arrives from multiple terminal devices 10 that are respectively installed in multiple automobiles M.

[0081] In step S1, when it is determined that information content has arrived (step S1: YES), the content selection unit 53 determines whether or not the arrived information content is useful or significant (step S2).

[0082] This determination is made, for example, by analyzing what the event image included in each information content is capturing, i.e., what kind of event is occurring. For example, when an event image is analyzed to be a useful image such as an "accident," "construction site," "traffic jam," "crowd," or "event," the information content including the event image may be determined to be useful or significant.

[0083] Specifically, for example, an event image is subjected to image analysis, and if it is determined that a large number of cars have not been moving for a certain period of time, it is determined to be a traffic jam. If the character "construction" is recognized, or if the yellow and black diagonal stripes seen at construction sites are recognized, it is determined to be a "construction site." If a large number of people are recognized, it is determined to be a "crowd" or an "event." If it is determined in step S2 that the information content is not useful or significant (step S2: NO), the routine then ends.

[0084] In step S2, if it is determined that the arrived information content is useful or significant (step S2: YES), the process proceeds to step S3. In step S3, the content selection unit 53 determines whether other information content including an event image captured in the same time period (for example, within 30 minutes before or after the capture time) and at the same location (or a nearby location) as the event image held by the information content determined to be useful or significant has arrived (step S3).

[0085] In step S3, if it is determined that other information content containing an event image taken at the same time and location has arrived (step S3: YES), the content selection unit 53 selects one information content from the information contents as a representative content (step S4).

[0086] As described in the first embodiment, the representative content may be selected, for example, by analyzing the event images held by each of the information contents, setting the clearest event image as the representative image, and selecting the information content holding the representative image as the representative content. Alternatively, the most recently captured event image among the event images held by the information contents may be set as the representative image, and the information content holding the representative image may be selected as the representative content.

[0087] The information content may include information on the vehicle speed pulse or vehicle speed of the automobile M when the event image was captured, and in that case, the vehicle speed pulse or vehicle speed information may be used to select the clearest event image. For example, the vehicle speed pulse when the event image was captured may be analyzed, the event image captured at the slowest speed may be set as the representative image, and the information content holding the representative image may be set as the representative content.

[0088] The information content may also include information on the direction (position) or line of sight of the occupant of the automobile M when the state change detection information is transmitted from the occupant monitoring unit 25 (i.e., when the event image is captured). In this case, for example, it may be determined in which lane the automobile M is traveling (in which lane it is stopped) from the event image, and the representative image may be determined.

[0089] For example, if the passenger is gazing to the left of the automobile M when the state change detection information is transmitted, the event image captured in the leftmost lane may be used as the representative image. Also, if the passenger is gazing to the left of the automobile M, the event image captured in the rightmost lane may be used as the representative image.

[0090] After step S4 is completed, and if it is determined in step S3 that other information content containing an event image taken at the same time and location has not arrived (step S3: NO), the content selection unit 53 transmits the information content containing the representative image as is to the terminal device 10 (step S5).

[0091] If it is determined in step S3 that other information content including an event image captured in the same time period and at the same location has not arrived, the information content determined to be useful or significant in step S2 is transmitted to the terminal device 10. After step S5 is completed, the operation routine ends.

[0092] In this case, the information content transmitted from the server 50 to the terminal device 10 may include the analysis result of the event image performed in step S2. That is, the transmitted information content may include information on the event shown in the event image (for example, "accident", "construction site", "traffic jam", "crowd", "event", etc.).

[0093] When information related to such an event is included in information content and transmitted to the terminal device 10, the information receiving unit may determine, based on the information related to the event, whether or not to display information based on the information content on the display 19. For example, when a user using the terminal device 10 requests only information on "accidents," only information based on the information content related to "accidents" may be displayed on the display 19.

[0094] Furthermore, if the automobile M is equipped with a navigation system, information on the event may be provided to the navigation system, and the navigation system may be prompted to change the route. For example, based on the information content, information that an accident has occurred on a route set in the navigation system may be provided, and the navigation system may be prompted to change the set route to avoid the accident.

[0095] Also, for example, based on the information content, information may be provided that an "event" such as fireworks is being held near the set route of the navigation system, and the set route may be prompted to be changed so as to pass through the location where the "event" is being held. Note that, in anticipation of congestion and traffic jams due to the "event," the set route may be prompted to be changed so as to avoid the location where the "event" is being held.

[0096] The above-mentioned information system 200, the terminal device 10, and the server 50 make it possible to collect information, including images, related to events that occur at unspecified positions on the moving path of a moving object, such as narrow roads, roads with little traffic, and other locations including routes where no information collection device is installed. Furthermore, it is possible to distribute the collected information to other moving objects.

[0097] In addition, the information system 200, the terminal device 10 and the server 50 generate information including images taken at the time of a change in status, such as an abnormality, of the occupant of a moving body, and analyze the information including the images to select and distribute useful or significant information, thereby making it possible to efficiently distribute only the necessary information.

[0098] Furthermore, the information system 200, the terminal device 10, and the server 50 can analyze images of an event captured and transmitted by a moving body, and distribute information including information on what kind of event is occurring. This makes it possible to distribute detailed information on the occurring event to other moving bodies such as automobiles. EXAMPLES

[0099] A system 300 according to a third embodiment will be described below with reference to Fig. 10. As shown in Fig. 10, in the system 300, the passenger monitoring unit 25 and the image extracting unit 27 provided on the terminal device 10 side in the first and second embodiments are provided on the server 50 side as a passenger monitoring unit 25S and an image extracting unit 27S. That is, a so-called cloud configuration is adopted in which the server side is responsible for the processing on the terminal device 10 side.

[0100] In the system 300, images and audio from the exterior camera 11, interior camera 13, and microphone 15, as well as position identification information from the vehicle's own position identification unit 29, are constantly transmitted by the information transmission unit 31 to the server 50 via the network NW. The transmitted images and audio are received by the information reception unit 51 of the server 50. The images and audio, as well as the position identification information, arrive from a plurality of terminal devices 10 mounted on a plurality of automobiles M, respectively.

[0101] Of the received images and sounds, the images and sounds from the in-vehicle camera 13 and microphone 15 are sent to the passenger monitoring unit 25S, and the images from the outside-vehicle camera 11 and the position identification information from the vehicle's own position identification unit 29 are sent to the image extraction unit 27S. In this transmission, the images and sounds and the position identification information may be transmitted in a manner that allows them to be identified for each automobile M that is the source of the transmission.

[0102] In the system 300, the passenger monitoring unit 25S monitors the state of the passenger by executing, for example, an operation routine similar to that of the passenger monitoring unit 25 of the first embodiment shown in Fig. 4. Also, the image extracting unit 27S executes, for example, an operation routine similar to that of the image extracting unit 27 of the first embodiment shown in Fig. 5. Note that, in step S3 of Fig. 5, the image extracting unit 27S transmits the shooting time and position identification information of the event image together with the extracted event image to the content selecting unit 53.

[0103] The content selection unit 53 executes, for example, an operation routine similar to that of the above-mentioned first or second embodiment (for example, the operation routine shown in FIG. 7 or FIG. 9) to select information content to be transmitted to the terminal device 10. Thereafter, the selected information content is transmitted from the information transmission unit 59.

[0104] The transmitted information content is received by the information receiving unit 33. The information receiving unit 33 executes, for example, an operation routine similar to that of the above-mentioned embodiment 1 or 2 (for example, the operation routine shown in FIG. 8). The information receiving unit 33 displays information on the display 19 as necessary in a manner similar to that described in the embodiment 1 or 2.

[0105] According to the system 300 of the third embodiment, in addition to the advantages of the first or second embodiment, it is possible to reduce the processing load of the terminal device 10 and the power consumption of the terminal device 10. It is also possible to reduce the size of the terminal device 10.

[0106] In the above embodiment, a wide-angle camera is used as an example of the outside-vehicle camera 11, but various other cameras can be used. For example, a camera that is mounted on an angle-adjustable mounting base and has a changeable shooting direction may be used. In this case, the shooting direction of the outside-vehicle camera 11 may be changed to the gaze direction based on the gaze direction information in the state change detection information transmitted by the passenger monitoring unit 25, and a part or all of the captured image may be used as an event image.

[0107] In addition, in the above-described embodiment, the passenger monitoring unit 25 determines that a change in the status of a passenger inside the vehicle has occurred from images from the interior camera 13 and audio from the microphone 15, which serve as a passenger status information collection unit, but the determination may also be made using other information.

[0108] For example, the passenger status information collecting unit may be provided with a biosensor that detects the passenger's bioinformation (vital signs), and based on information from the biosensor (e.g., change in heart rate or blood pressure), it may determine that a status change has occurred in the passenger in the vehicle and transmit status change detection information. This biosensor may be attached to the passenger, or may be provided in the steering wheel, seat, or seat belt, for example.

[0109] The passenger status information collecting unit may also include a means for collecting information regarding steering operations for the automobile M. In this case, the passenger monitoring unit 25 may detect operations such as sudden acceleration, sudden braking, and sudden steering, and transmit status change detection information based on these operations.

[0110] In addition, in order to reduce detection of unnecessary state changes, for example, when the shift lever is moved to the reverse position, the state change may be detected taking into consideration the situation. For example, when the shift lever is moved to the reverse position, gazing at the rear of the automobile M may not be detected as a state change.

[0111] In the above embodiment, the terminal device 10 has the display 19 as a notification means, but the notification means may include a speaker. In this case, a notification based on the information content may be made by display on the display 19 and sound from the speaker.

[0112] Furthermore, a notification based on the information content may be made only by voice. In the case of a notification only by voice, location-specific information included in the information content, that is, the location of an event, the time of the event, and the content of the event, may be notified by voice.

[0113] In the above embodiment, the content selection unit 53 selects one piece of information content as the representative content from the neighboring contents, but multiple pieces of information content may be selected as the representative content. In this case, for example, multiple pieces of information content may be selected as the representative content in order of clarity of the event image included.

[0114] In the above embodiment, the terminal device 10 has the information receiving unit 33 and the display 19, but the terminal device 10 does not have to include the information receiving unit 33 and the display 19. In other words, the terminal device 10 may be an information collecting device that only transmits information to the server 50.

[0115] Furthermore, in the above embodiment, the representative content is transmitted to all moving bodies M, but the representative content may be transmitted to some of the automobiles M. For example, the server 50 may hold position information of each automobile M, and the representative content may be transmitted to only automobiles M located in the vicinity of the position indicated by the position specifying information contained in the representative content, i.e., within a certain distance from the position indicated by the position specifying information.

[0116] In the above embodiment, the information content includes location specifying information indicating the location information of the automobile M, but the location specifying information does not have to be included. In this case, the server 50 may specify the location of the automobile M from information such as which base station of the mobile communication network the wireless communication from the automobile M is passing through. Also, the location of the event image may be specified from the event image included in the information content.

[0117] In addition, the representative content may be transmitted only to automobiles M that are located in a position where the arrival time at the position indicated by the position identification information contained in the representative content is within a certain period of time when the automobiles M head toward the position indicated by the position identification information.

[0118] Furthermore, the information content from the server 50 may be receivable by devices other than the terminal device 10, such as a smartphone, a tablet, a PDA, or a personal computer.

[0119] In the above embodiment, the terminal device 10 is mounted on an automobile M as an example of a moving body, but the moving body is not limited to an automobile. For example, the moving body on which the terminal device 10 is mounted may be a ship, an airplane, a motorcycle, a bicycle, etc.

[0120] In addition, a person moving on foot may carry the terminal device 10 and the terminal device 10 may transmit information content to the server 50. In this case, the in-car camera of the terminal device 10 as a passenger photographing means photographs the person moving.

[0121] For example, the terminal device 10 may be realized by a smartphone or a tablet. In that case, the inside camera of the smartphone or tablet may correspond to the in-vehicle camera described in the first to third embodiments, and the outside camera may correspond to the outside-vehicle camera described in the first to third embodiments.

[0122] 3 and 10, the configurations of the terminal device 10 and the server 50 are not limited to these examples. For example, the passenger monitoring units 25 and 25S, the image extraction units 27 and 27S, and the content selection unit 53 may be located in either the terminal device 10 or the server 50.

[0123] The various configurations, routines, etc. in the above-described embodiment are merely examples, and can be appropriately selected according to the application, etc. [Explanation of symbols]

[0124] 100, 200, 300 Information Collection System 10 Terminal Equipment 11. Exterior camera 13. In-car camera 15. Mike 17 GPS receiver 19 Display 21 Main unit 23 Control Unit 25, 25S Passenger Monitoring Unit 27, 27S Image extraction section 29 Self-location part 31 Information Transmission Department 33 Information Receiving Department 50 Servers 51 Information Receiving Unit 53 Content Selection Department 55 Content usefulness determination unit 57 Representative Content Selection Section 59 Information Transmission Department

Claims

[Claim 1] an occupant status information collection unit that collects the status of an occupant of the moving body as occupant status information; a state change detection unit that detects a state change of the occupant based on the occupant state information and generates state change detection information; an external image capturing unit that captures an image of the outside of the moving object and generates a captured image; an external situation information transmitting unit that generates external situation information including the captured image in response to the state change detection information and transmits the external situation information via a communication path; 13. An information collecting device comprising:

Images