Processing unit, system, and program

Abstract

We provide technology for utilizing images captured inside a moving vehicle. [Solution] The processing device 40 has at least one of the following functions: a selection function that selects content to be displayed on a display device 22 located inside the mobile body from among multiple contents based on images captured by a camera 35 that takes pictures inside the mobile body, and an evaluation information acquisition function that acquires information that forms the basis for evaluating the content displayed on the display device 22 based on images captured by the camera 35.

Description

Technical Field

[0001] The present invention relates to a processing device, a system, and a program.

Background Art

[0002] There is known a device that installs a video camera inside a train vehicle and extracts and displays the boarding rate from the image of the video camera (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One object of the present invention is to provide a technique for utilizing an image captured inside a moving body. For example, in a moving body such as a train, a bus, or an elevator, there may be arranged a display device for displaying, for example, commercial or other content. Another object of the present invention is to provide a technique for utilizing an image captured inside the moving body on which this content is displayed.

[0005] The purpose of the present invention is not limited thereto, and the applicant intends to obtain rights through divisional applications, amendments, etc., for configurations that aim to obtain the effects derived from the components of the configuration disclosed in this specification and the drawings, etc. For example, problems that can be described as "can be achieved" in this specification are disclosed here by reinterpreting them as "the problem is...". Each problem is described independently, and the applicant intends to obtain rights to the configurations for solving these problems individually through divisional applications, amendments, etc. Even if a problem is implicitly understood from the description in the specification, the applicant intends to include a part of the configuration described in this specification in the claims through amendment or divisional application. Furthermore, problems that combine these independent problems are also disclosed. [Means for solving the problem]

[0006] (1) The processing device may have at least one of the following functions: a selection function that selects content to be displayed on a display device placed inside the mobile body from among multiple contents based on images captured by a camera that takes pictures inside the mobile body, and an evaluation information acquisition function that acquires information that forms the basis for evaluating the content displayed on the display device based on images captured by the camera.

[0007] In this way, it is possible to select content to display based on images taken inside a mobile vehicle, or to obtain information that serves as the basis for evaluating the content, thereby providing technology for utilizing images taken inside a mobile vehicle on which content is displayed.

[0008] (2) The processing device may be equipped with at least the selection function, the selection function including a function to select content based on at least one of the attributes and actions of a person in the mobile device.

[0009] In this way, content can be selected that corresponds to at least one of the attributes and actions of the person inside the moving vehicle.

[0010] (3) The processing device is provided with at least the selection function, the selection function having the function of detecting a person looking at the display device based on an image taken by the camera, and selecting content to display on the display device based on the attributes of the person looking at the display device.

[0011] In this way, content tailored to the person actually looking at the display device can be displayed on the device.

[0012] (4) The camera is positioned on a moving vehicle that travels sequentially through a plurality of stops where people get on and off, and the processing device is equipped with at least the selection function, and the selection function is a processing device that includes a function to select content for each travel section between the stops and to display the content selected in accordance with the travel section on the display device during the period in which the vehicle is traveling through that section.

[0013] In this way, even if the people inside the moving vehicle may change from section to section, it is possible to select and display content on the display device that corresponds to the people inside the vehicle at the current time.

[0014] (5) The processing device is equipped with at least the evaluation information acquisition function, and the evaluation information acquisition function is preferably equipped with a function that acquires information about the number of people looking at the display device based on images taken by the camera while the display device is displaying content.

[0015] In this way, information about the number of people in a moving object while the content is being displayed can be obtained as basic information to evaluate the content.

[0016] (6) The evaluation information acquisition function may be a processing device that includes a function to detect the number of people inside the moving vehicle and the gaze of the people based on the image captured by the camera.

[0017] By doing so, it is possible to obtain information regarding the number of people in the moving body and the line of sight of the people during the display of the content, as information serving as a basis for evaluating the content.

[0018] (7) It is preferable that the processing device further has a function of associating and recording the image captured by the camera, the information for identifying the content displayed on the display device, and the information serving as a basis for evaluating the content.

[0019] By doing so, it is possible to associate and record the content and the information serving as a basis for evaluating the content.

[0020] (8) It is preferable that the processing device further has a function of detecting a person looking at the display device based on the image captured by the camera, and associating and recording information for identifying the person looking at the display device in association with the content displayed on the display device.

[0021] By doing so, it is possible to associate and record the content and the information for identifying the person actually looking at the display device.

[0022] (9) It is preferable that the processing device further has a function of detecting the position within the display device at which the person is looking, and associating and recording the content displayed on the display device and the position being looked at by the person.

[0023] By doing so, it is possible to record information for knowing at which position within the video content a person in the moving body is looking.

[0024] (10) It is preferable that the processing device has at least the evaluation information acquisition function, and the evaluation information acquisition function performs content evaluation based on the information acquired by the evaluation information acquisition function.

[0025] By doing so, it is possible to evaluate the content displayed on the display device arranged within the vehicle.

[0026] (11) Based on the image captured by the camera, for each person who has boarded the moving body, detect the movement path within the moving body from boarding to alighting, and it is preferable to further include a processing device having a function of recording the information of the boarding point, the detection result of the movement path within the moving body, and the information of the alighting point.

[0027] In this way, it is possible to record information for knowing the movement state of a person from boarding the moving body to alighting.

[0028] The function of the processing device described in the above (11) may be realized by a processing device that does not have the functions of the processing device described in (1) to (10).

[0029] (12) Instead of or in combination with causing the display device to display content, it is preferable to further include a processing device having a function of causing the display device to display the image captured by the camera or an image processed therefrom.

[0030] In this way, it is possible to show the image captured by the camera to the person in the moving body instead of or in combination with the content.

[0031] (13) A locking mechanism is provided at the luggage storage area, and it is preferable to use a processing device that sets the locking mechanism to either a locked state or an unlocked state based on the characteristics of the person who visited the luggage storage area in the image captured by the camera.

[0032] [[ID=​​​​​​​​In this way, it is possible to make it difficult to see or recognize information contained in a designated area of ​​an image captured by a camera.

[0035] (15) The processing device may further include a function for recording images captured by the camera in a recording device, and a function for reading the images recorded in the recording device and selectively outputting image data in which a predetermined image processing has been applied to the face of a person, or image data in which the predetermined image processing has not been applied.

[0036] In this way, it is possible to output an image after applying a predetermined image processing to the faces of people included in the image, or output it without applying any image processing.

[0037] (16) The mobile body includes a plurality of vehicles, the mobile body is operated so as to pass through a plurality of stations in order, and at each of the plurality of stations the plurality of vehicles stop at a predetermined stopping position, and the processing device further has a function to calculate the congestion rate for each vehicle based on images taken by the camera, and a function to select content from a plurality of contents to be displayed on a station display device positioned corresponding to the predetermined stopping position, according to the calculated congestion rate for each vehicle, for each predetermined stopping position.

[0038] In this way, the movement of a person intending to board a mobile vehicle can be guided by selecting the content to be displayed on the station's display device.

[0039] (17) The system comprises the above-described processing device and a photographing device, wherein the photographing device includes the camera, a housing for housing the camera, and a posture adjustment mechanism for changing and fixing the orientation of the camera lens relative to the housing.

[0040] This method allows you to adjust the camera's orientation while the housing remains fixed to the moving object.

[0041] (18) The posture adjustment mechanism may have an intermediate support member that is rotatably supported with respect to the housing, the camera is rotatably supported with respect to the intermediate support member, and the rotation axis of the intermediate support member with respect to the housing and the rotation axis of the camera with respect to the intermediate support member are mutually orthogonal to each other.

[0042] In this way, the camera's orientation can be adjusted in at least two directions.

[0043] (19) The housing includes a support portion for supporting the camera and a cover that covers the camera supported by the support portion, and further includes a reader / writer for reading and writing data to an external storage medium for recording images taken by the camera, and the system has an opening through which the external storage medium passes when the external storage medium is attached to the reader / writer and when the external storage medium is removed from the reader / writer, and a rotating lid that rotates to achieve a state in which the opening is closed and a state in which the opening is open.

[0044] In this way, the cover and rotating lid can be used to prevent theft of external storage media.

[0045] (20) It is preferable to provide a program for a computer to implement the functions of the processing device described in any of (1) to (19) above.

[0046] The inventions described in (1) to (19) above can be combined in any way. For example, one may combine all or part of the configuration of the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. In particular, it is preferable to create an invention that combines the invention described in (1) with at least part of the configuration of at least one of the inventions described in (2) and onward. Alternatively, one may extract any configuration from the inventions described in (1) to (19) and combine the extracted configurations. The applicant of this application intends to obtain rights to inventions that include these configurations.

[0047] (21) The image playback device may include a function to acquire image data taken by cameras mounted on each of the multiple cars of the train, a function to display information on the configuration of the multiple cars of the train and the mounting positions of the cameras inside the cars on a display screen, a function to select a specific camera from the information displayed on the display screen, and a function to display images taken by the selected camera on the display screen.

[0048] In this way, the operator of the image playback device can easily select the camera from which the image they want to view was taken.

[0049] (22) The image playback device may further include a function to acquire sensor detection information that is detected by a sensor installed on the vehicle and associated with image data captured by the camera, and a function to display the image captured by the camera and the sensor detection information on a display device in association with each other.

[0050] In this way, images captured by the camera can be associated with sensor detection information.

[0051] (23) It is preferable to provide a program for a computer to implement the functions of the image display device described in (21) or (22) above. [Effects of the Invention]

[0052] According to the present invention, images captured inside a mobile vehicle can be used for content selection and content evaluation.

[0053] The effects of the present invention are not limited thereto, and the effects produced by the components of the structure disclosed in this specification and drawings are also disclosed. The present invention intends to obtain rights to the components that produce such effects through divisional applications, amendments, etc. For example, the phrases "can do..." in this specification are descriptions that clearly indicate the effects produced, and there are components that produce effects even without such descriptions. Furthermore, there are effects that can be grasped by the component even without such descriptions. [Brief explanation of the drawing]

[0054] [Figure 1] Figure 1A is a side view of the interior of one train car equipped with a camera and display device that houses the processing device according to the embodiment, and Figure 1B is a plan view of the interior of one train car equipped with a camera and display device that houses the processing device according to the embodiment. [Figure 2] Figure 2A is a perspective view of the imaging apparatus containing the processing device according to the embodiment, and Figure 2B is a perspective view of the imaging apparatus with the cover removed. [Figure 3] Figures 3A, 3B, 3C, 3D, 3E, and 3F are the top view, left side view, front view, right side view, bottom view, and rear view of the imaging device, respectively. [Figure 4] Figure 4 is a block diagram of the processing apparatus according to an embodiment. [Figure 5] Figure 5 is a flowchart illustrating the content selection function based on passenger attributes. [Figure 6] Figure 6 is a flowchart illustrating the content selection function based on passenger actions. [Figure 7] Figure 7 is a flowchart illustrating the function for acquiring evaluation information of content displayed on a display device. [Figure 8] Figure 8 is a flowchart showing the function of recording after image processing. [Figure 9] Figure 9A is a flowchart showing the function of recording moving images captured by the camera, and Figure 9B is a flowchart showing the function of outputting image data. [Figure 10] Figures 10A and 10B are perspective views of the locking mechanism installed in the luggage compartment inside the electric train car, in the unlocked and locked states, respectively. [Figure 11] Figures 11A and 11B are front and top views, respectively, of other locking mechanisms installed in the luggage compartment inside the electric train. [Figure 12]Figures 12A and 12B are schematic diagrams showing a train on the tracks, a station display device installed on the station platform, and a line of passengers. [Figure 13] Figure 13 is a perspective view of the imaging device (Figures 2A and 2B) with the cover removed, based on other configuration examples. [Modes for carrying out the invention]

[0055] The apparatus according to the embodiment will be described with reference to Figures 1A to 13. The apparatus according to the embodiment constitutes a photographic device housed in a common housing together with a camera. This photographic device is mounted on a train together with a display device. In this embodiment, the case where the moving object is a train car will be used as an example. A train car is an example of a vehicle that operates carrying an unspecified number of passengers. In this example, the people inside the car are assumed to be train passengers.

[0056] Figures 1A and 1B show a side view of the interior of a train car equipped with a camera and display device, and a plan view of the interior of the train car, respectively, which house the processing devices according to each embodiment. Multiple doors 21 are arranged on both the left and right sides of the train car body 20 in the direction of travel. A display device (which may also be called a train digital signage) 22 and a box-shaped camera device 23 are mounted on the wall surface (called a fascia) above the doors 21. The camera device 23 is mounted, for example, on either the left or right side of the display device 22.

[0057] In this embodiment, three doors 21 are arranged on each side of the vehicle. The display device 22 is mounted on the fascia above all the doors 21. The display device 22 is positioned in a predetermined location within the vehicle, but it is preferable that it be positioned in a location where passengers inside the vehicle can easily see the images it displays. For example, it is preferable that it be positioned higher than the average height of a person, relative to the floor of the vehicle. The display device 22 may be a liquid crystal display device, an organic EL display device, or a display device having other display elements. The display device 22 is not limited to being installed on the wall of the vehicle, but may also be implemented by, for example, digital signage, electronic paper hanging advertisements, or other display devices placed inside the vehicle.

[0058] The camera 23 is mounted on the upper panel of one of the left and right doors 21, which are located in the same position in the direction of travel. On the right side wall of the vehicle (upper side in Figure 1B), the camera 23 is mounted above the odd-numbered doors 21. On the left side wall of the vehicle (lower wall in Figure 1B), the camera 23 is mounted above the even-numbered doors 21. These multiple camera 23s allow for imaging of almost the entire interior of the vehicle.

[0059] Figure 2A is a perspective view of the imaging device 23 housing the processing apparatus according to the embodiment, and Figure 2B is a perspective view of the imaging device 23 with the cover removed. Figures 3A, 3B, 3C, 3D, 3E, and 3F are the top view, left side view, front view, right side view, bottom view, and rear view of the imaging device 23, respectively.

[0060] The support section 31 and the cover 32 constitute the housing of the imaging device 23. The housing of the imaging device 23 houses the camera 35 and the recording unit body 34. The recording unit body 34 is fixed to the support section 31 on its front side. The recording unit body 34 covers approximately the lower half of the front side of the support section 31. The camera 35 is located on the front side of the support section 31 and above the recording unit body 34. The camera 35 is supported by the support section 31 via the recording unit body 34. The electronic circuit board of the processing unit and the like are housed inside the recording unit body 34. The support section 31 is fixed to the wall surface of the vehicle. The cover 32 covers the camera 35 and the recording unit body 34 from the front side of the support section 31.

[0061] The support portion 31 is made of, for example, metal. The cover 32 is supported by the support portion 31 and covers the camera 35. The cover 32 has a translucent area on its front side. This is so that the cover 32 does not obstruct the camera 35 from taking pictures. The translucent area is, for example, an area in which a transparent material (for example, resin or glass) is provided at the opening of the cover 32. The other parts of the cover 32 are made of, for example, die-cast aluminum. The camera 35 takes pictures through the translucent area. The cover 32 is attached to the support portion 31 by Torx screws 33. The upward-facing surface of the cover 32 is inclined downward toward the front.

[0062] Camera 35 captures images and generates image data showing the captured images. Camera 35 includes, for example, a CCD, CMOS, or other image sensor, and a lens that focuses light at the position of the image sensor. Camera 35 has the function of capturing moving images, but it is preferable that it also has the function of capturing still images. Camera 35 should be a camera with a wide-angle lens in order to capture as wide an area as possible. If one camera cannot capture the entire area inside the vehicle, multiple cameras should be placed to prevent blind spots inside the vehicle. If multiple display devices 22 are placed inside the vehicle, it is preferable to place at least one camera 35 for each display device 22. If it is necessary to capture a wider area with one camera 35, a 360-degree camera such as a hemispherical camera or a 360-degree camera should be used.

[0063] Furthermore, if the camera 35 placed inside the vehicle is facing the passenger, the passenger may feel uneasy, as if they are constantly being monitored. By enclosing the camera 35 in a housing, it becomes more difficult for passengers to perceive the camera's orientation, thus reducing the likelihood of causing such anxiety.

[0064] The camera 35 is supported by the recording unit body 34 via a posture adjustment mechanism 36. The posture adjustment mechanism 36 is a mechanism that changes and fixes the orientation of the camera 35's lens relative to the housing of the shooting device 23. The posture adjustment mechanism 36 allows the orientation of the camera 35 to be adjusted while the housing is fixed to the wall of a train or the like.

[0065] Generally, to increase the maximum swing angle of the camera's attitude, the housing that houses the camera must be larger. Conversely, if the maximum swing angle of the camera's attitude is too small, it may not be possible to capture the desired range within the camera's field of view while the housing is fixed inside the train. For this reason, for normal use inside a train, it is considered sufficient to be able to change the direction of the camera 35 within a range of ±10 degrees.

[0066] The posture adjustment mechanism 36 includes an intermediate support member 36A. The intermediate support member 36A is L-shaped in this embodiment. One side of the intermediate support member 36A, which extends in the left-right direction when the imaging device 23 is viewed from the front, is a plate-shaped portion that follows the upper surface of the recording unit body 34. This one side of the intermediate support member 36A extends in the left-right direction when the imaging device 23 is viewed from the front and is rotatably supported relative to the housing of the imaging device 23 and the recording unit body 34. The other side of the intermediate support member 36A is located on the left side when the imaging device 23 is viewed from the front and is a plate-shaped portion that follows the left side of the camera 35. The camera 35 is rotatably supported relative to the other side of the intermediate support member 36A. The rotation axis 36B of the intermediate support member 36A relative to the housing of the imaging device 23 and the recording unit body 34 and the rotation axis 36C of the camera 35 relative to the intermediate support member 36A intersect (orthogonal in this embodiment). The rotation axis 36B extends directly upward from the recording unit body 34. The rotation axis 36C passes through the other side of the intermediate support member 36A. The rotation angles of the rotation axes 36B and 36C vary between multiple (e.g., three) scale positions and can be fixed at each scale position. In this way, the camera 35 has two rotation axes relative to the recording unit body 34 and can rotate in two intersecting directions. In this embodiment, the rotation axis 36B is in the vertical direction (i.e., up and down direction), and the rotation axis 36C is in the horizontal direction (i.e., left and right direction). In this way, the camera 35 can adjust its orientation with respect to the horizontal and vertical directions.

[0067] It is preferable to provide a scale on the intermediate support member 36A or the housing so that the degree to which the camera 35 has rotated relative to the housing from its reference position can be visually confirmed. The scale should be marked in increments of 5 degrees, for example. The position of the camera 35 relative to the housing may be common to different types of vehicles and mounting locations on which the imaging device 23 is attached. If this common position is known in advance, it is preferable to adjust the position of the camera 35 to the optimal position at the factory before shipment. This eliminates the need to adjust the position of the camera 35 on-site, thereby improving on-site work efficiency.

[0068] Figure 4 is a block diagram of a processing unit 40 according to an embodiment. The processing unit 40 includes a central processing unit (e.g., CPU; hereinafter referred to as "CPU") 41, a reader / writer 42, a storage device 43, and an image data output terminal 44. The storage device 43 stores a program to be executed by the CPU 41. By executing this program, the CPU 41 functions as a control means to realize various functions of the processing unit 40. The storage device 43 includes a main memory used as a work area for arithmetic processing, and an auxiliary storage device, such as a hard disk drive.

[0069] The reader / writer 42 is an SD card reader / writer that reads and writes image data and other data to and from an SD card inserted in the reader / writer 42 under the control of the CPU 41. An SD card that records images captured by the camera 35 is inserted in the reader / writer 42. An SD card is an example of an external storage medium or removable media. The CPU 41 outputs an image signal from the image data output terminal 44. When attaching the shooting device 23 to the wall of the vehicle, the operator can connect a monitor to the image data output terminal 44 to check the shooting range of the camera 35.

[0070] The image data output terminal 44 is a terminal for outputting image data, such as an NTSC output terminal. The operator can connect an image monitoring device to the image data output terminal 44 and adjust the orientation of the camera 35 while checking the current image. To facilitate checking the current image, the image data output terminal 44 should be located on the front of the processing unit 40 when the shooting device 23 is fixed to the wall of the vehicle or the like. To allow the image to be viewed even while the shooting device 23 is in operation, the cable should be routed to the top of the housing and connected to the image signal cable provided in the vehicle.

[0071] Image data representing the image captured by camera 35 is input to processing unit 40. Based on this image data, processing unit 40 selects content to be displayed on display device 22. Processing unit 40 outputs information identifying the selected content (hereinafter referred to as "content type information") to display device 22. Display device 22 displays the video content identified by the content identification information input from processing unit 40. The video content is assumed to be stored in display device 22, but it may also be stored in storage device 43 or other storage device. In the latter case, processing unit 40 should output the content data to display device 22.

[0072] The processing unit 40 may be installed, for example, in the same vehicle as the vehicle on which the camera 35 is located. The camera 35 and the processing unit 40 may be housed in the same enclosure to form a single device. For example, if the display device 22 has already been installed in the vehicle by the vehicle owner (in this embodiment, the railway operator), the camera 35 and the processing unit 40 may be retrofitted to the vehicle. Alternatively, the display device 22 and the processing unit 40 may be housed in the same enclosure to form a single device. For example, if the camera 35 has already been installed in the vehicle by the vehicle owner, the display device 22 and the processing unit 40 may be retrofitted to the vehicle.

[0073] Next, the operation of this embodiment will be described. Note that where it is stated that "processing device 40" is the entity that executes the control, it may be read as "CPU 41".

[0074] [A. Content selection function based on passenger attributes] Next, referring to Figure 5, we will describe the content selection function based on passenger attributes, which is one of the functions of the processing unit 40 and the display device 22.

[0075] Figure 5 is a flowchart of the content selection function based on passenger attributes. The processing shown in Figure 5 is executed each time the processing unit 40 detects that the train has departed the station. The method for detecting that the train has departed the station is not limited, but for example, the processing unit 40 may detect that the train has arrived at and departed the station by detecting the opening and closing of the train's passenger doors based on images captured by the camera 35. When the train departs the station, the processing unit 40 first acquires image data from the camera 35 (step SA1).

[0076] The processing unit 40 performs image analysis on the acquired image data to determine the attributes of the passengers (step SA2). For image analysis, known image analysis techniques such as those using neural network technology may be used. Passenger attributes include, for example, gender and age. The processing unit 40 divides the passengers into eight age groups, for example, 0 to under 10 years old, 10 to under 20 years old, 20 to under 30 years old, 30 to under 40 years old, 40 to under 50 years old, 50 to under 60 years old, 60 to under 70 years old, and 70 years and older. The processing unit 40 counts the number of men and women for each age group. In total, the processing unit 40 counts the number of passengers for each of the 16 age and gender groups.

[0077] The processing unit 40 selects one or more pieces of content to be displayed from a plurality of commercial contents pre-registered in the storage device 43, for example, according to the distribution of the number of passengers by age and gender group (step SA3). The processing unit 40 selects, for example, multiple pieces of content that are likely to interest people of the age and gender group with the largest number of people. The processing unit 40 notifies the display device 22 of the identification information of the selected content (step SA4).

[0078] In this case, the processing unit 40 sequentially displays videos of multiple content selected from pre-registered content on the display screen of the display device 22. Thus, the processing unit 40 has the function of dynamically changing the content displayed on the display device 22 according to the attributes of the current passenger.

[0079] The content that the processing unit 40 displays on the display device 22 should be video content such as videos and still images. In particular, it is good to display on the display device 22 commercial videos introducing a company's products and services, company introduction videos introducing the company itself, job postings, and information that will attract the attention of passengers. The processing unit 40 should also select content from several pre-prepared contents that is likely to interest multiple passengers captured in the image and display it on the display device 22. The processing unit 40 should select content according to the attributes of the majority of passengers among the multiple passengers captured in the image. Note that the attributes of passengers are not limited to gender and age, but may also include clothing and other attributes.

[0080] The processing unit 40 determines the attributes of passengers each time they depart from a station, and can select content based on the attributes of new passengers after passengers have disembarked at the station. Therefore, the processing unit 40 can display content that is highly appealing according to the current passenger attributes.

[0081] The processing unit 40 selects content for each section of the train's journey between stops (in this embodiment, between stations), and displays the selected content on the display device 22 during the period the train is traveling through that section. In this way, even if passengers on the train change at stops, the display device 22 can display content that is appropriate for the current passengers.

[0082] [B. A modified version of the content selection function based on passenger attributes] (B-1) The processing unit 40 may adjust the time allocation of the types of content displayed on the display device 22 according to the ratio of passenger numbers for each attribute. For example, the processing unit 40 may increase the display time of the content selected according to an attribute as the proportion of passengers with a certain attribute increases.

[0083] If the majority of passengers are women, the processing unit 40 may select content for display on the display device 22 that is aimed at women, such as commercials for cosmetics or women's magazines. If the majority of passengers are elderly, the processing unit 40 may select content for display on the display device 22 that is aimed at the elderly, such as commercials for canes or nursing care products. If the majority of passengers are wearing ties, the processing unit 40 may select content for display on the display device 22 that is aimed at the economy. If the processing unit 40 determines, based on the age of the passengers, that there are many passengers who like games, the processing unit 40 may select content for display on the display device 22 that is aimed at games. If there are many passengers reading books, the processing unit 40 may select content for display on the display device 22 that is aimed at books. If there are many passengers using smartphones, the processing unit 40 may select content for display on the display device 22 that is aimed at smartphone apps or job advertisements for app development companies.

[0084] The processing unit 40, through its content selection function based on passenger attributes, displays videos of content likely to interest many passengers on the display device 22, thereby increasing the appeal of content for content providers. Furthermore, many passengers can view content that interests them.

[0085] (B-2) The processing unit 40 may exclude certain content from the selection of multiple content if there are passengers with specific attributes among the passengers. For example, if there are many female passengers but the proportion of male passengers is above a certain percentage, the processing unit 40 may exclude commercials for women's underwear from the list of content candidates to be displayed on the display device 22. Conversely, if the proportion of female passengers is above a certain percentage, the processing unit 40 may exclude commercials for men's underwear from the list of content candidates to be displayed on the display device 22.

[0086] (B-3) The processing device 40 should analyze the image not only when the train departs the station, but also for each piece of content, before the display of that content finishes, in order to determine the attributes of the passengers who are looking at the display device 22.

[0087] (B-4) The processing unit 40 may select content to display based on the attributes of passengers who are looking at the display device 22. Passengers looking at the display device 22 may be detected by analyzing the gaze of passengers captured in the image. The processing unit 40 may extract passenger faces from images captured by the camera 35 and detect passengers looking at the display device 22 based on the relationship between the position of the display device 22 and the orientation of the passengers' faces. Alternatively, the processing unit 40 may detect passengers looking at the display device 22 by applying known gaze detection techniques to images captured by the camera 35. The position of the display device 22 may be registered in the processing unit 40 in advance. In this way, for example, content that is likely to be of interest to passengers who are actually looking at the display device 22 can be displayed on the display device 22. As a result, the appeal effect of the content can be further enhanced.

[0088] Information on the number of passengers viewing content is valuable for content providers and creators, as it helps them develop new content. Content viewed by a large number of passengers is considered to be more engaging and appealing. This information can be expressed as, for example, the percentage of passengers viewing the content relative to the total number of passengers. By using a percentage rather than an absolute number, the comparison between information obtained when the number of passengers is high and information obtained when the number of passengers is low becomes more statistically significant.

[0089] [C. Content selection function based on passenger behavior] Next, referring to Figure 6, we will describe the content selection function based on passenger actions as one of the functions of the processing unit 40 and the display device 22.

[0090] Figure 6 is a flowchart of the content selection function based on passenger actions. The processing unit 40 periodically executes the process shown in Figure 6, for example, at regular intervals, for example, every 10 seconds. The processing unit 40 detects the actions of each passenger by analyzing the video images captured by the camera 35 (step SB1). The processing unit 40 determines whether the actions of each passenger fall under a predetermined pattern of actions (step SB2). The predetermined pattern of actions here refers to the pattern that occurs when a predetermined crime such as groping or pickpocketing is being committed. If the passenger's actions fall under a predetermined pattern of actions (step SB2; Y), the processing unit 40 issues a command to the display device 22 to select and display predetermined content (step SB3). The content here is crime prevention content, and this command includes the type of crime that is suspected to be currently being committed. Upon receiving this command, the display device 22 displays the content. The display device 22 displays crime prevention content according to the type of crime. For example, if an action corresponding to groping is detected, text information such as "Groping is a crime" and "Beware of gropers" or an image to deter groping is displayed. Furthermore, if an action that constitutes pickpocketing is detected, the device will display text information such as "Beware of pickpockets" or images designed to deter pickpocketing. The crime prevention content displayed on the display device 22 will motivate criminals to stop their criminal activities. In addition, video footage of criminal activities can serve as effective clues for identifying suspects in criminal investigations.

[0091] [D. Variations of the content selection function] (D-1) The camera 35 may have a pan-tilt function and a zoom function. When the processing unit 40 detects an action of a predetermined pattern, it may pan-tilt and zoom to the location where the action was detected and display the image of that location on the display device 22. For example, if the camera 35 detects a criminal act such as molestation, pickpocketing, or theft, it may pan-tilt and zoom to the location where the criminal act is suspected to be taking place and display the image of that location on the display device 22. Alternatively, instead of mechanically panning and tilting, the image of the location where the criminal act is suspected to be taking place may be enlarged and displayed on the display device 22.

[0092] (D-2) The processing unit 40 may select content based on the image captured by the camera 35 as well as the audio information picked up by the microphone (voice detection means) placed inside the vehicle. For example, based on keywords in the conversation picked up by the microphone, the processing unit may display content associated with those keywords on the display device 22.

[0093] (D-3) The processing unit 40 may have a function to select the next content to display based on the attributes of the passenger who is currently looking at the display unit 22. For example, the processing unit 40 may display on the display unit 22 content that is likely to be of interest to the passenger who is currently looking at the display unit 22. The passenger's attributes and the types of content that passengers with those attributes are likely to be of interest should be stored in the processing unit 40 in advance. By displaying content on the display unit 22 using this function, the likelihood of the passenger who is currently viewing content continuing to view the next content increases, thereby enhancing the appeal of the content to passengers. If there are multiple passengers viewing content, the processing unit 40 may select the next content to display by majority vote.

[0094] [E. Functions for acquiring information that forms the basis for evaluating content, and functions for evaluating content.] Next, referring to Figure 7, we will explain the function for acquiring basic information (hereinafter referred to as "evaluation information") for evaluating the content displayed on the display device 22. In this example, the content is a commercial, but other types of content may also be used.

[0095] Figure 7 is a flowchart showing the functions for acquiring evaluation information on content displayed on the display device 22 and for evaluating the content. The processing device 40 periodically executes the processes shown in Figure 7, for example, at regular intervals, such as every 10 seconds. The processing device 40 detects passengers' gaze by analyzing images captured by the camera 35 and counts the number of passengers looking at the display device 22 (step SC1). Step SC1 is an example of the process for acquiring content evaluation information. The processing device 40 associates the type of content that was displayed on the display device 22 when the original image for image analysis was taken with information on the number of passengers looking at the display device 22 and records it on the storage device 43 or an SD card installed in the reader / writer 42 (step SC2). As information on the number of passengers, for example, it may be the percentage of passengers looking at the display device 22 relative to the total number of passengers.

[0096] The processing unit 40 evaluates the content based on the type of content and the number of passengers who were viewing the display device 22 (step SC3). The content evaluation may be performed by, for example, calculating a parameter (hereinafter referred to as "evaluation value") that serves as an indicator of the degree of interest or attention of passengers in the vehicle to the content, based on the evaluation information. The evaluation value may be, for example, the viewing rate of the content. Note that the term "viewing rate" will be used here even if the video content does not include audio information. For example, the average or maximum viewing rate during the period in which the content being evaluated is displayed may be adopted as the evaluation value.

[0097] The content evaluation value may be the ratio of passengers in the vehicle who watched the content for a predetermined amount of time or longer. Alternatively, the content evaluation value may be the ratio of passengers who watched the content for a predetermined amount of time or longer among the passengers captured in the image that forms the basis of the evaluation. Or, the content evaluation value may be the product of the number of passengers watching the content and the time they watched it (hereinafter referred to as "viewing time"). For example, if two people watch the content for 15 seconds and three people watch it for 30 seconds, the evaluation value is 2 × 15 + 3 × 30 = 120 person-seconds. Alternatively, the content evaluation value may be the ratio of the actual viewing time to the viewing time that would be assumed if all passengers captured in the image that forms the basis of the evaluation watched the content for the entire duration that the content was displayed.

[0098] As described above, the processing unit 40 can be used as a train-based viewership checker. If a large number of passengers were looking at the display device 22, the content being displayed at that time can be given a high rating. This rating result provides useful information for creating new content.

[0099] [F. Variations of functions for acquiring information that forms the basis for evaluating content and functions for evaluating content] (F-1) The processing unit 40 may acquire and record, as evaluation information, information on which part of the display device 22 passengers are looking at, instead of or in combination with the number of passengers looking at the display device 22. The processing unit 40 may have a function to record, for example, images captured by the camera 35, information identifying the content displayed on the display device 22, and evaluation information in association. In this way, the content can be evaluated in more detail based on the recorded information and used to create content with a high appeal. The information that forms the basis for evaluating the content may include, for example, the number of passengers looking at the content, their attributes, information on which part of the video of the content they are looking at, and the location information of the vehicle when they are looking at the content.

[0100] (F-2) The processing unit 40 may store information identifying the content and evaluation information on an SD card. The content can be evaluated in more detail by an external device such as a personal computer that reads the contents recorded on the SD card.

[0101] The processing unit 40 can detect the passenger's gaze and determine the area the passenger is looking at based on the position of the display screen of the display device 22 and its positional relationship to the passenger's gaze. From this evaluation information, it is possible to obtain information on which parts of the video content displayed on the display device 22 are being viewed by the most passengers. This information allows for a more detailed evaluation of the content. This evaluation result provides useful information for creating highly appealing content. Thus, the evaluation basis should include, for example, the number of passengers viewing the content, the duration of viewing the content, and the attributes of the passengers viewing the content.

[0102] (F-3) The processing unit 40 should acquire and record, as evaluation information, information regarding the number of passengers looking at the display device 22, information regarding which part of the display device 22 they are looking at, and information identifying the passengers who are looking at the display device 22. From this information, it is possible to obtain information on what kind of passengers are looking at what kind of content. This information can be used to create more appealing content.

[0103] (F-4) The processing unit 40 should have a function to display the temporal change in the number of passengers looking at the display device 22, along with the change in the content displayed on the display device 22 and the change in the vehicle's position. It is even better if the processing unit 40 has a function to display the temporal change in the number of passengers looking at each position in the video of the content. This information allows for analysis of the reason why passengers stopped looking at the content. For example, it is possible to determine whether the reason for stopping the viewing of the content was that the vehicle was approaching a station and passengers were preparing to disembark, or that the content had changed. If the number of passengers viewing the content decreases without any external factors, it means that the number of passengers interested in that content has decreased.

[0104] (F-5) The processing device 40 may have a function to record the position of passengers in the vehicle who are looking at the display device 22. Based on the recorded information, it is possible to know which passengers in which positions in the vehicle are most likely to be viewing the content. This information is useful for considering a preferred installation location for the display device 22 that displays the content.

[0105] (F-6) The processing unit 40 should have a function to acquire information necessary to analyze at what time of day, at what location (e.g., between stations) the train was traveling, in which carriage, and what kind of passengers with what attributes were frequently viewing which content (e.g., commercials). Based on these analysis results, the processing unit 40 will be able to optimize the content to be displayed.

[0106] (F-7) The processing device 40 should have a function to acquire information that allows for similar analysis to be performed on passengers looking at the hanging advertisements inside the train, in addition to the content displayed on the display device 22.

[0107] For example, a processing unit (e.g., a computer) that displays advertisements on the display device 22 is connected to the processing unit of the camera device 23 so as to be able to communicate with each other. The processing unit of the display device 22 outputs a signal indicating advertisement identification information (hereinafter referred to as "advertisement ID") to the processing unit of the camera device 23 when the display of each advertisement begins, and the processing unit of the camera device 23 records the start and end times of the display of the advertisement for that advertisement ID in association with the data of the image captured by the camera 35 (hereinafter referred to as "camera image" in this modified example). For example, it is possible to record the advertisement ID in association with the camera image, assuming that the video content of a certain advertisement ID is being displayed on the display device 22 from the time an advertisement ID is received until the next advertisement ID is received. Alternatively, for example, the image signal of the video content being displayed on the display device 22 is input to the processing unit of the camera device 23, and this processing unit records the video data of the video content together with the camera image data.

[0108] As a method for recording video data of video content displayed on the display device 22 together with the camera image, it is preferable to record the camera image and video content with time information attached so that they can be played back in sync with the camera image. Alternatively, as a method for recording video data of video content displayed on the display device 22 together with the camera image, it is preferable to record the video content displayed on the display device 22 superimposed on the camera image (for example, like a wipe).

[0109] The processing unit 40 should record images in a way that allows for later extraction, recording, and viewing of only those images taken during the period when passengers were viewing advertisements. For example, it would be good to record information that identifies images taken during the period when passengers were viewing advertisements, such as a flag. When displaying information such as "Next stop is XX station" instead of an advertisement, it would be good to record information indicating that information (also called informational information) was being displayed, preferably a unique information ID for each piece of information, instead of the advertisement ID. It would also be good to record information that allows for the identification of the specific location in which car of which train, between which stations, etc.

[0110] (F-8) In addition to evaluation information, the processing device 40 should record the video content displayed on the display device 22 at that time as image data. This information makes it easy to understand what many passengers are looking at in the video content.

[0111] (F-9) The processing device 40 may have a function to detect the position in the display device 22 where the passenger is looking, and to record the video of the content displayed on the display device 22 in association with that position.

[0112] In this way, based on the recorded information, it is possible to know where passengers are directing their gaze within the video content. For example, it is possible to obtain information such as whether they are looking at the products in the video content or the talent appearing in it. Based on this information, a more detailed evaluation of the content can be made. For example, this information can be used as useful information for creating content that is highly appealing to products. It is advisable to record the necessary information so that it is easy to analyze from the recorded information which specific points of focus are within the video content, and which passengers, in which positions within the video content, were looking at which points in the video content at what time.

[0113] The device that reads the recorded information, for example, a personal computer with a viewer installed, should have a function to simultaneously display the video content on its screen along with various information that was present when the video was being displayed on the in-vehicle display device 22, such as the number of passengers watching the content, the time, the location where the vehicle is traveling, and the attributes of the passengers watching the content. This makes it possible to intuitively understand what kind of content is being watched by which types of passengers.

[0114] (F-10) The device that reads the recorded information should display passengers who are looking at video content such as advertisements in a different display manner than other passengers. For example, it should detect the position of the face of a passenger who is watching video content and display that passenger's face surrounded by a red circle. If multiple people are watching video content, it should display the faces of each person surrounded by circles of different colors.

[0115] The device reading the recorded information should also clearly indicate the location of the gaze (for example, by adding a red circle) for video content that the gaze is directed towards. Regarding the use of a red circle as a method of indication, if multiple people are watching, the circle displayed at the location of the gaze within the video content and the circle surrounding the face of the passenger looking at that location should be the same color. This allows for easy analysis of where the passenger's attention was directed within the video content, such as whether they were looking at the talent or the product. If multiple circles of different colors are displayed near the product, it can be determined that the product received the most attention. If multiple circles of different colors are displayed near the talent, it can be determined that the talent received the most attention. Furthermore, it allows for easy analysis of the temporal changes in the location of attention within the video content. It is also advisable to display content targeted at passengers who are currently looking at the video content.

[0116] (F-11) The processing unit 40 should have a function to record information necessary for analyzing the relationship between passenger boarding and alighting status and video content. Based on the information recorded by this function, the relationship between passenger boarding and alighting status, such as opening and closing of doors, and video content can be analyzed.

[0117] (F-12) The processing unit 40 may refrain from performing at least one of the following processes during boarding and alighting: the process of recording the advertising ID along with the camera image, and the process of detecting the passenger's gaze. Alternatively, the processing unit 40 may perform at least one of these processes and record information to identify that it is a period of boarding and alighting. For example, the processing unit 40 may determine that it is a period of boarding and alighting based on the detection of door opening and closing, station arrival and departure announcements, and the detection that information indicating that the doors are open is displayed on the display device 22. The processing unit 40 may record the period during which it has determined that boarding and alighting is taking place, along with the advertising ID, in a way that makes it identifiable.

[0118] (F-13) The processing unit 40 should have a function to display the video content and images of the vehicle interior side by side on a display screen, and to display them in a way that allows for the correspondence between the vehicle's passengers and their positions in the video content they are viewing. For example, the position of the vehicle's passengers' faces and their positions in the video content they are viewing should be marked with the same color. This makes it possible to intuitively understand which passenger is looking at which part of the video content.

[0119] The processing unit 40 should display markers in the video content at the points that multiple passengers are looking at. Since the density of markers will be higher in areas that many passengers are looking at, it is possible to know which parts of the video content attracted the passengers' interest the most.

[0120] (F-14) The processing unit 40 should have a function to detect which display screen a passenger is looking at and record the detection result when the display device 22 has multiple display screens. Based on the information recorded by this function, it is possible to easily analyze which of the multiple display screens a passenger is looking at. If passengers within the range captured by one camera 35 can see multiple display screens, the same processing should be performed for each of the multiple display screens as when there is only one display screen. For example, if there are two display screens above one door and one camera 35 is placed next to them, the camera 35 should record information indicating which display screen each passenger was looking at.

[0121] The processing unit 40 should have a function to detect which display screen a passenger is looking at when a single piece of content is displayed across multiple display screens. The information obtained by this function will be useful in finding the most effective display method when displaying a single piece of content across multiple screens.

[0122] (F-15) The processing unit 40 may also acquire, as evaluation information, the current time the content is being displayed, the location of the vehicle, the time and station information of the passengers who boarded the vehicle, and at least one of the attributes of the passengers viewing the content. By including this information as the basis for evaluating the content, it becomes possible to perform a more detailed evaluation of the content.

[0123] The processing unit 40 can obtain information such as when a passenger started looking at the display device 22. For example, it can obtain information such as how many minutes have passed since the passenger boarded the train, or how many stations have passed since boarding, before the passenger started looking at the content. For example, it can obtain information such as the passenger was using their smartphone immediately after boarding, but started looking at the content after passing two stations. It can also obtain information such as what kind of passengers were looking at what kind of content. The attributes of passengers looking at the content should include, for example, gender, age, and facial expression (for example, whether they are smiling or angry). This information should be obtained, for example, at regular intervals (for example, 1 second). This information does not necessarily need to be obtained in real time, so it is fine to record the images taken by the camera 35 as image data on a recording medium, and then read the image data recorded on the recording medium to obtain this information.

[0124] Furthermore, since the processing unit 40 has a function to detect the number of people in the vehicle and the passengers' gazes based on the images captured by the camera 35, it is possible to obtain information on which part of the display screen of the display unit 22 a passenger is looking at, based on the direction of the gaze of passengers looking at the display unit 22 and the location of the display screen of the display unit 22. Based on the video content that was displayed on the display unit 22 at the time the images captured by the camera 35 were acquired and the position where the passenger was looking on the display screen, it is possible to know which part of the video content the passenger is looking at. For example, it is possible to obtain information on whether the passenger is looking at a product in the video content or at a talent appearing in it. Based on this information, a more detailed evaluation of the content can be performed. For example, this information can be used as useful information for creating content that is highly appealing to products.

[0125] [G. Function to record after applying image processing] Next, referring to Figure 8, we will explain the function of processing and recording images. Figure 8 is a flowchart illustrating the function of processing and recording images. The processing unit 40 repeatedly performs the process shown in Figure 8 from the time the power is turned on until it is turned off. The processing unit 40 acquires the video captured by the camera 35 (step SD1). Subsequently, the processing unit 40 processes a specified area within the video and records the resulting video onto an SD card inserted in the reader / writer 42.

[0126] A predetermined area to be processed may be stored in the storage device 43 beforehand. The processing device 40 reads information from the storage device 43 that identifies the area to be processed and applies image processing to the video (step SD2). If the area in which a passenger's face is visible is specified as the area to be processed, the processing device 40 detects the passenger's face in the video and blurs that part of the face. If a specific area within the video screen is specified as the area to be processed, the processing device 40 clips the specified specific area.

[0127] By blurring passengers' faces in images, the risk of privacy violations is reduced. Furthermore, the data size of image data can be reduced by clipping specific designated areas. For example, areas such as the ceiling that do not contain passengers can be designated as clipping areas. This allows for data reduction while still retaining the necessary information about the passengers. The image processing applied to passengers' faces should be such that it obstructs the visibility or recognition of their faces; this could include blurring, clipping, or even reducing the resolution.

[0128] Images captured by the in-vehicle camera 35 should be stored securely to prevent public access, as they may infringe on individual privacy. By using a function that applies image processing to the recording medium to prevent the visibility or recognition of a designated area, and by applying image processing that makes it impossible to identify individuals, information that is highly likely to infringe on individual privacy will not be recorded on the recording medium.

[0129] The "specified area" could be, for example, the area where a passenger's face is visible. Image processing that obstructs the visibility or recognition of the specified area could be employed, such as clipping, blurring, or reducing the resolution. Alternatively, the face could be recorded in an abstracted form so that the passenger cannot be identified, but their gender and age can be determined.

[0130] The "specified area" should be an area where passengers are not visible. For example, the ceiling or walls of the vehicle can be specified. By applying processes such as clipping, blurring, or reducing resolution to this area, the amount of image data can be reduced without reducing the amount of information about the passengers.

[0131] Cameras are often installed at a height higher than the average passenger's eye level. In such cases, if the image contains parts that would cause significant embarrassment or anxiety to passengers, such as underwear or body parts normally concealed by clothing, it is advisable to specify those areas and apply the aforementioned image processing to them.

[0132] [H. Function to process images and output image data] Next, referring to Figures 9A and 9B, we will explain the function that performs image processing and outputs image data.

[0133] Figure 9A is a flowchart illustrating the function for recording video footage captured by camera 35. The processing unit 40 continuously performs the process shown in Figure 9A from the time the power is turned on until it is turned off. The processing unit 40 acquires video footage captured by camera 35 (step SE1). Subsequently, the acquired video footage is recorded onto an SD card installed in the reader / writer 42.

[0134] Figure 9B is a flowchart showing the function for outputting image data. In step SE2 (Figure 9A), an information terminal such as a personal computer, which has an SD card on which video footage is recorded, executes the process shown in Figure 9B. The processing unit of the information terminal has functions to receive an image data output command, a command to receive whether or not to apply image processing to the video footage to be output, and a command to receive a command to identify the target to which image processing should be applied. When the operator inputs an image data output command to the information terminal, the processing unit of the information terminal executes the process shown in Figure 9B. The hardware configuration of the processing unit of the information terminal is not shown in the diagram, but it can be the same as that of a typical personal computer. That is, the processing unit of the information terminal has an arithmetic processing unit (exemplified by a CPU), memory, an operating device, an interface for external storage media, a communication module, and an auxiliary storage device. The memory is used as a work area for arithmetic processing. The operating device receives user input. For example, a keyboard or pointing device can be used as the operating device. The interface for external storage media is for inserting an external storage medium such as an SD card. The communication module is for communicating with external devices (e.g., wired or wireless). For auxiliary storage, a hard disk drive, for example, is used to store programs, viewers, and other data. The arithmetic processing unit reads the program and other data into memory and performs various calculations according to the program.

[0135] The processing unit of the information terminal determines, based on the input command, whether or not to apply image processing to the image data to be output (step SE5). If image processing is to be applied (step SE5; Y), the image data is read from the SD card and the image processing is applied (step SE6). The predetermined image processing applied to the passenger's face should be such that it obstructs the visibility or recognition of the passenger's face. After that, the image data with the image processing applied is output (step SE7). If no image processing is to be applied (step SE5; N), the image data read from the SD card is output as is (step SE7). The output of the image data is performed by writing to another SD card, transmitting to other devices via a communication network such as a wired LAN or wireless LAN, etc.

[0136] Image data processed by targeting only the areas containing passengers' faces poses less risk of infringing on privacy. Unprocessed image data can provide valuable information for solving incidents such as crimes that occur on the train or disputes between passengers. For example, facial recognition technology can be used to search for wanted criminals or suspects.

[0137] [I. Modified versions of the function that processes images and outputs image data] (I-1) The processing unit 40 may include a function to record images captured by the camera 35 into a recording device, and a function to read the images recorded in the recording device and selectively output image data in which a predetermined image processing has been applied to the passenger's face, or image data in which the predetermined image processing has not been applied.

[0138] By recording images captured by camera 35 to a recording device, the recorded images can be effectively used by public investigative agencies such as the police to solve crimes and other incidents that occur inside the vehicle. By outputting image data with predetermined image processing applied to passengers' faces, images that are less likely to infringe on individual privacy can be provided. Image data without image processing can be useful information for solving incidents that occur inside the vehicle. For example, it can be useful information for solving crimes such as groping, pickpocketing, and theft that occur inside the vehicle.

[0139] The prescribed image processing applied to the passenger's face may be such that it obstructs the visibility or recognition of the passenger's face, for example, by processing the image so that an individual cannot be identified from the face portion. For example, this could involve clipping, blurring, or reducing the resolution of the image.

[0140] The image data output function should include, in addition to outputting image data from an image output terminal, outputting image data via wireless LAN, and writing image data to external storage media such as external storage media. When outputting image data from an image output terminal, the above-mentioned predetermined image processing should be applied. When outputting image data via wireless LAN, it should be possible to specify whether or not to apply the predetermined image processing depending on the attributes of the recipient. For example, if the recipient is an investigative agency such as the police, where the confidentiality of information is considered to be maintained, the predetermined image processing should not be applied. The processing device should have a function to register a list of recipients to whom image processing should not be applied.

[0141] With respect to (I-2) and (I-1), it is preferable that the processing unit 40 has a function to specify the object or area within the image to be processed. For example, it is preferable that the object to be processed be able to specify the faces of passengers, etc. As for the area to be processed, it is preferable that the user be able to specify a specific area or the entire area within the image.

[0142] (I-3) When the processing unit 40 records image data to an external storage medium without performing image processing, it is preferable to record the image data in a format that requires a password to read it. Alternatively, when recording image data to an external storage medium without performing image processing, it is preferable to format the external storage medium in a special format so that general reading applications cannot read the image data on the external storage medium. This suppresses the occurrence of situations where image data is read by unspecified persons.

[0143] The processing unit 40 should be configured to automatically perform predetermined image processing when recording image data without a password to an external storage medium formatted in a common format. This helps protect individual privacy.

[0144] For recording images captured by camera 35, a recording device such as an external storage medium and its reader / writer may be used. A viewer for playing back images recorded on the external storage medium on a terminal such as a personal computer may also be provided.

[0145] [J. Passenger movement path detection function] Next, I will explain the passenger movement path detection function. The information terminal's processing unit has a function to allow the user to specify a particular passenger from among the passengers captured in images recorded on the SD card. When the information terminal's processing unit detects that a specific passenger has been specified, it performs image analysis on the image data read from the SD card to search for the specified passenger. Upon detecting the specific passenger, it advances the timeline of the video to detect the passenger's movement path within and between vehicles. The detected movement path is displayed as an image on the information terminal's display screen. The processing unit further has a function to detect the movement path within the vehicle from boarding to alighting for each passenger who boarded the vehicle, based on the images captured by the camera 35, and to record information on the boarding point, the detection result of the movement path within the vehicle, and the alighting point.

[0146] This function allows us to obtain information on which stations a specific passenger boarded a train, how they moved within and between train cars, where they stood or sat, and at which stations they alighted. It is preferable to place cameras 35 in each car of a train, including multiple cars, and record the above information for each car. Furthermore, it is preferable for the processing unit to have a function that displays the images captured by the cameras 35 and allows the user to identify a specific passenger from the image. Once a specific passenger is identified, it is preferable for the processing unit to have a function that detects and displays the passenger's movement path within and between train cars from the time they boarded to the time they alighted. This information can be effectively used in planning the number of train cars, train operation plans, and station platform designs. Moreover, in cases where there is a large variation in congestion rates between cars, the passenger's movement path within the car can be used as useful information for planning strategies to guide passengers to relatively empty cars.

[0147] [K. A function that displays images captured by the camera 35 on the in-vehicle display device 22.] Next, we will explain the function of displaying images captured by the camera 35 on the in-vehicle display device 22. The processing unit 40 has a function to display on the display device 22 an image captured by the camera 35 or an image processed therefrom, instead of or in combination with the process of displaying content on the display device 22. The processing unit 40 also has a function to prompt and allow the operator to input a command (e.g., a command) specifying whether to display the image captured by the camera 35 as is, or to display a processed image, and if processed, what kind of processing to apply.

[0148] If the processing unit 40 receives a command to display the image captured by the camera 35 as is, it displays the image captured by the camera 35 on the display device 22 in near real-time. If the processing unit 40 receives a command to process and display the image captured by the camera 35, it applies the commanded processing to the image captured by the camera 35 and displays it on the display device 22 in near real-time.

[0149] Train passengers may feel anxious about how they are being filmed by the cameras 35 inside the train. Displaying the actual images on the display device 22 inside the train allows passengers to see how they are being filmed, thus alleviating their anxiety. Displaying processed images of the actual images provides various additional effects compared to displaying them as they are.

[0150] The display device 22 that displays the images captured by camera 35 should be located near camera 35. In particular, it should be positioned so that passengers within the range of camera 35 can see the display device 22.

[0151] As part of the image processing, it would be beneficial to replace areas other than the passengers (background) with other images instead of images of the train interior. In particular, it would be good to display situations on the screen of the display device 22 that would not occur in reality. For example, if an image of a pond with carp swimming in it, taken from above, is used as the background, a situation where passengers are standing on this pond will be virtually recreated. This is expected to alleviate the passengers' anxiety about being photographed. Furthermore, if there are children among the passengers, it would be good to display images that children would find interesting and enjoyable next to the children on the screen of the display device 22. For example, an animated image of a popular character would be good to display next to the children on the screen. This would make the children feel happy.

[0152] It is preferable to display images captured by the camera 35 inside the train and pre-prepared video content alternately, for example, at a fixed interval. This allows both images and video content from inside the train to be displayed on a single display device 22.

[0153] As a variation, a display device 22 corresponding to each vehicle could be installed in the driver's cab, allowing the crew to check the passenger situation in the vehicle in real time. The crew could easily grasp the degree of congestion in each vehicle. By understanding the degree of congestion, they could make announcements to guide passengers to relatively empty vehicles.

[0154] If an item is left behind on the luggage rack or other areas inside the train, the crew can easily check for the item's presence by viewing the image. This reduces the time required to retrieve lost items at stations.

[0155] [L. Anti-theft function for luggage inside electric train cars] Next, with reference to Figures 10A and 10B, the luggage theft prevention function inside the train car will be explained. The luggage theft prevention function is a function that sets the locking mechanism to either the locked or unlocked state based on the characteristics of passengers who visit the luggage area as captured in images taken by camera 35.

[0156] Figures 10A and 10B are perspective views of the locking mechanism in the luggage compartment inside the train car in its unlocked and locked states, respectively. The locking mechanism includes two bars 55 and 56. In the unlocked state (Figure 10A), a gap is provided between the two bars 55 and 56. This allows one of the bars 55 to pass through the handle 51 of luggage 50, such as a suitcase. In the locked state (Figure 10B), a closed loop is formed by the two bars 55 and 56, preventing the handle 51 of the luggage 50 from being removed from the bars 55 and 56. The luggage compartment is located within the field of view of the camera 35 inside the train car. Therefore, when the owner of the luggage 50 is operating the locking mechanism, the owner of the luggage is photographed by the camera 35.

[0157] When an owner intends to place luggage 50 in the luggage area, they pass bar 55 through the handle 51 of luggage 50, and then operate bars 55 and 56 to create a closed loop. When the locking mechanism detects that bars 55 and 56 are in a closed loop, it transitions to the locked state. The processing unit 40 has a function to detect when the locking mechanism has transitioned to the locked state.

[0158] When the processing unit 40 detects that the locking mechanism is locked, it analyzes the image captured by the camera 35 to detect the passenger who placed the luggage 50 in the luggage area, identifies the passenger using facial recognition technology, and stores the information relating the storage location where the luggage 50 was placed with the information identifying the passenger (owner) who placed the luggage.

[0159] The processing unit 40 analyzes images captured by the camera 35 and performs facial recognition of passengers near the luggage storage area. When the processing unit 40 detects that a passenger associated with that storage area is near the storage area, it unlocks the locking mechanism of that storage area. This allows the luggage owner to retrieve their luggage from the storage area.

[0160] If the owner of luggage 50 is not detected near the storage area, the processing unit 40 keeps the locking mechanism locked. This deters luggage theft. Passengers who leave their luggage in the luggage area do not need to carry the key to the locking mechanism, so they do not have to worry about losing the key. In addition, passengers do not need to operate the locking mechanism with a key, which increases convenience.

[0161] Figures 11A and 11B are front and top views, respectively, of other locking mechanisms installed in the luggage compartment of an electric train. The luggage compartment is divided into multiple storage spaces 60 by partition plates 61. A swing door 62 is provided for each storage space 60. In the unlocked state, the swing door 62 can be opened and closed freely, and in the locked state, the swing door 62 remains closed and cannot be opened. A lock button 63 is provided for each storage space 60.

[0162] The owner of luggage 50 opens the swing door 62, stores the luggage 50 in the storage area 60, and presses the lock button 63. The processing unit 40 has a function to detect when the lock button 63 is pressed. Upon detecting that the lock button 63 has been pressed, the processing unit 40 identifies the passenger standing in front of the storage area 60 using facial recognition technology. The processing unit 40 associates and stores the storage area where the luggage 50 is placed with the information identifying the identified passenger (owner). After locking, if the processing unit 40 detects that the luggage owner is near the storage area, it unlocks the lock mechanism of the storage area. The same effect can be obtained using the lock mechanisms shown in Figures 11A and 11B as when using the lock mechanisms shown in Figures 10A and 10B.

[0163] If the package is not removed from the storage location after a certain period of time has elapsed since the locking mechanism was unlocked, the processing unit 40 should have a function to re-lock it. If the package owner simply passes by the storage location and the device detects the owner and unlocks the device, the processing unit 40 will re-lock it, thus ensuring the safe storage of the package.

[0164] This feature makes it less likely for luggage to be stolen. The luggage rack can be located anywhere in the train car, either in the passenger compartment or on the deck. The luggage rack should be located in a place where passengers cannot see it when they are seated in their seats. Even if luggage is placed in a location out of the passengers' sight, the theft prevention effect is still achieved, further enhancing the effectiveness of the above feature. In addition, by placing the luggage rack away from the seats, the layout of seats, luggage racks, doorways, etc. inside the train car becomes more flexible.

[0165] Passengers who leave their luggage in the luggage compartment do not need to carry the key to the locking mechanism, eliminating the worry of losing the key. Furthermore, since passengers do not need to operate the locking mechanism with a key, convenience is enhanced.

[0166] [A modified example of a luggage theft prevention function inside a train car] (M-1) The luggage storage area may be a space within the passenger compartment. The luggage storage area may also be the space between the rearmost seats of a Shinkansen or other train and the wall behind them. The processing unit 40 may perform the above processing on passengers using specific seats within the train as the owners of the luggage. The processing unit 40 may identify passengers using the rearmost seats of the train as the owners of the luggage.

[0167] (M-2) The configuration of the locking mechanism is not limited to those described above, but it is best not to use a mechanism that stores luggage in a sealed space like a locker or safe, preventing third parties from taking the luggage. For example, it is best to adopt a locking mechanism that allows the owner of the luggage or a third party to see the luggage placed in the luggage storage area, but makes it difficult to easily remove the luggage from the storage area. For example, a bar that can be passed through the handle of luggage (e.g., a suitcase) can be installed in the luggage storage area. The locking mechanism should be such that when the bar is passed through the luggage handle and locked, the handle cannot be removed from the bar. Alternatively, a swing door can be installed between the passageway and the luggage storage area, and a mechanism should be adopted as a locking mechanism that prevents the swing door from opening when locked. The top of the storage area is open, and it is possible to take out luggage by passing over the top of the swing door, but it is expected to have an effect in deterring luggage theft.

[0168] (M-3) When the luggage owner places their luggage in the luggage storage area and locks the locking mechanism, the processing unit 40 detects that it has been locked and stores the association between the luggage and its owner based on the image of the passenger standing in front of the luggage in the luggage storage area at that time. For example, facial recognition technology may be used to identify the luggage owner. When the luggage is locked, if a person stands in front of the luggage storage area, the processing unit 40 determines, based on the person's image, whether the passenger standing in front of the storage area is the owner of the luggage being stored, and if it is the owner of the luggage being stored, it should unlock the locking mechanism.

[0169] (M-4) The processing unit 40 may have a function to detect passengers standing in front of the storage area who are looking at a specific location, such as the location where the camera 35 is installed, and to determine whether or not that passenger is the owner of the luggage. Alternatively, the processing unit 40 may have a function to unlock the locking mechanism when it detects, by analyzing the image captured by the camera 35, that the owner standing in front of the luggage storage area has started to retrieve their luggage. This function prevents the luggage from being accidentally unlocked when the owner passes by the storage area for purposes other than retrieving their luggage.

[0170] (M-5) The processing device 40 is even better if it has an alarm device installed in the storage location that notifies the owner of the luggage when the locking mechanism switches from the locked state to the unlocked state and when it switches from the unlocked state to the locked state. The alarm device may be one that allows the owner of the luggage to notice the change in state visually or aurally, such as an LED or other light-emitting device, or a speaker or other sound-emitting device. This allows the owner of the luggage to confirm that the locking mechanism is locked when they place the luggage in the storage location. They can also confirm that the locking mechanism is unlocked before retrieving the luggage.

[0171] (M-6) The processing device 40 should have a function to relock the package if the package is not removed from the storage location after a certain period of time has elapsed since the locking mechanism was unlocked. If the package owner passes by the storage location and the device detects the owner and unlocks the package, the package can be safely stored by re-unlocking it.

[0172] [N. Function to display content on station display devices] Next, with reference to Figures 12A and 12B, we will explain the function of displaying content on the station's display device.

[0173] Figures 12A and 12B are schematic diagrams showing a train 70 on the tracks, a station display device 72 installed on a station platform 71, and a line of passengers 75. The train 70 includes six coupled cars. The train 70 is operated so that it passes through multiple stations in sequence, with multiple cars stopping at the designated stopping positions on platform 71 at each of the stations.

[0174] A station display device 72 is installed at each stopping position of the doors of each train car on the station platform 71. Passengers lined up at the door stopping positions can view the video content displayed on these station display devices 72. The processing unit 73 of the station display device 72 selects the content to be displayed on the station display device 72 at each stopping position.

[0175] Each car of the train 70 is equipped with a camera 23. The processing unit of the camera 23 analyzes images captured by the camera 35 to count the number of passengers and calculate the congestion rate for each car. The processing unit of the camera 23 transmits the calculated congestion rate information for each car to a processing unit 73 installed at the station. This transmission uses public lines, dedicated lines, etc., including wireless transmission lines. The congestion rate is divided into four stages, for example, "empty," "normal," "crowded," and "full."

[0176] The station's processing unit 73 has the function of selecting content from multiple contents to be displayed on the station's display device 72, which is positioned corresponding to the stopping position, based on the received information indicating the congestion rate for each train car. The processing unit 73 displays content that will make passengers want to move from the stopping position of a relatively crowded train car to the stopping position of a relatively empty train car, in other words, content that is likely to attract passengers' interest.

[0177] Figure 12A shows an example of a line of passengers 75 before the selected content is displayed, and Figure 12B shows an example of a line of passengers 75 when the selected content is displayed. As passengers 75 move to the stopping position where content of interest is displayed, the line of passengers 75 lining up at the stopping position of a relatively empty car becomes longer. This prevents situations where, for example, only certain cars near escalators are crowded while the first and last cars are empty, thus equalizing the congestion rate between cars.

[0178] As a variation, the station's processing device 73 may display the congestion level of the train in question on the station's display device 72 at each stopping position, as well as the congestion levels of other trains.

[0179] The camera 23 should ideally have a function to transfer information representing the level of congestion for each train car to a server. The server should ideally have a function that allows passengers to access information representing the level of congestion in real time using mobile information terminals such as smartphones. Furthermore, the server should ideally have a function to provide information representing past levels of congestion. Train users can use past levels of congestion as a reference to decide on their commuting time and which train car to use.

[0180] By appropriately selecting the content to be displayed on the station display device 72, it becomes possible to guide passengers to stopping positions corresponding to relatively empty cars. This makes it possible to equalize the congestion rate. Examples of trains containing multiple cars include electric trains, monorails, and maglev trains.

[0181] For example, each time a train departs from a station, the processing unit can calculate the congestion rate based on images captured by cameras 35 located inside each train car. The processing unit installed in the train car calculates the congestion rate and transmits the result wirelessly to a station display device 72 installed at the next station. The processing unit in the station display device 72 then displays content corresponding to the received congestion rate. For example, the station display device 72 at each stopping position can display information such as the congestion rate and level of congestion of the train stopping at that position, using numbers, colors, graphics, etc.

[0182] It is advisable to guide passengers to the position where the next available train or other train will stop by displaying content that is likely to attract their interest on the station display device 72 positioned at the location where the next available train or other train will stop. For example, if there are many children among the passengers waiting for the next train or other train, it is advisable to use images of popular characters that are likely to attract children as content that is likely to attract their interest. The processing device should have a function that displays boring commercial videos on the station display device 72 at the location where a crowded train or other train will stop, and commercial videos featuring popular idols on the station display device 72 at the location where an available train or other train will stop.

[0183] The processing device should have a function to analyze the number of passengers, their attributes, whether or not they are carrying luggage, etc., based on images taken of passengers lined up on the station platform, and change the content displayed on the station display device 72 based on the analysis results. For example, based on the attributes of passengers lined up at the stopping position of a crowded train, the device should select content that passengers are likely to be interested in, and display a message on the station display device 72 at the stopping position of the crowded train that passengers are likely to be interested in content displayed on the station display device 72 at the stopping position of the XXth car. This can enhance the effect of guiding passengers to the stopping position of less crowded trains. Alternatively, the processing device should have a function to display a short, narrative video on the station display device 72 at the stopping position of a crowded train, and display a message such as, "The rest will be displayed on the station display device 72 at the stopping position of the XXth car." This can guide passengers from the stopping position of a crowded train to other stopping positions.

[0184] It would be beneficial to have a processing device that can display information on bus stops along a bus route. When a crowded bus stops at a bus stop and the next bus is relatively empty, the processing device should inform passengers that the next bus is available and display content to motivate them to wait for the next bus. For example, the processing device should have a function to notify passengers waiting at the bus stop of the estimated time until the next bus arrives, the number of empty seats on the next bus, and the content that will be displayed on the display device inside the next bus. On the next bus, it would be beneficial to display content that is likely to interest the passengers waiting at the bus stop. For example, if there are many junior high and high school students waiting at the bus stop, it would be beneficial to display a message on the bus stop's display device such as "The next bus's display device will show practice questions and answers for an exam."

[0185] [O. Other configuration examples of the imaging device 23] (O-1) Figure 13 is a perspective view of the imaging device 23 (Figures 2A and 2B) with the cover removed, based on other configuration examples. The following explanation will focus on the differences between this device and the imaging device 23 shown in Figures 2A and 2B.

[0186] The housing of the imaging device 23 is provided with an opening 37. In this configuration example, the opening 37 is provided on the side of the recording unit body 34. The opening 37 is the part through which the external storage medium passes when attaching the external storage medium to the reader / writer 42 and when removing the external storage medium from the reader / writer 42. The SD card is attached to the reader / writer 42 (Figure 4), which is located inside the recording unit body 34, through the opening 37. A rotating cover 38 is rotatably attached to the recording unit body 34. By rotating the rotating cover 38, it is possible to achieve a state where the opening 37 is closed and a state where the opening 37 is open.

[0187] The direction in which the SD card is inserted and removed is parallel to the circuit board housed in the recording unit 34. This circuit board is approximately parallel to the wall when the camera 23 is mounted on the wall of the train. Therefore, the direction in which the SD card is inserted and removed is also parallel to the wall of the train. A label for identifying the card is attached to the SD card. The reader / writer 42 is positioned so that the label side faces forward when the SD card is removed from the recording unit 34.

[0188] The axis of rotation of the rotating lid 38 is parallel to the opening surface of the opening 37 (for example, perpendicular to the SD card when inserting or removing it), and the lid rotates in a plane parallel to the wall of the train. When the housing is attached to the wall of the vehicle, it is preferable that the axis of rotation of the rotating lid 38 be parallel to the wall. With this structure, the rotating lid 38 rotates in a plane perpendicular to the wall of the vehicle, eliminating the need to secure space next to the housing for rotating the rotating lid 38. The rotating lid 38 is fixed with Torx screws when the opening 37 is closed.

[0189] An image data output terminal 44 is located at the bottom of the recording unit body 34. The image data output terminal 44 is mounted on the support unit 31 in a position that allows the cable to be plugged in and unplugged from the front of the recording unit body 34.

[0190] An interlocking switch 46 is provided that turns the power of the imaging device 23 on and off in conjunction with the opening and closing of the rotating lid 38. The interlocking switch 46 is turned on when the rotating lid 38 is closed, and the interlocking switch 46 is turned off when the rotating lid 38 is open.

[0191] A power cable 45 extends upward from the recording unit body 34. The power cable 45 passes through an opening in the support unit 31 and an opening in the wall of the train car, and is connected to a power adapter mounted on the back of the train car wall. Power is supplied to the camera 23 from this power adapter. Depending on the train, the voltage and whether the power cables are DC or AC may differ from those of the train car. The power type supplied by the pre-wired power cables should be converted to the power type for the camera 23 using the power adapter. This allows the existing power cables to be used as the power source for the camera 23, eliminating the need to wire new power cables.

[0192] To remove the SD card from the recording device 23, the cover 32 (Figure 2A) must be removed from the support part 31, and then the rotating lid 38 of the recording unit body 34 must be rotated to open the opening 37. In this way, the cover 32 and the rotating lid 38 provide double theft prevention, thus increasing the effectiveness of preventing SD card theft. Because the lid that closes the opening 37 is rotating, it is possible to prevent the lid from falling off. The rotating lid 38 should be secured with special screws that cannot be removed with a common Phillips or flathead screwdriver, such as Torx screws. Doing so will further enhance the theft prevention effect.

[0193] By setting the relationship between the reader / writer 42 and the train's wall as described above, the label on the SD card can be easily viewed immediately after removing the SD card from the reader / writer 42.

[0194] Removing an SD card while it is powered on can cause the SD card to malfunction. Opening the rotating cover 38 turns off the interlocking switch 46, which in turn turns off the power to the camera 23, thus eliminating the risk of the SD card being removed from the reader / writer 42 while it is powered on.

[0195] (O-2) The rotation axis of the rotating lid 38 should be parallel to the opening surface of the opening 37, and the lid should be attached to the case via a hinge to open and close the rotating lid 38. A sliding lid can also be used instead of the rotating lid 38.

[0196] (O-3) The direction in which the SD card is inserted or removed should be parallel to the circuit board on which the electronic circuit components that realize the functions of the processing unit 40 are mounted. A label for identifying the SD card should be attached to the SD card. When the SD card is removed from the reader / writer 42 with the housing attached to the wall of the vehicle, the label side should face forward. This makes it easy to see the label immediately after removing the SD card from the reader / writer 42.

[0197] (O-4) The imaging device 23 and the processing device 40 should be equipped with a power supply for short-term operation, such as an electric double-layer capacitor. The processing device 40 should have a function that, when it detects that the interlocking switch 46 has been turned off, receives power from the power supply for short-term operation and performs termination processing on the SD card. In this way, it is possible to prevent situations in which the writing process to the SD card ends incompletely, making it impossible to write new data. The termination processing on the SD card will be completed in about 2 to 3 seconds, for example. Therefore, the power supply for short-term operation should have a capacity that allows the processing device 40 to operate for this amount of time.

[0198] (O-5) The support portion 31 and cover 32 of the housing may be made of non-combustible or flame-retardant material. For example, metal may be used for the support portion 31. Transparent resin may be used for the front part of the cover 32 of the camera lens, and metal, such as die-cast aluminum, may be used for the other parts. If a GPS antenna or a wireless LAN antenna is placed inside the housing, the parts near the antennas may be made of non-metallic material, such as resin. GPS information may be used to correct the time of the real-time clock of the processing unit 40.

[0199] (O-6) The circuit board on which the backup battery and the power supply for short-term operation are mounted should be removable from the main circuit board. This will allow the operator of the camera 23, such as a railway company, to easily replace the backup battery and the power supply for short-term operation. The circuit board on which the backup battery and the power supply for short-term operation are mounted should have an area to indicate information about when they were replaced, such as the year and month or year, month, and day. Such an area could be, for example, an area for attaching a sticker with the replacement date printed on it, or an area for writing handwritten text. By recording the date on which the circuit board on which the backup battery and the power supply for short-term operation are mounted was replaced, the next replacement date can be easily determined. In addition to the structure on which the backup battery and the power supply for short-term operation are mounted on a common circuit board, it is also possible to mount the backup battery and the power supply for short-term operation on separate circuit boards. With this structure, it becomes possible to easily replace only one of the backup battery or the power supply for short-term operation.

[0200] (O-7) When the housing is mounted on the wall of the vehicle, the lower part of the cover 32 should be open. The circuit board on which the backup battery and the power supply for short-term operation are mounted should be positioned so that they can be replaced through the open part of the cover 32. Providing an open part in the cover 32 is also effective from the viewpoint of heat dissipation. The backup battery and the power supply for short-term operation should be positioned below the main heat-generating components of the processing unit 40. This can suppress thermal degradation of the backup battery and the power supply for short-term operation.

[0201] (O-8) When the housing is attached to the wall of the vehicle, the side of the cover 32 facing upward should be shaped to slope downward toward the front. This makes it difficult for passengers of the vehicle to reach out and touch the upper side of the cover 32.

[0202] (O-9) The cover 32 is best secured to the support part 31 with screws. These screws should have a fall prevention mechanism so that they do not fall out of the cover even if they are completely loosened. For example, with the screw passed through the through hole in the cover, a nylon washer or the like can be fitted onto the screw to prevent it from coming loose, so that the cover is sandwiched between the screw head and the washer.

[0203] [Function to play back videos and images recorded on a P.SD card] Next, we will describe an image playback device that plays back moving images recorded on an SD card. While the hardware configuration of the image playback device is not shown in the diagram, it can be similar to that of a typical personal computer. That is, the image playback device may include a processing unit (exemplified by a CPU), memory, an operating device, an interface for external storage media, a communication module, and auxiliary storage. The memory is used as a work area during image playback processing. The operating device accepts user input. For example, a keyboard or pointing device may be used as the operating device. The communication module communicates with external devices (e.g., wired or wireless). The auxiliary storage device includes, for example, a hard disk drive and stores programs, viewers, and other data. The processing unit reads the viewer and other necessary programs into memory and performs various calculations according to those programs.

[0204] This image playback device has the function of reading video data captured by cameras 35 installed in multiple train cars and recorded on an SD card, and storing it in a storage device. At this time, the image playback device associates the identification information of the camera 35 recorded on the SD card with the video data. The image playback device has the function of obtaining the train name, car number, location information of the camera 35 within the car, etc., from the identification information of the camera 35. The image playback device has the function of searching a database containing the identification information of the camera 35 and information about the train on which the camera 35 is installed, and obtaining this information.

[0205] The image playback device displays information on the configuration of multiple train cars coupled together and the mounting positions of cameras 35 inside the cars on a display screen, and has a function to allow the user to select a specific camera 35 from the information displayed on the display screen. Once a camera 35 is selected by the operator, the image playback device has a function to display the image captured by the selected camera 35 on the display screen.

[0206] This function allows you to easily select the camera 35 that captured the video you want to review and play it back.

[0207] [Q. Function to link sensor detection results with video images captured by camera 35] Next, we will explain the function of linking the sensor detection results with the video images captured by the camera 35.

[0208] Various sensors, such as a 3-axis accelerometer, gyroscope, temperature sensor, and gas sensor, are installed in the train car. The processing unit of the imaging device 23 has the function of recording sensor detection information on an SD card in association with the image data captured by the camera 35.

[0209] The image playback device has the function of associating images recorded on an SD card with sensor detection information and displaying them on a display screen. For example, if the sensors installed in the vehicle are a 3-axis accelerometer or a gyroscope, the image playback device synchronizes the detection results of these sensors with moving images and displays them simultaneously on the display screen. This provides information on how passengers inside the vehicle behave when a certain type of acceleration is applied to the vehicle. In particular, it is advisable to install 3-axis accelerometers or gyroscopes in different locations on a single vehicle. The vehicle's behavior can be analyzed from the detection results of these sensors. From the vehicle's behavior, abnormalities in track conditions, vehicle conditions, driving abnormalities, etc., can be identified. From the images captured by camera 35, information on how passengers inside the vehicle behave when a certain type of acceleration is applied to the vehicle can be obtained. Furthermore, the vehicle's behavior obtained from the sensor detection results can be reproduced in a driving simulator, etc., allowing the driver to experience an impact equivalent to that received by passengers. Normally, a driver can only experience the behavior of the driver's seat in the lead vehicle. Experiencing how their driving actions are reflected in the vehicle's behavior and how passengers feel motivates drivers to drive in a way that provides a comfortable ride.

[0210] For example, if a sensor installed in the vehicle is a temperature sensor, the image playback device synchronizes the detection results from the temperature sensor with the image captured by the camera 35 and displays them simultaneously on the display screen. This allows for understanding the relationship between temperature fluctuations and actions such as opening and closing doors, raising and lowering shades, and changes in the number of passengers. This information is useful for finding ways to maintain a more comfortable temperature inside the vehicle.

[0211] For example, a temperature sensor could be included among the sensors installed in the vehicle. Based on the temperature sensor's detection results and the images captured by the camera 35, it is possible to understand the relationship between temperature fluctuations and actions such as opening and closing doors, raising and lowering shades, and changes in the number of passengers. This information is useful for finding ways to maintain a more comfortable temperature inside the vehicle.

[0212] For example, the temperature sensor's detection results should be displayed in the driver's cab so that train crew can see them immediately. Based on the temperature inside the train car, the crew can decide whether or not to make an announcement prompting the opening and closing of shades, or whether or not to change the air conditioning settings. Based on the door opening and closing status and temperature changes, it can be determined whether or not the temperature change is caused by the opening and closing of the doors.

[0213] For example, if the sensor installed in the vehicle is a thermal camera, the image playback device has the function of identifying passengers with high fevers by comparing the images captured by camera 35 with the temperature distribution obtained by the thermal camera. Furthermore, the image playback device has the function of exploring, for example, the movement path taken by passengers with high fevers within the vehicle based on the detection results of the thermal camera. This information is an important clue for exploring, for example, the movement path of a person infected with a dangerous virus. Based on the detection results of the thermal camera, it is possible to explore, for example, the movement path taken by passengers with high fevers within the vehicle. By comparing the images captured by camera 35 with the temperature distribution obtained by the thermal camera, passengers with high fevers can be identified. This information is an important clue for exploring, for example, the movement path of a person infected with a dangerous virus.

[0214] For example, a gas sensor could be included in the sensors installed in the vehicle. Based on the detection results of the gas sensor, hazardous gases could be detected. If a hazardous gas is detected, the processing unit should immediately sound an alarm in the driver's cab. If the crew notices this alarm, they can take appropriate action before the damage becomes extensive.

[0215] The operator of the image playback device can easily select the camera 35 that captured the image they want to view. The functions of the image playback device can be implemented, for example, using an information terminal such as a personal computer and an application such as a viewer installed on this information terminal. The data of images captured by camera 35 can be acquired, for example, via an external storage medium such as an SD card or a communication network such as a wireless LAN. The information recorded on the external storage medium containing the image data should include information indicating the location of camera 35 within the vehicle and the device ID of camera 35.

[0216] The viewer should have a function to associate and store the device ID of camera 35 with information identifying the vehicle in which camera 35 is installed, for example, "car 7 of Nozomi 100". The terminal on which the viewer is installed should have a function to import image data recorded on multiple external storage media, and the viewer should be able to access any image data. When the viewer operator performs an operation to specify a vehicle, the viewer should display the locations of the multiple cameras 35 installed in the specified vehicle on the display screen. For example, the vehicle should be displayed as a plan view, and marks representing camera 35 should be displayed at the locations of the cameras 35 in this plan view.

[0217] The viewer should have a function that allows the user to select one camera 35 from several displayed cameras 35 and display the image captured by that camera 35 on the display screen. It would be preferable to embed the time information, vehicle identification information, and the location information of the camera 35 within the vehicle into the image using text or graphics. It would be even better if the viewer had a function to save the displayed image as a single video file. Since this video file can be played back using a general video playback device, it can be used for various purposes, such as employee training.

[0218] The viewer should ideally have a function to analyze the skeletal structure of passengers from images of them inside the vehicle and display the bones (also called skeleton). The bone display makes it easier to recognize the passenger's posture. For example, it becomes easier to recognize various postures, such as whether they are holding onto a handrail, a bar, or nothing at all. This allows for insights into desirable handrail positions and heights, and desirable bar positions and orientations.

[0219] Furthermore, by associating the images captured by camera 35 with sensor detection information, it is possible to obtain information that cannot be obtained from images captured by camera 35 alone or from sensor detection information acquired by the sensor alone. The sensor detection information should be recorded in chronological order. Camera 35 should be placed near each of the multiple individual doors of the vehicle. Alternatively, multiple cameras 35 should be arranged in a zigzag pattern in a plan view near the right-hand door and the left-hand door.

[0220] [Communication function with R server] Next, we will explain the communication function between the camera 23 mounted on the train and an external server. The recording device 23 functions as the parent, and the server functions as the child. In other words, communication can only be initiated by the recording device 23, and cannot be initiated by the server. The recording device 23 has the function of recording moving images captured by the camera 35 onto an SD card and continuously transmitting moving image data to the server (hereinafter referred to as the "continuous transmission function").

[0221] Furthermore, the recording device 23 has a function (hereinafter referred to as the "specific transmission function") that transmits video data to the server only in specific situations, such as in an emergency when recording to the SD card becomes impossible, or when it receives an instruction from the server to upload video data. The recording device 23 also has a function that enables one of the two functions, the continuous transmission function or the specific transmission function, and disables the other.

[0222] As a variation, the recording device 23 may have the function of a child, and the tablet terminal may have the function of a parent. The recording device 23 continuously records video images captured by the camera 35 onto an SD card and maintains a waiting state for connection start requests for communication. When it receives a connection start request from the tablet terminal, communication becomes possible between the recording device 23 and the tablet terminal. When it receives a request from the tablet terminal to send video data, the recording device 23 reads the video data from the SD card and sends it to the tablet terminal. The operator of the tablet terminal can check whether video images have been properly recorded on the SD card without removing the SD card from the recording device 23.

[0223] [S. Other variations] (S-1) Mobile vehicles include vehicles that carry an unspecified number of passengers, such as trains and buses; vehicles that carry an unspecified one or a small number of passengers, such as taxis and hired cars; and elevators that move up and down within a building, carrying an unspecified number of passengers. In particular, it is good to place cameras on mobile vehicles in which the same passengers are photographed by cameras for a certain period of time without passenger change, and then the passengers change and the attributes of the passengers change after a certain period of time. For example, it is good to place cameras on mobile vehicles that operate by passing through multiple stops such as train stations and bus stops in sequence, such as trains and buses. When the passengers on the mobile vehicle change at a station or bus stop, it is good to perform the image-based content selection again. The acquisition of evaluation information is good to be performed continuously, for example at a fixed time interval, regardless of passenger changes. Also, mobile vehicles may be vehicles that carry a specific large number of passengers or a specific small number of passengers. Furthermore, if the mode of transport is a train, the stop corresponds to a station; if the mode of transport is a bus, the stop corresponds to a bus stop; and if the mode of transport is an elevator, the stop corresponds to each floor.

[0224] (S-2) In the embodiment, the external storage medium was described as an SD card, but the external storage medium may be an external storage medium other than an SD card (for example, a USB memory stick). The external storage medium may also be a storage medium realized by a cloud computing environment or a storage medium provided outside of the processing unit.

[0225] (S-3) In the above-described embodiment, some functions of the processing unit may be implemented in the camera-side processing unit, and the remaining functions may be implemented in the display device-side processing unit. For example, the function of analyzing images captured by the camera may be implemented in the camera-side processing unit, and the function of selecting content based on the image analysis results may be implemented in the display device-side processing unit. In this case, the image analysis results may be transferred from the camera-side processing unit to the display device-side processing unit. The analysis results may include information on passenger attributes and the proportion of passengers with those attributes. In this case, the display device-side processing unit may select at least one content from multiple content based on the passenger attributes, etc. Alternatively, information on the type of content to be displayed may be transferred from the camera-side processing unit to the display device-side processing unit. In this case, the display device-side processing unit may select content according to the information on the type of content.

[0226] (S-4) Some or all of the functions of the processing device described in the above-described embodiment may be implemented on an external server. In this case, images captured by the camera may be transmitted to the server via a communication network. The communication network may be a public communication network, such as the Internet. The communication network may be wired, wireless, or a combination thereof. The server may, for example, acquire information that forms the basis for evaluating the content displayed on the display device based on the images captured by the camera.

[0227] The scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the present invention disclosed herein. While the configurations for which patent protection is sought are specified in the appended claims, the present invention intends to include configurations disclosed herein that are not currently specified in the claims in the future.

[0228] The present invention is not limited to the configuration described in the embodiments above. The components of each embodiment and modification described above may be arbitrarily selected and combined. Furthermore, any component of each embodiment and modification may be arbitrarily combined with any component described in the means for solving the invention, or any component that embodies any component described in the means for solving the invention. We intend to obtain rights for these as well in amendments or divisional applications of this application.

[0229] Furthermore, by converting to a design registration application, we intend to acquire rights to the overall design or a partial design. The drawing depicts the entire device with solid lines, but it is a drawing that includes not only the overall design but also partial designs claimed for parts of the device. For example, it is a drawing that includes not only a partial design for a part of the device's components, but also a partial design for a part of the device regardless of its components. A part of the device may be a component of the device, or a part of a component. We intend to acquire rights not only to the overall design, but also to a partial design where any part of the solid lines in the drawing is represented by dashed lines. In addition, all modules, components, and parts inside the device's casing that are shown in the drawing are independently tradable, and similarly, we intend to acquire rights to them by converting to a design registration application. [Explanation of Symbols]

[0230] 20 Vehicle bodies 21 doors 22 Display device 23. Imaging device 31 Support part 32 Cover 33 Torx screws 34 Recording Unit 35 Camera 36 Posture adjustment mechanism 36A Intermediate support member 36B, 36C Rotation axis 37 Opening 38 Rotating Lid 40 Processing Unit 41 CPU 42 Reader / Writer 43 Storage device 44 Image data output terminal 45 Power Cables 46 Interlocking switch 50 pieces of luggage 51 Handle 55, 56 bars 60 Storage location 61 Partition Plate 62 Swing Doors 63 Lock button 70 Train 71 Home 72 Station display devices 73. Processing device for station display devices 75 passengers

Claims

1. A reader / writer that reads and writes data to an external storage medium that records images taken by a camera, An opening through which the external storage medium passes when the external storage medium is inserted into the reader / writer and when the external storage medium is removed from the reader / writer, A rotating lid that enables a state in which the opening is closed and a state in which the opening is open, An interlocking switch that turns the power of the imaging device on and off in conjunction with the opening and closing of the aforementioned rotating lid, A power supply for short-term operation, Processing device and It has, The processing device has a function that, upon detecting that the interlocking switch has been turned off, receives power from the short-time operation power supply and performs termination processing on the external storage medium. A photographic device.

2. The interlocking switch is turned on when the rotating lid is blocking the opening, and turned off when the rotating lid is opening the opening. The imaging apparatus according to claim 1.

3. When the rotating lid opens the opening and the interlocking switch is turned off, the risk of the external storage medium being removed from the reader / writer while power is supplied to the external storage medium is eliminated. The imaging apparatus according to claim 1 or 2.

Images