Processor, system and program

The processing device addresses the challenge of utilizing images inside moving bodies for personalized content display and theft prevention by selecting and evaluating content based on passenger attributes and movements, enhancing engagement and security.

JP2024167373AActive Publication Date: 2024-12-03YUPITERU CORP
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024151421
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-03
Estimated Expiration
2039-09-17

AI Technical Summary

Technical Problem

Existing systems fail to effectively utilize images taken inside moving bodies, such as trains or buses, for content display and evaluation, lacking the ability to dynamically adapt content based on passenger attributes and movements.

Method used

A processing device that selects and evaluates content based on images taken inside moving objects, utilizing attributes and actions of passengers, and adjusts content display according to changing passenger demographics and movements, while incorporating image processing and theft prevention mechanisms.

Benefits of technology

Enables dynamic and personalized content display, enhances passenger engagement, and prevents theft by adapting to passenger attributes and movements, while ensuring privacy and security.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167373000001_ABST
    Figure 2024167373000001_ABST
Patent Text Reader

Abstract

To provide a method by which a picture created by taking an inside of a movable body is utilized.SOLUTION: A processor 40 comprises at least one of a selection function to select a content displayed on a display device 22 arranged in a movable body from plural contents, based on pictures taken by a camera 35 taking inside of the movable body and an estimation information acquisition function to acquire basic information to estimate the content displayed on the display device 22, based on the pictures taken by the camera 35.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a processing device, a system, and a program. [Background technology]

[0002] 2. Description of the Related Art There is known a device that installs a video camera inside a train carriage, extracts and displays the passenger occupancy rate from an image captured by the video camera (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-254440 Summary of the Invention [Problem to be solved by the invention]

[0004] One of the objects of the present invention is to provide a technique for utilizing images captured inside a moving body. For example, a display device for displaying, for example, commercials or other content may be installed in a moving body such as a train, a bus, or an elevator. In this case, it is also one of the objects of the present invention to provide a technique for utilizing images captured inside the moving body where the content is displayed.

[0005] The purpose of the present invention is not limited to this, and the applicant intends to obtain rights for configurations that aim to obtain effects from parts of the configurations disclosed in this specification and drawings, etc., by filing a divisional application, amendment, etc. For example, this specification discloses a problem in which the part described as "can be" in this specification is read as "the problem is to be." The problems are described as being independent, and the applicant intends to obtain rights for the configurations for solving these problems alone by filing a divisional application, amendment, etc. Even if the problem is implicitly understood from the description in the specification, the applicant intends to include part of the configurations described in this specification in the scope of the patent claim by amendment or divisional application. Problems that combine these independent problems are also disclosed. [Means for solving the problem]

[0006] (1) The processing device should have at least one of the following functions: a selection function for selecting content to be displayed on a display device placed inside a moving body from a plurality of contents based on an image captured by a camera that captures images inside the moving body, and an evaluation information acquisition function for acquiring information that serves as the basis for evaluating the content displayed on the display device based on the image captured by the camera.

[0007] In this way, it is possible to select content to be displayed based on images taken inside the moving body, or to obtain information that serves as the basis for evaluating the content, thereby providing technology for utilizing images taken inside the moving body in which content is displayed.

[0008] (2) The processing device may be provided with at least the selection function, the selection function preferably including a function for selecting content based on at least one of the attributes and the actions of a person within the moving body.

[0009] In this way, it is possible to select content according to at least one of the attributes and the actions of the person inside the moving object.

[0010] (3) The processing device may be provided with at least the selection function, which has the function of detecting a person directing his / her gaze toward the display device based on an image captured by the camera, and selecting content to be displayed on the display device based on the attributes of the person directing his / her gaze toward the display device.

[0011] In this way, it is possible to display on the display device content that corresponds to the person who is actually directing his or her gaze toward the display device.

[0012] (4) The camera is disposed on a mobile vehicle that travels in sequence through a number of stops where people get on and off, and the processing device has at least the selection function, and the selection function may include a function of selecting content for each traveling section between the stops and displaying the selected content corresponding to that traveling section on the display device during the period when the vehicle is traveling along that traveling section.

[0013] In this way, in a case where the people present inside the moving body may change for each travel section, content according to the people currently inside the moving body can be selected and displayed on the display device.

[0014] (5) The processing device may be provided with at least the evaluation information acquisition function, and the evaluation information acquisition function may include at least a function for acquiring information regarding the number of people viewing the display device based on an image captured by the camera while content is being displayed by the display device.

[0015] In this way, information regarding the number of people within the moving object during the display of the content can be obtained as information that serves as the basis for evaluating the content.

[0016] (6) The evaluation information acquisition function may be a processing device that includes a function for detecting the number of people within the moving body and the line of sight of people based on images captured by the camera.

[0017] In this way, information regarding the number of people within the moving object while the content is being displayed and the people's line of sight can be obtained as information that forms the basis for evaluating the content.

[0018] (7) It is preferable that the processing device further has a function of associating and recording an image captured by the camera, information identifying the content displayed on the display device, and information serving as the basis for evaluating the content.

[0019] In this way, the content can be recorded in association with information that is the basis for evaluating the content.

[0020] (8) The processing device may further include a function of detecting a person directing his / her gaze toward the display device based on an image captured by the camera, and recording information identifying the person directing his / her gaze toward the display device in association with the content displayed on the display device.

[0021] In this way, the content can be recorded in association with information that identifies the person who is actually directing his or her gaze toward the display device.

[0022] (9) It is preferable that the processing device further includes a function of detecting a position within the display device to which a person is directing his or her gaze, and recording the content displayed on the display device in association with the position to which the person is looking.

[0023] In this way, it is possible to record information for knowing at what position within the video content the person inside the moving object is directing his / her gaze.

[0024] (10) The processing device may include at least the evaluation information acquisition function, which evaluates content based on the information acquired by the evaluation information acquisition function.

[0025] In this way, the content displayed on the display device disposed in the vehicle can be evaluated.

[0026] (11) The processing device may further have a function of detecting the movement path within the moving body for each person riding on the moving body from boarding to disembarking based on images captured by the camera, and recording information on the boarding point, the detection results of the movement path within the moving body, and information on the disembarking point.

[0027] In this way, it is possible to record information that allows one to know the state of movement of a person from the time he or she gets on to the vehicle until the time he or she gets off.

[0028] The function of the processor described in (11) above may be realized by a processor that does not have the functions of the processors described in (1) to (10).

[0029] (12) It is preferable that the processing device further has a function of causing the display device to display an image captured by the camera or an image that has been processed, instead of or in combination with displaying content on the display device.

[0030] In this way, an image captured by the camera can be shown to a person inside the moving body instead of or in combination with the content.

[0031] (13) A locking mechanism may be provided in the luggage storage area, and the processing device may set the locking mechanism to either a locked or unlocked state based on the characteristics of a person visiting the luggage storage area in an image captured by the camera.

[0032] This makes it possible to make it more difficult for luggage placed in the luggage storage area to be stolen.

[0033] (14) It is preferable that the processing device further has a function of performing image processing on a designated area of ​​an image captured by the camera so as to prevent the designated area from being seen or recognized, and recording the processed area on a recording medium.

[0034] In this way, it is possible to make it difficult for information contained in the designated area in the image captured by the camera to be viewed or recognized.

[0035] (15) It is preferable that the processing device further comprises a function of recording an image captured by the camera in a recording device, and a function of reading out the image recorded in the recording device and selectively outputting image data in which a specified image processing has been applied to a person's face or image data in which the specified image processing has not been applied.

[0036] In this way, it is possible to output the image after performing a predetermined image processing on the face of a person included in the image, or to output the image without performing any image processing.

[0037] (16) The mobile body may include a plurality of vehicles, the mobile body may pass through a plurality of stops in order, and the plurality of vehicles may be operated so as to stop at a specified stopping position at each of the plurality of stops. The processing device may further have a function of calculating a congestion rate for each of the vehicles based on images taken by the camera, and a function of selecting, for each of the specified stopping positions, content to be displayed on a station display device arranged corresponding to the specified stopping positions from a plurality of contents in accordance with the calculated congestion rate for each of the vehicles.

[0038] In this way, the movement of people who wish to board a moving object can be guided by selecting the content to be displayed on the station display device.

[0039] (17) It is preferable that the system includes the above-mentioned processing device and an image capturing device, the image capturing device having the camera, a housing for accommodating the camera, and an attitude adjustment mechanism for changing and fixing the orientation of the lens of the camera relative to the housing.

[0040] In this way, the camera orientation can be adjusted while the housing is fixed to the moving body.

[0041] (18) The attitude adjustment mechanism may have an intermediate support member rotatably supported on the housing, the camera may be rotatably supported on the intermediate support member, and a rotation axis of the intermediate support member relative to the housing and a rotation axis of the camera relative to the intermediate support member may be perpendicular to each other.

[0042] In this way, the camera attitude can be adjusted in each of at least two axial directions.

[0043] (19) The housing may include a support portion that supports the camera and a cover that covers the camera supported by the support portion, and may further include a reader / writer that reads and writes data to an external storage medium that records images taken by the camera, and may have an opening through which the external storage medium passes when the external storage medium is inserted into the reader / writer and when the external storage medium is removed from the reader / writer, and may have a rotating lid that rotates to close the opening and open the opening.

[0044] In this way, the cover and the rotating lid can be used to protect the external storage medium from theft.

[0045] (20) It is preferable to provide a program for causing a computer to realize the functions of the processing device according to any one of (1) to (19) above.

[0046] The inventions shown in (1) to (19) above can be combined in any way. For example, at least a part of the configuration of at least one of the inventions from (2) onwards may be added to all or a part of the configuration of the invention shown in (1). In particular, it is preferable to add at least a part of the configuration of at least one of the inventions from (2) onwards to the invention shown in (1). In addition, any configuration may be extracted from the inventions shown in (1) to (19) and the extracted configurations may be combined. The applicant of this application intends to obtain rights to inventions including these configurations.

[0047] (21) It is preferable that the image playback device has a function of acquiring image data taken by cameras attached to each of multiple cars of a train, a function of displaying information regarding the coupling configuration of the multiple cars of the train and the installation positions of the cameras within the cars on a display screen, a function of selecting a specific camera from the information displayed on the display screen, and a function of displaying images taken by the selected camera on the display screen.

[0048] In this way, the operator of the image reproducing device can easily select the camera that captured the image that he or she wishes to see.

[0049] (22) The image playback device may further include a function for acquiring sensor detection information that is detected by a sensor installed in the vehicle and associated with image data captured by the camera, and a function for associating the image captured by the camera with the sensor detection information and displaying the associated image on a display device.

[0050] In this way, images captured by the camera can be associated with sensor detection information.

[0051] (23) It is preferable to provide a program for causing a computer to realize the functions of the image display device according to (21) or (22) above. Effect of the Invention

[0052] According to the present invention, images captured inside a moving body can be utilized for content selection and content evaluation.

[0053] The effect of the invention of the present application is not limited to this, and the effect achieved from the configuration disclosed in the specification and drawings, etc. is also disclosed, and there is an intention to obtain rights to the configuration achieving said effect through divisional application, amendment, etc. For example, in this specification, the part that states "can..." is a description that clearly indicates the effect achieved, and there are parts that show the effect even without the description "can...". Also, there are effects that can be understood from the configuration even without such a description. [Brief description of the drawings]

[0054] [Figure 1] FIG. 1A is a side view of the interior of one train car equipped with an imaging device and a display device in which a processing device according to an embodiment is housed, and FIG. 1B is a diagram showing a plan layout of the interior of one train car equipped with an imaging device and a display device in which a processing device according to an embodiment is housed. [Diagram 2] FIG. 2A is a perspective view of an image capturing device in which a processing device according to an embodiment is housed, and FIG. 2B is a perspective view of the image capturing device with a cover removed. [Diagram 3] 3A, 3B, 3C, 3D, 3E, and 3F are a plan view, a left side view, a front view, a right side view, a bottom view, and a rear view of the imaging device, respectively. [Figure 4] FIG. 4 is a block diagram of a processing device according to an embodiment. [Diagram 5] FIG. 5 is a flow chart illustrating the passenger attribute based content selection function. [Figure 6] FIG. 6 is a flow chart illustrating the passenger action based content selection function. [Figure 7] FIG. 7 is a flowchart showing a function of acquiring evaluation information of content displayed on a display device. [Figure 8] FIG. 8 is a flow chart showing the function of performing image processing and recording. [Figure 9] FIG. 9A is a flowchart showing a function for recording moving images captured by a camera, and FIG. 9B is a flowchart showing a function for outputting image data. [Figure 10] 10A and 10B are perspective views of a locking mechanism provided in a baggage storage area in a train, in an unlocked state and a locked state, respectively. [Figure 11] 11A and 11B are a front view and a plan view, respectively, of another locking mechanism provided in a baggage storage area in a train. [Figure 12]12A and 12B are schematic diagrams showing a train on a track, a station display device installed on a station platform, and a queue of passengers. [Figure 13] FIG. 13 is a perspective view of the imaging device (FIGS. 2A and 2B) according to another configuration example with the cover removed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0055] A processing device according to an embodiment will be described with reference to Figs. 1A to 13. The processing device according to the embodiment constitutes an imaging device housed together with a camera in a common housing. This imaging device is mounted on a train together with a display device. In this embodiment, a case where the moving body is a train car will be described as an example. A train car is an example of a vehicle that operates with an unspecified number of passengers on board. In this example, the people inside the car are considered to be train passengers.

[0056] 1A and 1B are a side view of the interior of one carriage of a train equipped with an image capturing device and a display device housing a processing device according to each embodiment, and a planar layout of the interior of one carriage. A plurality of doors 21 are arranged on both the left and right sides of the carriage body 20 in the direction of travel. A display device (which may be called a digital signage for a carriage) 22 and a box-shaped image capturing device 23 are attached to the wall surface (called a fascia board) above the doors 21. The image capturing device 23 is attached to, for example, either the left or right side of the display device 22.

[0057] In this embodiment, three doors 21 are arranged on each of the two sides of the vehicle. The display device 22 is attached to the upper panel of all the doors 21. The display device 22 is arranged at a predetermined position in the vehicle, and it is preferable that the display device 22 is arranged at a position where the image displayed by the display device 22 is easily viewable by passengers in the vehicle. For example, the display device 22 is arranged at a position higher than the height of an average person based on the position of the floor of the vehicle. The display device 22 is preferable to be a liquid crystal display device, an organic electroluminescence display device, or a display device having other display elements. The display device 22 is not limited to being installed on the wall surface of the vehicle, and may be realized, for example, by a digital signage arranged in the vehicle, a hanging advertisement made of electronic paper, or other display device.

[0058] The camera 23 is attached to the upper panel of one of the left and right doors 21 that are located at the same position in the traveling direction. On the right wall surface of the vehicle (the upper side surface in FIG. 1B), the camera 23 is attached above the odd-numbered doors 21. On the left wall surface of the vehicle (the lower wall surface in FIG. 1B), the camera 23 is attached above the even-numbered doors 21. These multiple camera devices 23 make it possible to capture images of almost the entire interior of the vehicle.

[0059] Fig. 2A is a perspective view of the photographing device 23 in which the processing device according to the embodiment is housed, and Fig. 2B is a perspective view of the photographing device 23 with the cover removed. Figs. 3A, 3B, 3C, 3D, 3E, and 3F are a top view, a left side view, a front view, a right side view, a bottom view, and a back view of the photographing device 23, respectively.

[0060] The support part 31 and the cover 32 form the housing of the photographing device 23. The camera 35 and the recording part main body 34 are housed in the housing of the photographing device 23. The recording part main body 34 is fixed to the support part 31 on the front side of the support part 31. The recording part main body 34 covers almost the lower half of the front side of the support part 31. The camera 35 is located on the front side of the support part 31 and above the recording part main body 34. The camera 35 is supported by the support part 31 via the recording part main body 34. The recording part main body 34 houses an electronic circuit board and the like of the processing device. The support part 31 is fixed to the wall surface of the vehicle. The cover 32 covers the camera 35 and the recording part main body 34 from the front side of the support part 31.

[0061] The support part 31 is formed of, for example, metal. The cover 32 is supported by the support part 31 and covers the camera 35. The cover 32 has a light-transmitting area on the front side. This is because the cover 32 does not interfere with the shooting of the camera 35. The light-transmitting area is, for example, an area in which a transparent member (for example, resin or glass) is provided at the opening of the cover 32. The other parts of the cover 32 are formed of, for example, aluminum die-cast. The camera 35 takes pictures through the light-transmitting area. The cover 32 is attached to the support part 31 by a Torx screw 33. The surface of the cover 32 facing upward is inclined so as to descend toward the front.

[0062] The camera 35 captures images and generates image data showing the captured images. The camera 35 includes, for example, a CCD, a CMOS, or other imaging element, and a lens that focuses light on the position of the imaging element. The camera 35 has a function of capturing moving images, but may also have a function of capturing still images. The camera 35 may have a wide-angle lens to capture as wide an area as possible. If one camera cannot capture the entire area inside the vehicle, multiple cameras may be arranged to prevent blind spots from occurring inside the vehicle. If multiple display devices 22 are arranged inside the vehicle, at least one camera 35 may be arranged for each display device 22. If a wider area needs to be captured with one camera 35, a 360-degree camera such as a semi-spherical camera or a celestial sphere camera may be used.

[0063] If the camera 35 installed in the vehicle is facing the passenger, the passenger may feel uneasy as if he or she is being constantly watched. If the camera 35 is housed in a housing, the passengers in the vehicle will have a hard time recognizing the direction of the camera 35, and this uneasiness is less likely to be caused to the passengers.

[0064] Camera 35 is supported by recording unit main body 34 via attitude adjustment mechanism 36. Attitude adjustment mechanism 36 is a mechanism that changes and fixes the orientation of the lens of camera 35 with respect to the housing of imaging device 23. By using attitude adjustment mechanism 36, it is possible to adjust the orientation of camera 35 while the housing is fixed to the wall surface of a train or the like.

[0065] Generally, in order to increase the maximum value of the swing angle of the camera's attitude, the housing that houses the camera must be made larger. Conversely, if the maximum value of the swing angle of the camera's attitude is too small, it may not be possible to fit the desired range within the camera's angle of view when the housing is fixed inside the electric vehicle. For this reason, it is considered sufficient to be able to change the direction of the camera 35 within a range of ±10 degrees for normal use inside the vehicle.

[0066] The posture adjustment mechanism 36 includes an intermediate support member 36A. The intermediate support member 36A is L-shaped here. One side of the intermediate support member 36A extending in the left-right direction when the photographing device 23 is viewed from the front is a plate-like portion along the upper surface of the recording unit main body 34. One side of the intermediate support member 36A extends in the left-right direction when the photographing device 23 is viewed from the front, and is supported rotatably with respect to the housing of the photographing device 23 and the recording unit main body 34. The other side of the intermediate support member 36A is located on the left side when the photographing device 23 is viewed from the front, and is a plate-like portion along the left side surface of the camera 35. The camera 35 is supported rotatably with respect to the other side of the intermediate support member 36A. A rotation axis 36B of the intermediate support member 36A relative to the housing of the photographing device 23 and the recording unit main body 34 and a rotation axis 36C of the camera 35 relative to the intermediate support member 36A intersect each other (orthogonal in this embodiment). The rotation axis 36B extends directly upward from the recording unit main body 34. The rotation axis 36C penetrates the other side of the intermediate support member 36A. The rotation angles of the rotation axes 36B and 36C change between a plurality of (e.g., three) scale positions and can be fixed at each scale position. In this way, the camera 35 has two rotation axes relative to the recording unit main body 34 and can rotate in two mutually intersecting directions. In this embodiment, the rotation axis 36B is in the vertical direction (i.e., up-down direction), and the rotation axis 36C is in the horizontal direction (i.e., left-right direction). In this way, the camera 35 can adjust its posture with respect to the horizontal and vertical directions.

[0067] It is preferable that a scale is provided on the intermediate support member 36A or the housing so that it is possible to visually confirm how many degrees the camera 35 has rotated from a reference attitude relative to the housing. The scale may be provided in increments of 5 degrees, for example. The attitude of the camera 35 relative to the housing may be common, for example, for each type of vehicle on which the image capturing device 23 is attached and each installation location. If this common attitude is known in advance, it is preferable to adjust the attitude of the camera 35 to an optimal position at the time of shipment from the factory. In this way, it becomes unnecessary to adjust the attitude of the camera 35 on site, and it is possible to improve the efficiency of work on site.

[0068] 4 is a block diagram of a processing device 40 according to an embodiment. The processing device 40 includes a central processing unit (e.g., a CPU, hereinafter referred to as "CPU") 41, a reader / writer 42, a storage device 43, and an image data output terminal 44. The storage device 43 stores a program executed by the CPU 41. The CPU 41 functions as a control means for realizing various functions of the processing device 40 by executing this program. The storage device 43 includes a main storage device used as a work area for arithmetic processing, and an auxiliary storage device exemplified by a hard disk device.

[0069] The reader / writer 42 is an SD card reader / writer that reads and writes image data and the like to an SD card attached to the reader / writer 42 under the control of the CPU 41. An SD card that records images captured by the camera 35 is attached to the reader / writer 42. The SD card is an example of an external storage medium or removable medium. The CPU 41 outputs an image signal from the image data output terminal 44. When attaching the imaging device 23 to the wall surface of the vehicle, an operator can connect a monitor to the image data output terminal 44 to check the imaging range of the camera 35.

[0070] The image data output terminal 44 is a terminal for outputting image data, and is, for example, an NTSC output terminal. An operator can connect an image monitor to the image data output terminal 44 and adjust the attitude of the camera 35 while checking the current image. In order to make it easy to check the current image, the image data output terminal 44 should be provided in a location that is in front of the processing device 40 with the photographing device 23 fixed to the wall of the vehicle or the like. In order to be able to view the image even while the photographing device 23 is operating, it is preferable to pull out a cable above the housing and connect it to an image signal cable provided in the vehicle.

[0071] Image data showing an image captured by camera 35 is input to processing device 40. Processing device 40 selects content to be displayed on display device 22 based on this image data. Processing device 40 outputs information identifying the selected content (hereinafter referred to as "content type information") to display device 22. Display device 22 displays video content identified by the content identification information input from processing device 40. Note that the video content is assumed to be stored in display device 22, but may be stored in storage device 43 or another storage device. In the latter case, processing device 40 may output content data to display device 22.

[0072] The processing device 40 may be arranged, for example, in the same vehicle in which the camera 35 is arranged. The camera 35 and the processing device 40 may be housed in the same housing and integrated into one device. For example, if the display device 22 has already been installed in the vehicle by the owner of the vehicle (in this embodiment, the railway operator), a device integrating the camera 35 and the processing device 40 may be retrofitted to the vehicle. The display device 22 and the processing device 40 may also be housed in the same housing and integrated into one device. For example, if the camera 35 has already been installed in the vehicle by the owner of the vehicle, a device integrating the display device 22 and the processing device 40 may be retrofitted to the vehicle.

[0073] Next, the operation of this embodiment will be described. Note that the expression "the processor 40" as the subject of control may be read as "the CPU 41".

[0074] [A. Content selection function based on passenger attributes] Next, with reference to FIG. 5, a content selection function based on passenger attributes will be described as one of the functions of the processing device 40 and the display device 22.

[0075] Fig. 5 is a flowchart showing the content selection function based on passenger attributes. Each time the processing device 40 detects that a train has departed from a station, the processing shown in Fig. 5 is executed. The method of detecting that a train has departed from a station is not important, but for example, the processing device 40 may detect that a train has arrived at and departed from a station by detecting the opening and closing of the boarding and alighting doors of the train based on an image taken by the camera 35. When the train departs from a station, the processing device 40 first acquires image data from the camera 35 (step SA1).

[0076] The processing device 40 performs image analysis on the acquired image data and determines the attributes of the passengers (step SA2). The image analysis may be performed using a known image analysis technique such as a technique using a neural network. The attributes of the passengers include, for example, gender and age. The processing device 40 classifies the passengers into eight age groups, for example, ages 0 to under 10, ages 10 to under 20, ages 20 to under 30, ages 30 to under 40, ages 40 to under 50, ages 50 to under 60, ages 60 to under 70, and ages 70 and over. The processing device 40 counts the number of men and women for each age group. The processing device 40 counts the number of passengers for a total of 16 age and gender groups.

[0077] The processing device 40 selects one or more pieces of content to be displayed from a plurality of commercial contents pre-registered in the storage device 43, for example, according to the distribution of the number of passengers in each age and gender group (step SA3). The processing device 40 selects a plurality of pieces of content likely to attract the interest of people of the largest age and gender group, for example. The processing device 40 notifies the display device 22 of the identification information of the selected content (step SA4).

[0078] Here, the processing device 40 sequentially displays images of a plurality of contents selected from pre-registered contents on the display screen of the display device 22. In this manner, the processing device 40 has a function of dynamically changing the contents to be displayed on the display device 22 according to the attributes of the current passenger.

[0079] The content that the processing device 40 causes the display device 22 to display may be video content such as moving images or still images. In particular, commercial videos introducing the products and services of a company, company introduction videos introducing the company itself, job information, information to alert passengers, and the like may be displayed on the display device 22. The processing device 40 may select content that is likely to interest the multiple passengers captured in the image from multiple pieces of content prepared in advance, and display the selected content on the display device 22. The processing device 40 may select content according to the attributes of the passengers that make up the majority of the attributes of the multiple passengers captured in the image. The attributes of the passengers are not limited to gender and age, and may be clothing or other attributes.

[0080] The processing device 40 determines the attributes of passengers every time a train departs from a station, and can select content based on the attributes of new passengers after passengers are replaced at the station. Therefore, the processing device 40 can display content that is highly appealing according to the attributes of the current passengers.

[0081] The processing device 40 may select content for each section of travel between stops (between stations in this embodiment) of the vehicle, and while the vehicle is traveling along that section, display the selected content corresponding to that section on the display device 22. In this way, even if the passengers in the vehicle change at the stops, the display device 22 can display content that is preferred according to the passengers at the current time.

[0082] [B. Variation of content selection function based on passenger attributes] (B-1) The processing device 40 may adjust the time allocation of the types of content to be displayed on the display device 22 according to the ratio of the number of passengers for each attribute. For example, the processing device 40 may extend the display time of the content selected according to a certain attribute as the ratio of passengers having that attribute increases.

[0083] For example, when the majority of passengers are female, the processing device 40 may select commercials for women, such as commercials for cosmetics and magazines for women, as the content to be displayed on the display device 22. When the majority of passengers are elderly, the processing device 40 may select commercials for elderly, such as commercials for walking sticks and nursing care products, as the content to be displayed on the display device 22. When the majority of passengers are wearing ties, the processing device 40 may select economic-related commercials as the content to be displayed on the display device 22. When it is determined that many passengers like games based on the age of the passengers, the processing device 40 may select game commercials as the content to be displayed on the display device 22. When many passengers are reading books, the processing device 40 may select book commercials as the content to be displayed on the display device 22. When many passengers are operating smartphones, the processing device 40 may select smartphone app commercials, job advertisements for app development companies, and the like, as the content to be displayed on the display device 22.

[0084] The processing device 40 uses a content selection function based on passenger attributes to display on the display device 22 images of content likely to interest many passengers, thereby enabling content providers to increase the appeal of the content. Also, many passengers can view content that interests them.

[0085] (B-2) When passengers include passengers with specific attributes, the processing device 40 may exclude specific content from a plurality of content items to be selected. For example, even if there are many female passengers, when the ratio of male passengers is equal to or greater than a certain ratio, the processing device 40 may exclude commercials for women's underwear from the candidates for content to be displayed on the display device 22. Conversely, when the ratio of female passengers is equal to or greater than a certain ratio, the processing device 40 may exclude commercials for men's underwear from the candidates for content to be displayed on the display device 22.

[0086] (B-3) The processing device 40 should analyze images for each piece of content not only when the train departs from the station, but also before the display of that content ends, to determine the attributes of passengers who are looking at the display device 22.

[0087] (B-4) The processing device 40 may select the content to be displayed based on the attributes of the passenger who is looking at the display device 22. The passenger who is looking at the display device 22 may be detected by analyzing the gaze of the passenger who is captured in the image. The processing device 40 may extract the face of the passenger from the image taken by the camera 35, and detect the passenger who is looking at the display device 22 from the relationship between the position of the display device 22 and the direction of the passenger's face. Alternatively, the processing device 40 may apply a known gaze detection technology to the image taken by the camera 35 to detect the passenger who is looking at the display device 22. The position of the display device 22 may be registered in advance in the processing device 40. In this way, for example, the content that is likely to be of interest to the passenger who is actually looking at the display device 22 can be displayed on the display device 22. As a result, the appeal effect of the content can be further enhanced.

[0088] Information regarding the number of passengers viewing the content is useful information for content providers and content creators in creating new content. The more passengers view a piece of content, the more it attracts passenger interest and has a high appeal. Information regarding the number of passengers viewing the content may be, for example, the ratio of the number of passengers viewing the content to the total number of passengers. By providing information regarding the number of passengers viewing the content as a ratio rather than an absolute number, a comparison result between information obtained when the number of passengers is large and information obtained when the number of passengers is small becomes meaningful.

[0089] [C. Content selection function based on passenger behavior] Next, a content selection function based on passenger actions as one of the functions of the processing device 40 and the display device 22 will be described with reference to FIG.

[0090] FIG. 6 is a flowchart showing a content selection function based on passenger behavior. The processing device 40 periodically executes the process shown in FIG. 6 at a certain time interval, for example, every 10 seconds. The processing device 40 detects the behavior of each passenger by analyzing the video captured by the camera 35 (step SB1). The processing device 40 judges whether the behavior of each passenger matches a predetermined pattern of behavior (step SB2). The predetermined pattern of behavior indicates a pattern when a predetermined crime such as molestation or pickpocketing is being committed. If the passenger's behavior matches the predetermined pattern of behavior (step SB2; Y), the processing device 40 issues a command to the display device 22 to select and display a predetermined content (step SB3). The content here is a content for crime prevention, and this command includes the type of crime suspected to be currently being committed. The display device 22 that receives this command displays the content. The display device 22 displays the content for crime prevention according to the type of crime. For example, if a behavior corresponding to molestation is detected, text information such as "Molestation is a crime" and "Beware of molestation" or an image for deterring molestation is displayed. Moreover, if a pickpocketing behavior is detected, text information such as "Beware of pickpockets" and images to deter pickpockets are displayed. Crime prevention content displayed on the display device 22 motivates people to stop committing crimes. Videos showing criminal acts are also useful clues for identifying suspects in criminal investigations.

[0091] [D. Modifications of the Content Selection Function] (D-1) The camera 35 may have a pan / tilt function and a zoom function. When the processing device 40 detects a predetermined pattern of movement, it may pan / tilt and zoom to the location where the movement was detected, and display an image of that location on the display device 22. For example, when the camera 35 detects a criminal act such as molestation, pickpocketing, or baggage theft, it may pan / tilt to the location where the criminal act is suspected, zoom in, and display an image of that location on the display device 22. Also, instead of mechanically panning and tilting, it may enlarge an image of the location where the criminal act is suspected to be occurring and display it on the display device 22.

[0092] (D-2) The processing device 40 may select content based on audio information picked up by a microphone (audio detection means) installed in the vehicle in addition to images captured by the camera 35. For example, based on a keyword picked up during a conversation by the microphone, content associated with the keyword may be displayed on the display device 22.

[0093] (D-3) The processing device 40 may have a function of selecting the next content to be displayed based on the attributes of the passenger currently looking at the display device 22. For example, the processing device 40 may cause the display device 22 to display next content that is likely to interest the passenger currently looking at the display device 22. The attributes of the passengers and the types of content that passengers with those attributes are likely to be interested in may be stored in the processing device 40 in advance. By displaying the content on the display device 22 using this function, the passengers currently looking at the content are more likely to continue to view the next content, thereby enhancing the appeal of the content to the passengers. When multiple passengers are looking at the content, the processing device 40 may select the next content to be displayed by majority vote.

[0094] [E. Function to obtain information that is the basis for evaluating content and function to evaluate content] 7, a function for acquiring information (hereinafter referred to as "rating information") that is the basis for evaluating the content displayed on the display device 22 will be described. The content is a commercial in this example, but may be other content.

[0095] FIG. 7 is a flowchart showing the acquisition of evaluation information of the content displayed on the display device 22 and the function of evaluating the content. The processing device 40 executes the process shown in FIG. 7 periodically, for example, at a certain time interval, for example, every 10 seconds. The processing device 40 detects the gaze of passengers by analyzing the image captured by the camera 35, and counts the number of passengers directing their gaze to the display device 22 (step SC1). Step SC1 is an example of the process of acquiring evaluation information of the content. The processing device 40 associates the type of content displayed on the display device 22 when the original image to be subjected to image analysis was captured with information regarding the number of passengers directing their gaze to the display device 22, and records the information in the storage device 43 or the SD card attached to the reader / writer 42 (step SC2). The information regarding the number of passengers may be, for example, the ratio of passengers looking at the display device 22 to the total number of passengers.

[0096] The processing device 40 evaluates the content based on the type of content and the number of passengers viewing the display device 22 (step SC3). The content may be evaluated, for example, by calculating a parameter (hereinafter referred to as an "evaluation value") that is an index of the degree of interest or concern in the content among the passengers in the vehicle based on the evaluation information. The evaluation value may be, for example, the viewer rating of the content. Note that the term "viewer rating" is used here even when the video content is not accompanied by audio information. For example, the average or maximum viewer rating during the period in which the content to be evaluated is displayed may be adopted as the evaluation value.

[0097] The evaluation value of the content may be the ratio of the number of passengers in the vehicle who viewed the content for a predetermined period of time or more among the passengers in the vehicle. The evaluation value of the content may be the ratio of the number of passengers who viewed the content for a predetermined period of time or more among the passengers in the image on which the evaluation is based. Alternatively, the evaluation value of the content may be the product of the number of passengers viewing the content and the viewing time (hereinafter referred to as "viewing amount"). For example, if two people view the content for 15 seconds and three people view the content for 30 seconds, the evaluation value is 2 x 15 + 3 x 30 = 120 person-seconds. Alternatively, the evaluation value of the content may be the ratio of the actual viewing amount to the viewing amount that would be expected if all passengers in the image on which the evaluation is based were to view the content for the entire period during which the content to be evaluated is displayed.

[0098] As described above, the processing device 40 can be used as a train version of an audience rating checker. When a large number of passengers are watching the display device 22, the content being displayed at that time can be given a high rating. This evaluation result becomes useful information for creating new content.

[0099] [F. Variations of the Function for Obtaining Information on Which Content is to be Evaluated and the Function for Evaluating Content] (F-1) The processing device 40 may acquire and record, as evaluation information, information on which part of the display device 22 the passengers are looking at, instead of or in combination with the number of passengers looking at the display device 22. The processing device 40 may have a function for, for example, correlating and recording an image taken by the camera 35, information identifying the content displayed on the display device 22, and evaluation information. In this way, the content can be evaluated in more detail based on the recorded information, and the information can be used to create content with high appeal. The information on which the content is evaluated may include, for example, the number of passengers viewing the content, their attributes, information on which part of the content image the passengers are viewing, and information on the position of the vehicle when viewing the content.

[0100] (F-2) The processing device 40 may store information for identifying the content and the evaluation information in an SD card. The content can be evaluated in more detail by an external device such as a personal computer that reads the contents recorded in the SD card.

[0101] The processing device 40 can detect the passengers' line of sight and determine the location where the passengers are looking from the position of the display screen of the display device 22 and the positional relationship between the line of sight. This evaluation information can provide information on which part of the video content displayed on the display device 22 most passengers are looking at. This information allows the content to be evaluated in more detail. This evaluation result is useful information for producing content with high appeal. In this way, the evaluation basis may include, for example, the number of passengers viewing the content, the time spent viewing the content, the attributes of the passengers viewing the content, etc.

[0102] (F-3) The processing device 40 may obtain and record, as evaluation information, information identifying the passengers looking at the display device 22, in addition to information regarding the number of passengers looking at the display device 22 and information regarding which part of the display device 22 they are looking at. From this information, information regarding what type of passengers look at what content is obtained. This information can be used to create content with a higher appeal.

[0103] (F-4) The processing device 40 may have a function to display on the display screen the change over time in the number of passengers looking at the display device 22, together with the change in the content displayed on the display device 22 and the change in the position where the vehicle is traveling. It is even better if the processing device 40 has a function to display the change over time in the number of passengers looking at each position in the video of the content. This information makes it possible to analyze the reason why passengers stopped watching the content. It is possible to know whether the reason why passengers stopped watching the content is, for example, because the vehicle is approaching a station and passengers have started preparing to get off, or because the content has changed. If the number of passengers watching the content decreases without any external factors, it means that the number of passengers interested in that content has decreased.

[0104] (F-5) The processing device 40 may have a function of recording the positions in the vehicle of passengers who are looking at the display device 22. Based on the recorded information, it is possible to know which positions in the vehicle passengers are most likely to view the content. This information is useful for considering a preferable installation location for the display device 22 that displays the content.

[0105] (F-6) The processing device 40 may have a function of acquiring information necessary for analyzing what content (e.g., commercials) was frequently viewed by passengers of what attributes in what car, at what time of day, at what position (e.g., between stations) the train was traveling. The processing device 40 can optimize the content to be displayed based on the results of this analysis.

[0106] (F-7) The processing device 40 may have a function of acquiring information that can be used to perform a similar analysis on passengers who are looking at train hanging advertisements, in addition to the content displayed on the display device 22.

[0107] For example, a processing device (e.g., a computer) that displays advertisements on the display device 22 and a processing device of the photographing device 23 are connected to be able to communicate with each other. The processing device of the display device 22 outputs a signal indicating advertisement identification information (hereinafter referred to as "advertising ID") to the processing device of the photographing device 23 at the start of displaying each advertisement, and the processing device of the photographing device 23 records the advertisement ID in association with data of an image (hereinafter referred to as "camera image" in this modified example) captured by the camera 35 so that the display start time and end time of the advertisement of the advertisement ID can be known. For example, it is preferable to record the advertisement ID in association with the camera image, assuming that the video content of a certain advertisement ID is being displayed on the display device 22 from the time when a certain advertisement ID is received until the next other advertisement ID is received. Also, for example, it is preferable to input an image signal of the video content being displayed on the display device 22 to the processing device of the photographing device 23, and the processing device records the video data of the video content together with the data of the camera image.

[0108] As a method for recording the video data of the video content being displayed on the display device 22 together with the camera image, it is preferable to record the camera image and the video content with time information added so that they can be played back in time with the camera image. Also, as a method for recording the video data of the video content being displayed on the display device 22 together with the camera image, it is preferable to record the video content being displayed on the display device 22 by superimposing it on the camera image (for example, by inserting it like a wipe).

[0109] The processing device 40 should record only images taken during the period when passengers were viewing the advertisement so that they can be extracted and recorded or viewed later. For example, it is preferable to record information such as a flag that identifies images taken during the period when passengers were viewing the advertisement. When a guide such as "Next stop: XX station" is displayed instead of an advertisement, it is preferable to record information indicating that notification information (also called guidance information) was being displayed, preferably a unique notification information ID for each piece of notification information, instead of an advertisement ID. It is also preferable to record the position of which car of which train between which stations it is located so that it can be identified.

[0110] (F-8) The processing device 40 may record, in addition to the evaluation information, the video content currently being displayed on the display device 22 as image data. This information makes it easy to know what in the video content most passengers are looking at.

[0111] (F-9) The processing device 40 may have a function of detecting the position within the display device 22 at which the passenger is looking, and recording the image of the content being displayed on the display device 22 in association with that position.

[0112] In this way, it is possible to know which position in the video content the passengers are looking at based on the recorded information. For example, it is possible to obtain information on whether the passengers are looking at the product in the video of the content or the celebrity appearing in the video. Based on this information, the content can be evaluated in more detail. For example, this information can be used as useful information for creating content that is highly appealing to products. It is advisable to record necessary information so that it is possible to easily analyze, from the recorded information, the specific focus position in the video content, and what type of passengers are riding at what position in the video content and what position in the video content they are looking at at what time.

[0113] The device that reads out the recorded information, for example a personal computer with a viewer installed, should have a function to simultaneously display on the display screen the image of the content together with various information when the image is displayed on the display device 22 in the vehicle, such as the number of passengers viewing the content, the time, the location where the vehicle is traveling, the attributes of the passengers viewing the content, etc. This allows one to intuitively grasp what kind of content is often viewed by what kind of passengers.

[0114] (F-10) The device that reads out the recorded information may display passengers who are looking at video content such as advertisements in a different display mode from that of other passengers. For example, the position of the face of a passenger who is watching the video content may be detected, and the face of the passenger may be displayed surrounded by a red circle symbol. When multiple people are watching the video content, the faces of the multiple people may be displayed surrounded by circles of different colors.

[0115] The device that reads out the recorded information may also display the video content to which the gaze is directed, clearly indicating the position to which the gaze is directed (for example, by attaching a red circle symbol). In the case where a red symbol is used as a method of indicating, when multiple people are watching, it is preferable to use the same color for the circle symbol displayed at the position in the video of the content to which the gaze is directed and the circle symbol surrounding the face of the passenger looking at that position. By displaying in this manner, it is possible to easily analyze where the passenger to whom the circle symbol was attached was looking in the video of the content, for example, where the passenger's attention (also called attention) to the video of the content, such as whether the passenger was looking at a celebrity or a product, was directed. In a situation where multiple circle symbols of different colors are displayed in a position close to the product, it can be determined that the attention to the celebrity is high. In a situation where multiple circle symbols of different colors are displayed in a position close to the celebrity, it can be determined that the attention to the celebrity is high. In addition, it is possible to easily analyze the temporal transition of the attention position in the video of the content. It is preferable to display content targeted at passengers who are looking at the video of the content next.

[0116] (F-11) The processing device 40 may have a function of recording information required for analyzing the relationship between the passenger boarding and alighting status and the video content. Based on the information recorded by this function, the relationship between the passenger boarding and alighting status, such as the opening and closing of doors, and the video content can be analyzed.

[0117] (F-12) The processing device 40 may not perform at least one of the processes of recording the advertising ID together with the camera image and the process of detecting the passenger's line of sight while passengers are getting on and off. Alternatively, the processing device 40 may perform at least one of these processes and record information for identifying the period during which passengers are getting on and off. For example, the processing device 40 may determine that the current time is the period during which passengers are getting on and off based on detection of the opening and closing of doors, detection of station arrival and departure announcements, and detection of guidance information notifying that the doors are open being displayed on the display device 22. The processing device 40 may record the period during which it has been determined that passengers are getting on and off together with the advertising ID so as to be able to identify it.

[0118] (F-13) The processing device 40 may have a function of displaying the image of the content and the image of the inside of the vehicle side by side on the display screen, and displaying the images so that passengers in the vehicle can be associated with the positions in the image of the content that the passengers are looking at. For example, it may be possible to display a mark of the same color at the positions of the faces of the passengers in the vehicle and at the positions in the image of the content that the passengers are looking at. This allows the user to intuitively understand which passengers are looking at which parts of the image of the content.

[0119] When the processing device 40 is displaying the video of the content, it is preferable that the processing device 40 displays marks at the locations in the video that are being looked at by each of the passengers. Since the distribution density of marks becomes higher in the locations that many passengers are looking at, it is possible to know which parts of the video of the content are attracting the most interest of the passengers.

[0120] (F-14) When the display device 22 has multiple display screens, the processing device 40 may have a function of detecting which display screen a passenger is looking at and recording the detection result. Based on the information recorded by this function, it is possible to easily analyze which of the multiple display screens a passenger is looking at. When passengers within the range photographed by one camera 35 can see multiple display screens, it is preferable to perform processing for each of the multiple display screens in the same manner as when one display screen is arranged. For example, when two display screens are arranged above one door and one camera 35 is arranged next to them, it is preferable to record information indicating which display screen a passenger was looking at with the camera 35.

[0121] When one content is displayed across multiple display screens, the processing device 40 should have a function for detecting which display screen the passenger is looking at. Information obtained by this function is useful for finding a display method that will be most appealing when one content is displayed across multiple screens.

[0122] (F-15) The processing device 40 may further acquire, as the evaluation information, at least one of the current time when the content is displayed, the position of the vehicle, the time when the passenger boarded the vehicle and information on the stop, and the attributes of the passenger who is viewing the content. By including this information in the basis for the evaluation of the content, it becomes possible to perform a more detailed evaluation of the content.

[0123] The processing device 40 can obtain information on, for example, when a passenger started to look at the display device 22. For example, information on the time when the passenger started to look at the content, how many minutes had passed since boarding the vehicle, and how many stations had passed since boarding, can be obtained. For example, information such as the passenger operated the smartphone immediately after boarding, but started to look at the content after passing two stations can be obtained. Information on what attributes of passengers and what types of content they were looking at can be obtained. The attributes of passengers looking at the content may include, for example, gender, age, and facial expression (e.g., whether they are smiling or angry). These pieces of information may be obtained, for example, at regular time intervals (e.g., one second). Since these pieces of information do not necessarily need to be obtained in real time, it is preferable to record the image captured by the camera 35 as image data in a recording medium, and to read the image data recorded in the recording medium to obtain these pieces of information.

[0124] Furthermore, since the processing device 40 has a function of detecting the number of people in the vehicle and the line of sight of the passengers based on the image captured by the camera 35, it is possible to obtain information on which part of the screen of the display device 22 the passengers are looking at, based on the direction of the line of sight of the passengers looking at the display device 22 and the information on the installation location of the display screen of the display device 22. It is possible to know which part of the video content the passengers are looking at, based on the video content displayed on the display device 22 at the time the image captured by the camera 35 was acquired and the position on the display screen to which the passengers were looking. For example, it is possible to obtain information on whether the passengers are looking at the product in the video content or the talent appearing in the video content. Based on this information, it is possible to perform a more detailed evaluation of the content. For example, this information can be used as useful information for creating content that is highly appealing to the product.

[0125] [G. Image processing and recording function] Next, the function of performing image processing and recording will be described with reference to FIG. Fig. 8 is a flow chart showing the function of performing image processing and recording. The processing device 40 repeatedly executes the process shown in Fig. 8 during the period from when the power is turned on to when it is turned off. The processing device 40 acquires a moving image captured by the camera 35 (step SD1). After that, the processing device 40 performs image processing on a designated area in the moving image, and records the moving image in an SD card attached to the reader / writer 42.

[0126] The predetermined area to be subjected to image processing may be stored in advance in the storage device 43. The processing device 40 reads information specifying the area to be subjected to image processing from the storage device 43, and performs image processing on the video (step SD2). When an area containing a passenger's face is specified as the area to be subjected to image processing, the processing device 40 detects the passenger's face from the video and blurs the face. When a specific area within the screen of the video is specified as the area to be subjected to image processing, the specified specific area is clipped.

[0127] By recording the passengers' faces in the images with a blurred appearance, the risk of being accused of violating privacy is reduced. Furthermore, by clipping a specified specific area, the amount of image data can be reduced. For example, an area such as the ceiling in which passengers are not captured can be specified as the area to be clipped. This makes it possible to ensure the necessary amount of information about passengers even if the amount of data is reduced. The specified image processing applied to the passengers' faces is preferably image processing that prevents the passengers' faces from being seen or recognized, and may be blurring, clipping, or processing such as reducing the resolution.

[0128] Images taken by the camera 35 inside the vehicle are desirably stored securely so that they cannot be accessed by the general public, as they may violate personal privacy. By using the function of performing image processing that prevents the viewing or recognition of designated areas and recording the images on a recording medium, image processing that makes it impossible to identify individuals is performed, and information that is likely to violate personal privacy is not recorded on the recording medium.

[0129] The "designated area" may be, for example, an area in which the passenger's face is captured. Image processing that prevents the designated area from being seen or recognized may be performed, for example, by clipping, blurring, reducing the resolution, etc. Alternatively, the face may be abstracted and recorded so that the passenger in the image cannot be identified, but the gender and age can be determined.

[0130] The "designated area" may be an area in which passengers are not captured. For example, the ceiling or walls of the vehicle may be designated. By applying processing such as clipping, blurring, or reducing the resolution to this area, the amount of image data can be reduced without reducing the amount of information about the passengers.

[0131] The camera is often installed at a position higher than the eye level of a normal passenger. In this case, if the image shows a part that may embarrass or make a passenger feel uneasy, such as underwear or a body that is hidden by normal clothing, it is advisable to specify the part and apply the above-mentioned image processing to that area.

[0132] [H. Function to process images and output image data] Next, a function of performing image processing and outputting image data will be described with reference to FIGS. 9A and 9B.

[0133] 9A is a flow chart showing a function of recording moving images captured by the camera 35. The processing device 40 constantly executes the process shown in FIG. 9A during the period from when the power is turned on to when it is turned off. The processing device 40 acquires moving images captured by the camera 35 (step SE1). Then, the acquired moving images are recorded on an SD card attached to the reader / writer 42.

[0134] FIG. 9B is a flowchart showing the function of outputting image data. An information terminal such as a personal computer to which an SD card on which a moving image is recorded in step SE2 (FIG. 9A) is attached executes the process shown in FIG. 9B. The processing device of the information terminal has a function of receiving an output command for image data, a function of receiving an instruction as to whether or not to apply image processing to the moving image to be output, and a function of receiving an instruction specifying an object to be subjected to image processing. When an operator inputs an output command for image data to the information terminal, the processing device of the information terminal executes the process of FIG. 9B. Although the hardware configuration of the processing device of the information terminal is not shown, it may be the same as that of a general personal computer. That is, the processing device of the information terminal has an arithmetic processing device exemplified by a CPU, a memory, an operation device, an interface for an external storage medium, a communication module, and an auxiliary storage device. The memory is used as a work area for arithmetic processing. The operation device accepts user operations. For example, a keyboard, a pointing device, or the like is used as the operation device. The interface for an external storage medium is for attaching an external storage medium such as an SD card. The communication module is for communicating (for example, wired or wireless) with an external device. The auxiliary storage device may be, for example, a hard disk drive, and stores the programs, the viewer, and other data. The arithmetic processing unit reads the programs and other data into the memory, and performs various arithmetic processing according to the programs.

[0135] The processing device of the information terminal determines whether or not to apply image processing to the image data to be output based on the input command (step SE5). If image processing is to be applied (step SE5; Y), the image data is read from the SD card and image processing is applied (step SE6). The predetermined image processing to be applied to the passenger's face may be image processing that prevents the passenger's face from being seen or recognized. The image data that has been subjected to image processing is then output (step SE7). If image processing is not to be applied (step SE5; N), the image data read from the SD card is output as is (step SE7). The image data is output by writing it to another SD card, transmitting it to another device via a communication network such as a wired LAN or wireless LAN, or the like.

[0136] Image data that has been processed by setting the target of image processing to the area where the passenger's face is captured is less likely to violate privacy. Image data that has not been processed can be useful information for solving crimes that occur in vehicles or disputes that occur between passengers. For example, face recognition technology can be used to search for wanted criminals or suspects.

[0137] [I. Modification of the function of outputting image data after image processing] (I-1) The processing device 40 may have a function of recording images captured by the camera 35 in a recording device, and a function of reading out the images recorded in the recording device and selectively outputting image data in which a specified image processing has been applied to the passenger's face, or image data in which the specified image processing has not been applied.

[0138] By recording the images captured by the camera 35 in a recording device, the recorded images can be effectively used by public investigative agencies such as the police to solve crimes and other incidents that occur in the vehicle. By outputting image data in which a specified image processing has been applied to the passenger's face, it is possible to provide images that are less likely to infringe on personal privacy. Image data that has not been subjected to image processing is useful information for solving incidents that occur in the vehicle. For example, it is useful information for solving crimes such as molestation, pickpocketing, and baggage theft that occur in the vehicle.

[0139] The predetermined image processing applied to the passenger's face may be image processing that prevents the passenger's face from being seen or recognized, for example, processing that makes it impossible to identify an individual from the image of the face part, such as clipping, blurring, or reducing the resolution.

[0140] The image data output function may include, in addition to outputting image data from an image output terminal, a function for outputting image data via a wireless LAN, and a function for writing image data to an external storage medium such as an external storage medium. When outputting image data from an image output terminal, the above-mentioned predetermined image processing may be performed. When outputting image data via a wireless LAN, it may be possible to specify whether or not to perform the predetermined image processing depending on the attributes of the destination. For example, when the destination is an investigative agency such as the police, where confidentiality of information is expected to be maintained, it is preferable not to perform the predetermined image processing. The processing device may have a function for registering a list of destinations to which image processing is not performed.

[0141] Regarding (I-2) and (I-1), the processing device 40 may have a function for specifying an object or range within an image to be subjected to image processing. For example, a passenger's face or the like may be specified as an object to be subjected to image processing. For example, a specific area or the entire area within the image screen may be specified as the range to be subjected to image processing.

[0142] (I-3) When the processing device 40 records image data in an external storage medium without performing image processing, it is preferable to record the image data in a format that requires a password to read the recorded image data. Alternatively, when recording image data in an external storage medium without performing image processing, it is preferable to format the external storage medium in a special format so that the image data in the external storage medium cannot be read by a general reading application. This prevents the image data from being read by an unspecified person.

[0143] When image data is recorded on an external storage medium formatted in a common format without a password, the processing device 40 may automatically perform a predetermined image processing, thereby protecting personal privacy.

[0144] An external storage medium and its reader / writer may be used as a recording device for recording images captured by the camera 35. A viewer may be provided for playing back images recorded on the external storage medium on a terminal such as a personal computer.

[0145] [J. Passenger movement route detection function] Next, the passenger movement route detection function will be described. The processing device of the information terminal has a function of allowing a specific passenger to be specified from passengers captured in the image recorded on the SD card. When the processing device of the information terminal detects that a specific passenger has been specified, the processing device of the information terminal performs image analysis of the image data read from the SD card to search for the specified specific passenger. When a specific passenger is detected, the time of the video image is advanced to detect the passenger's movement route within the vehicle and between vehicles. The detected movement route is displayed as an image on the display screen of the information terminal. The processing device further has a function of detecting the movement route within the vehicle from boarding to disembarking for each passenger who boarded the vehicle based on the image captured by the camera 35, and recording information on the boarding point, the detection result of the movement route within the vehicle, and information on the disembarking point.

[0146] This function can obtain information on which station a specific passenger boarded the train, how he or she moved within and between the cars, where he or she stood or sat, and at which station he or she got off. It is preferable to install a camera 35 in each car of a train that includes multiple cars, and record the above information for each car. Furthermore, it is preferable that the processing device has a function of displaying an image taken by the camera 35 and allowing a specific passenger to be specified from the image. When a specific passenger is specified, it is preferable that the processing device has a function of detecting and displaying the route of movement within and between the cars of the specified passenger from the time of boarding to the time of disembarking. This information can be effectively used for planning the number of cars, train operation plans, etc., and station platforms. Furthermore, when there is a large variation in the congestion rate between cars, it can be used as useful information for planning a strategy to guide passengers to relatively empty cars based on the passenger's movement route within the car.

[0147] [K. Function of displaying images captured by camera 35 on display device 22 inside the vehicle] Next, a function for displaying an image captured by the camera 35 on the display device 22 inside the vehicle will be described. The processing device 40 has a function of displaying an image captured by the camera 35 or an image obtained by processing the image, instead of or in combination with the process of displaying the content on the display device 22, on the display device 22. The processing device 40 also has a function of requesting and inputting an instruction (e.g., a command) from the operator as to whether to display the image captured by the camera 35 as is, or to display an edited image, and, if editing is to be performed, what kind of editing should be performed.

[0148] When the processing device 40 is instructed to display the image captured by the camera 35 as is, the processing device 40 causes the image captured by the camera 35 to be displayed on the display device 22 almost in real time. When the processing device 40 is instructed to process and display the image captured by the camera 35, the processing device 40 processes the image captured by the camera 35 as instructed and causes the image to be displayed on the display device 22 almost in real time.

[0149] Train passengers are anxious about how they are photographed by the camera 35 inside the train. If an actual photographed image is displayed on the display device 22 inside the train, passengers can see how they are photographed, and this eases their anxiety. Displaying an image that has been processed from an actual photograph can provide various additional effects compared to displaying the image as is.

[0150] The display device 22 that displays the image captured by the camera 35 should be the display device 22 that is placed near the camera 35. In particular, it is preferable to display the image on the display device 22 at a position where passengers within the range captured by the camera 35 can see it.

[0151] The processing of the image may be such that the area other than the passengers (background) is replaced with another image, rather than an image of the inside of the electric vehicle. In particular, it is preferable to display a situation that would not occur in reality on the screen of the display device 22. For example, if an image of a pond in which carp are swimming taken from above is used as the background, a situation in which a passenger is standing on the pond is virtually reproduced. This is expected to reduce the passenger's anxiety that he or she is being photographed. Furthermore, if there are children among the passengers, it is preferable to display an image that the child is likely to be interested in and enjoy next to the child on the screen of the display device 22. For example, it is preferable to display an animated image of a popular character next to the child on the display screen. This can make the child feel happy.

[0152] It is advisable to alternately display the images captured by the camera 35 inside the train and the video content prepared in advance, for example, at a fixed cycle. This allows the images inside the train and the video content to be displayed on a single display device 22.

[0153] As a modified example, a display device 22 corresponding to each car may be installed in the driver's compartment so that the driver can check the state of the passengers in the car in real time. The driver can easily grasp the degree of congestion in each car. By grasping the degree of congestion, an announcement can be made to guide passengers to cars that are relatively empty.

[0154] If an item is left behind on a luggage rack or other place inside the train, the train attendant can easily check whether the item is there or not by looking at the image, which can shorten the time it takes to retrieve the lost item at the station.

[0155] [Luggage theft prevention function in trains] Next, the baggage theft prevention function in the train will be described with reference to Figures 10A and 10B. The baggage theft prevention function is a function that sets the locking mechanism to either a locked state or an unlocked state based on the characteristics of passengers who visit the baggage storage area in an image captured by camera 35.

[0156] 10A and 10B are perspective views of a locking mechanism provided in a baggage area in a train in an unlocked state and a locked state, respectively. The locking mechanism includes two bars 55, 56. In the unlocked state (FIG. 10A), a gap is provided between the two bars 55, 56. This allows one bar 55 to pass through a handle 51 of luggage 50 such as a suitcase. In the locked state (FIG. 10B), a closed loop is formed by the two bars 55, 56, making it impossible to remove the handle 51 of the luggage 50 from the bars 55, 56. The baggage area is located within the angle of view of the camera 35 in the train. This allows the owner of the luggage 50 to be photographed by the camera 35 while operating the locking mechanism.

[0157] An owner who intends to place baggage 50 in the baggage storage area passes bar 55 through handle 51 of baggage 50, and then operates bars 55, 56 to create a closed loop state. When the locking mechanism detects that bars 55, 56 have been placed in the closed loop state, it transitions to a locked state. The processing device 40 has a function of detecting that the locking mechanism has transitioned to the locked state.

[0158] When the processing device 40 detects that the locking mechanism has been locked, it detects the passenger who has placed the luggage 50 in the luggage storage area by analyzing the image captured by the camera 35, identifies the passenger using facial recognition technology, and stores the storage location where the luggage 50 has been placed in association with information identifying the passenger (owner) who placed the luggage.

[0159] The processing device 40 analyzes the images captured by the camera 35 and performs facial recognition of passengers who are near a storage location for luggage. When the processing device 40 detects that a passenger associated with the storage location is near the storage location, the processing device 40 unlocks the locking mechanism of the storage location. This allows the owner of the luggage to retrieve their luggage from the storage location.

[0160] If the owner of the baggage 50 is not detected near the storage area, the processing device 40 keeps the locking mechanism in the locked state. This makes it possible to prevent theft of the baggage. Passengers who leave their baggage in the baggage storage area do not need to carry a key for the locking mechanism, so they do not need to worry about losing the key. In addition, passengers do not need to use the key to open or close the baggage, which increases convenience.

[0161] 11A and 11B are respectively a front view and a plan view of another locking mechanism provided in a luggage storage area in a train. The luggage storage area is divided into a plurality of storage areas 60 by partitions 61. A swing door 62 is provided for each storage area 60. In the unlocked state, the swing door 62 can be freely opened and closed, and in the locked state, the swing door 62 remains closed and cannot be opened. A lock button 63 is provided for each storage area 60.

[0162] The owner of the luggage 50 opens the swing door 62, stores the luggage 50 in the storage location 60, and presses the lock button 63. The processing device 40 has a function of detecting that the lock button 63 has been pressed. When detecting that the lock button 63 has been pressed, the processing device 40 identifies the passenger in front of the storage location 60 by face recognition technology. The processing device 40 associates the storage location where the luggage 50 is placed with information that identifies the identified passenger (owner) and stores the information. After locking, when the processing device 40 detects that the owner of the luggage is near the storage location, it unlocks the lock mechanism of the storage location. Even when the lock mechanism shown in FIG. 11A and FIG. 11B is used, the same effect as when the lock mechanism shown in FIG. 10A and FIG. 10B is adopted can be obtained.

[0163] It is preferable that the processing device 40 has a function of relocking if the luggage is not taken out of the storage location within a certain time period after the locking mechanism is unlocked. If the luggage owner is detected and unlocked simply by passing in front of the storage location, the processing device 40 will unlock it again, allowing the luggage to be stored safely.

[0164] This function makes it more difficult for luggage to be stolen. The luggage storage area may be located anywhere in the train car, and may be located in the passenger compartment or on the deck. It is preferable to locate the luggage storage area in a location that is out of sight of passengers when they are seated in their seats in the train car. Since theft can be prevented even if luggage is placed in a location out of sight of passengers, the effect of the above function is further enhanced. Furthermore, by locating the luggage storage area away from the seats, there is more freedom in the layout of the seats, luggage storage area, entrances and exits, etc. in the train car.

[0165] Passengers who leave their luggage in the luggage storage area do not need to carry a key for the locking mechanism, so they do not need to worry about losing the key, etc. In addition, passengers do not need to open or close the luggage using a key, which increases convenience.

[0166] [M. Modification of the function to prevent baggage theft in trains] (M-1) The baggage storage area may be a space within the vehicle cabin. The baggage storage area may be a space between the rear seat of a vehicle such as a Shinkansen and the wall behind the rear seat. The processing device 40 may perform the above-mentioned processing by treating a passenger using a specific seat among the seats in the vehicle as the owner of the baggage. The processing device 40 may determine that a passenger using a seat at the rear of the vehicle is the owner of the baggage.

[0167] (M-2) The configuration of the locking mechanism is not limited to the above, but it is preferable not to use a mechanism that stores luggage in a sealed space like a locker or a safe to prevent a third party from removing the luggage. For example, a mechanism that allows the owner of the luggage or a third party to see the luggage placed in the luggage storage area but does not allow the luggage to be easily removed from the luggage storage area may be adopted as the locking mechanism. For example, a bar that can be passed through the handle of the luggage (e.g., a suitcase) may be provided in the luggage storage area. A locking mechanism may be provided that makes it impossible to remove the handle from the bar when the bar is passed through the handle of the luggage and locked. In addition, a swing door may be provided between the passage and the luggage storage area, and a mechanism that does not open the swing door in the locked state may be adopted as the locking mechanism. The upper part of the storage area is open, and it is possible to remove the luggage through the upper part of the swing door, but the effect of deterring theft of luggage can be expected.

[0168] (M-3) When the owner of the luggage places the luggage in the luggage storage area and locks the locking mechanism, the processing device 40 detects that the luggage has been locked and may store the luggage and the owner in association with each other based on an image of a passenger standing in front of the luggage in the luggage storage area at that time. The owner of the luggage may be identified using, for example, face recognition technology. When a person stands in front of the luggage storage area while the luggage is locked, the processing device 40 may determine whether the passenger standing in front of the storage area is the owner of the stored luggage based on the image of the person, and if the passenger is the owner of the stored luggage, may unlock the locking mechanism.

[0169] (M-4) The processing device 40 may have a function of detecting passengers standing in front of a storage location who direct their gaze toward a specific location, for example, the location where the camera 35 is installed, and determining whether or not the passenger is the owner of the luggage. Alternatively, the processing device 40 may have a function of unlocking the locking mechanism when it detects, by analyzing an image taken by the camera 35, that the owner of the luggage standing in front of the storage location has started to take out the luggage. This function makes it possible to avoid a situation in which the locking mechanism is mistakenly unlocked when the owner of the luggage passes in front of the storage location for a purpose other than taking out the luggage.

[0170] (M-5) It is further preferable that the processing device 40 has a function of providing an alarm device at the storage location and notifying the owner of the luggage that the locking mechanism has been locked and unlocked when the locking mechanism is switched from the locked state to the unlocked state and when the locking mechanism is switched from the unlocked state to the locked state. As the alarm device, for example, a device that allows the owner of the luggage to notice the state change visually or audibly, such as a light-emitting device such as an LED or a sound-producing device such as a speaker, can be used. This allows the owner of the luggage to confirm that the locking mechanism has been locked when he or she places the luggage in the luggage storage location. Also, the owner of the luggage can confirm that the locking mechanism has been unlocked before taking out the luggage.

[0171] (M-6) The processing device 40 may have a function to relock the locking mechanism if the luggage is not taken out of the storage location within a certain time after unlocking the locking mechanism. If the luggage owner is simply passing in front of the storage location and the lock is unlocked, the luggage can be safely stored by unlocking it again.

[0172] [N. Function to display content on station display devices] Next, a function for displaying content on a display device in a station will be described with reference to Figs. 12A and 12B.

[0173] 12A and 12B are schematic diagrams showing a train 70 on a track, a station display device 72 installed on a station platform 71, and a queue of passengers 75. The train 70 includes six interlocking cars. The train 70 passes through a number of stations in order, and is operated such that the cars stop at prescribed stopping positions on the platform 71 at each of the stations.

[0174] A station display device 72 is installed at each door stop position of each train car on a station platform 71. Passengers waiting in line at the door stop positions can view the video content displayed on the station display device 72. A processing device 73 of the station display device 72 selects the content to be displayed on the station display device 72 at each stop position.

[0175] A photographing device 23 is mounted on each carriage of the train 70. The processing device of the photographing device 23 has a function of counting the number of passengers and calculating the congestion rate of each carriage by analyzing the images taken by the camera 35. The processing device of the photographing device 23 transmits information relating to the calculated congestion rate of each carriage to a processing device 73 installed at the station. This transmission uses public lines, dedicated lines, etc., including wireless transmission paths. The congestion rate is classified into four levels, for example, "empty," "normal," "crowded," and "full."

[0176] The station processing device 73 has a function of selecting, for each stop position, from a plurality of contents, content to be displayed on the station display device 72 arranged corresponding to the stop position, based on the received information indicating the congestion rate for each car. The processing device 73 displays content that makes passengers want to move from a stop position of a relatively crowded car to a stop position of a relatively empty car, that is, content that is likely to attract passengers' interest.

[0177] 12A shows an example of a line of passengers 75 before the selected content is displayed, and FIG. 12B shows an example of a line of passengers 75 when the selected content is displayed. When passengers 75 move to a stop position where interesting content is displayed, the line of passengers 75 waiting at the stop position of a relatively empty car becomes longer. This prevents, for example, a particular car near an escalator from being crowded while the first or last car is empty, and it is possible to equalize the congestion rate between cars.

[0178] As a modified example, the station processing device 73 may cause the station display device 72 at each stop position to display the congestion level of the relevant vehicle, as well as the congestion levels of other vehicles.

[0179] The image capturing device 23 may have a function of transferring information indicating the degree of congestion in each carriage to a server. The server may have a function that allows passengers to access the information indicating the degree of congestion in real time using a mobile information terminal such as a smartphone. Furthermore, the server may have a function of providing information indicating the degree of congestion in the past. Train users can decide which time of day to commute and which carriage to use by referring to the degree of congestion in the past.

[0180] By appropriately selecting the content to be displayed on the station display device 72, it is possible to guide passengers to stopping positions corresponding to cars that are relatively empty. This makes it possible to level out the congestion rate. Examples of trains that include multiple cars include electric trains, monorails, and linear motor cars.

[0181] For example, each time a train departs from a stop (here, a station), a processing device may calculate the congestion rate based on an image captured by a camera 35 installed in each train. A processing device mounted on the train may calculate the congestion rate and transmit the result by wireless communication to a station display device 72 installed at the next stop, and the processing device of the station display device 72 may display content corresponding to the received congestion rate. For example, the station display device 72 for each stop position may display information indicating the congestion rate, congestion level, etc. of the train stopping at that position using numbers, colors, figures, etc.

[0182] It is preferable to guide passengers to a location where a relatively empty train will stop by displaying content that is likely to attract passengers' interest on station display devices 72 arranged at a location where an empty train or other vehicle will stop. For example, if there are many children among the passengers waiting for the next train, etc., it is preferable to use an image of a popular character that is likely to attract children's interest as content that is likely to attract passengers' interest. It is preferable that the processing device has a function of displaying boring commercial videos on station display devices 72 at locations where crowded trains will stop, and displaying commercial videos featuring popular idols on station display devices 72 at locations where empty trains will stop.

[0183] The processing device may have a function of analyzing the number of passengers, their attributes, whether they are carrying luggage, etc., based on an image of passengers lined up on the platform of a station, and changing the content to be displayed on the station display device 72 based on the analysis result. For example, based on the attributes of passengers lined up at the stop position of a crowded train, content that passengers are likely to be interested in may be selected, and a message may be displayed on the station display device 72 at the stop position of the crowded train to notify that the content that passengers are likely to be interested in is displayed on the station display device 72 at the stop position of the XX train. This can enhance the effect of guiding passengers to the stop position of an empty train. Alternatively, the processing device may have a function of displaying a short video with a narrative on the station display device 72 at the stop position of the crowded train, and displaying a message such as "The rest will be displayed on the station display device 72 at the stop position of the XX train." This can guide passengers from the stop position of the crowded train to another stop position.

[0184] A display device may be arranged at a bus stop for a route bus, and when a crowded bus stops at the bus stop and the next bus is relatively empty, the processing device may have a function of displaying on the display device content that informs passengers that the next bus is available and motivates passengers to wait for the next bus. For example, the processing device may have a function of notifying passengers waiting at the bus stop of the estimated time until the next bus arrives, the number of vacant seats on the next bus, and content to be displayed on the display device inside the next bus vehicle. On the next bus, content that is likely to interest passengers waiting at the bus stop may be displayed. For example, if there are many junior high school and high school students among the passengers waiting at the bus stop, a message such as "sample exam questions and answers will be displayed on the display device of the next bus" may be displayed on the display device at the bus stop.

[0185] [O. Other configuration examples of the imaging device 23] (O-1) Figure 13 is a perspective view of the photographing device 23 (Figures 2A and 2B) according to another configuration example with the cover removed. The following mainly describes the differences from the photographing device 23 shown in Figures 2A and 2B.

[0186] An opening 37 is provided in the housing of the photographing device 23. In this configuration example, the opening 37 is provided on the side of the recording unit main body 34. The opening 37 is a portion through which the external storage medium passes when the external storage medium is attached to the reader / writer 42 and when the external storage medium is removed from the reader / writer 42. An SD card is attached to the reader / writer 42 (FIG. 4) disposed in the recording unit main body 34 through the opening 37. A rotating cover 38 is rotatably attached to the recording unit main body 34. By rotating the rotating cover 38, the opening 37 can be switched between a closed state and an open state.

[0187] The direction in which the SD card is inserted and removed is parallel to the circuit board housed in the recording unit main body 34. This circuit board is approximately parallel to the wall surface when the image capture device 23 is attached to the wall surface of the train. Therefore, the direction in which the SD card is inserted and removed is also parallel to the wall surface of the train. A label for identifying the card is affixed to the SD card. The reader / writer 42 is positioned so that when the SD card is removed from the recording unit main body 34, the label surface faces forward.

[0188] The rotation axis of the rotating lid 38 is parallel to the opening surface of the opening 37 (for example, perpendicular to the SD card when the SD card is inserted or removed), and the lid rotates in a plane parallel to the wall of the train. It is preferable that the rotation axis of the rotating lid 38 is parallel to the wall when the housing is attached to the wall of the train. With such a structure, the rotating lid 38 rotates in a plane perpendicular to the wall of the train, so there is no need to secure space next to the housing for rotating the rotating lid 38. The rotating lid 38 is fixed with a Torx screw when the opening 37 is closed.

[0189] An image data output terminal 44 is disposed at the bottom of the recording unit main body 34. The image data output terminal 44 is attached to the support part 31 in such a position that a cable can be inserted and removed from the front of the recording unit main body 34.

[0190] An interlocking switch 46 is provided which turns on and off the power of the imaging device 23 in conjunction with the opening and closing of the rotary lid 38. When the rotary lid 38 is closed, the interlocking switch 46 is turned on, and when the rotary lid 38 is open, the interlocking switch 46 is turned off.

[0191] A power cable 45 is drawn upward from the recording unit main body 34. The power cable 45 passes through an opening in the support part 31 and an opening in the wall of the train, and is connected to a power adapter attached to the back side of the wall of the train. Power may be supplied to the image capture device 23 from this power adapter. The voltage and AC / DC of the power cable pre-wired in the wall differ depending on the train. The type of power supplied by the pre-wired power cable may be converted to the type of power for the image capture device 23 by the power adapter. This allows the existing power cable to be used as the power source for the image capture device 23, eliminating the need to wire a new power cable.

[0192] To remove the SD card from the photographing device 23, the cover 32 (FIG. 2A) must be removed from the support 31, and the rotating lid 38 of the recording unit main body 34 must be rotated to open the opening 37. In this way, double theft prevention measures are taken with the cover 32 and the rotating lid 38, which enhances the theft prevention effect of the SD card. Since the lid that covers the opening 37 is a rotating type, it is possible to prevent the lid from falling off. It is preferable to fix the rotating lid 38 with a special screw, such as a Torx screw, that cannot be removed with a general Phillips screwdriver or flathead screwdriver. In this way, the theft prevention effect can be further enhanced.

[0193] By setting the relationship between the reader / writer 42 and the wall surface of the train as described above, the label of the SD card can be easily seen immediately after the SD card is removed from the reader / writer 42.

[0194] Removing the SD card while it is still energized can cause the SD card to break down. When the rotary cover 38 is opened, the interlocking switch 46 is turned off, and the power to the imaging device 23 is turned off, eliminating the risk of the SD card being removed from the reader / writer 42 while it is still energized.

[0195] (O-2) The axis of rotation of the rotating lid 38 may be parallel to the opening surface of the opening 37, and the lid may be attached to the case via a hinge to open and close the rotating lid 38. Instead of the rotating lid 38, a sliding lid may be used.

[0196] (O-3) The direction in which the SD card is inserted and removed should be parallel to the circuit board on which the electronic circuit components that realize the functions of the processing device 40 are mounted. A label for identifying the SD card should be affixed to the SD card. When the SD card is removed from the reader / writer 42 with the housing attached to the wall of the vehicle, the label should face forward. This makes it easy to see the label immediately after the SD card is removed from the reader / writer 42.

[0197] (O-4) The photographing device 23 and the processing device 40 may be equipped with a short-term power supply such as an electric double layer capacitor. The processing device 40 may have a function of receiving power from the short-term power supply and performing a termination process for the SD card when it detects that the interlock switch 46 has been turned off. In this way, it is possible to prevent the writing process to the SD card from ending halfway, making it impossible to write new data. The termination process for the SD card is completed in, for example, about 2 to 3 seconds. Therefore, the short-term power supply may have a capacity that can operate the processing device 40 for this amount of time.

[0198] (O-5) The support 31 and cover 32 of the housing may be made of a non-flammable or flame-retardant material. For example, the support 31 may be made of metal. The portion of the cover 32 in front of the camera lens may be made of transparent resin, and the other portions may be made of metal, such as aluminum die-casting. When a GPS antenna or a wireless LAN antenna is disposed inside the housing, the portions near the antennas may be made of a non-metal material, such as resin. The GPS information may be used to correct the time of the real-time clock of the processing device 40.

[0199] (O-6) The board on which the backup battery and the short-term power supply are mounted may be mounted so as to be removable from the main board. This allows the installation company of the imaging device 23, for example, a user such as a railroad company, to easily replace the backup battery and the short-term power supply. The board on which the backup battery and the short-term power supply are mounted may be provided with an area for clearly indicating information regarding the time of replacement, such as the year and month or the date. Such an area may be, for example, an area for attaching a sticker on which the time of replacement is printed, or an area on which handwritten characters can be written. By recording on the board on which the backup battery and the short-term power supply are mounted the time of replacement, the next time of replacement can be easily known. In addition to a structure in which the backup battery and the short-term power supply are mounted on a common board, the backup battery and the short-term power supply may be mounted on separate boards. This structure allows easy replacement of only one of the backup battery and the short-term power supply.

[0200] (O-7) It is preferable that the lower part of the cover 32 is open when the housing is attached to the wall of the vehicle. The board on which the backup battery and the short-term power supply are mounted should be positioned so that they can be replaced through the opening in the cover 32. Providing an opening in the cover 32 is also effective from the standpoint of heat dissipation. It is preferable that the backup battery and the short-term power supply are positioned below the main heat-generating components of the processing device 40. This makes it possible to suppress deterioration of the backup battery and the short-term power supply due to heat.

[0201] (O-8) The side of the cover 32 that faces upward when the housing is attached to the wall of the vehicle should preferably be inclined downward toward the front. This makes it difficult for passengers in the vehicle to put their hands on the upper side of the cover 32.

[0202] (O-9) The cover 32 may be fixed to the support portion 31 by means of a screw. The screw may be provided with a fall prevention structure so that the screw will not fall off the cover even if it is completely loosened. For example, with the screw passing through the through hole of the cover, a nylon washer or the like may be fitted to the screw to prevent it from falling out, and the cover may be sandwiched between the head of the screw and the washer.

[0203] [P. Function to play back video recorded on SD card] Next, an image reproducing device that reproduces moving images recorded on an SD card will be described. Although the hardware configuration of the image reproducing device is not shown, it may be the same as that of a general personal computer. That is, the image reproducing device may have an arithmetic processing device exemplified by a CPU, a memory, an operation device, an interface for an external storage medium, a communication module, and an auxiliary storage device. The memory is used as a work area when performing image reproducing processing. The operation device accepts user operations. For example, a keyboard, a pointing device, etc. may be used as the operation device. The communication module communicates with an external device (for example, wired or wireless). The auxiliary storage device includes, for example, a hard disk device, etc., and stores programs, a viewer, and other data. The arithmetic processing device reads the viewer and other necessary programs into the memory, and performs various arithmetic processing according to the programs.

[0204] This image playback device has a function of reading video data captured by cameras 35 attached to each of the multiple cars of the train and recorded on an SD card, and storing the data in a storage device. At this time, the image playback device associates the identification information of the camera 35 recorded on the SD card with the video image data. The image playback device has a function of acquiring the train name, car number, position information of the camera 35 within the car, etc. from the identification information of the camera 35. The image playback device has a function of searching a database that contains the identification information of the camera 35 and information about the train on which the camera 35 is mounted, and acquiring this information.

[0205] The image reproducing device has a function of displaying information on the coupling configuration of the train's multiple cars and the installation positions of the cameras 35 inside the cars on a display screen, and allowing a specific camera 35 to be selected from the information displayed on the display screen. When one camera 35 is selected by the operator, the image reproducing device has a function of displaying an image captured by the selected camera 35 on the display screen.

[0206] This function makes it possible to easily specify the camera 35 that captured the video image that you want to check and play it back.

[0207] [Q. Function to link the sensor detection results with video images captured by camera 35] Next, a function for linking the detection result of the sensor with the video image captured by the camera 35 will be described.

[0208] Various sensors such as a three-axis acceleration sensor, a gyro sensor, a temperature sensor, a gas sensor, etc. are attached to the train cars. The processing device of the photographing device 23 has a function of recording the sensor detection information on an SD card in association with the data of the image photographed by the camera 35.

[0209] The image reproducing device has a function of displaying the image recorded in the SD card and the sensor detection information on the display screen in association with each other. For example, the sensor installed in the vehicle is a three-axis acceleration sensor or a gyro sensor, and the image reproducing device synchronizes the detection results of these sensors with the moving image and simultaneously displays them on the display screen. This allows information on how the passengers in the vehicle behave when a certain acceleration is applied to the vehicle to be obtained. In particular, it is preferable to attach a three-axis acceleration sensor or a gyro sensor to different locations on one vehicle. The behavior of the vehicle can be analyzed from the detection results of these sensors. From the behavior of the vehicle, abnormalities in the track condition, abnormalities in the vehicle condition, abnormalities in operation, etc. can be grasped. From the image taken by the camera 35, information on how the passengers in the vehicle behave when a certain acceleration is applied to the vehicle can be obtained. In addition, the behavior of the vehicle obtained from the sensor detection result can be reproduced by a driving simulator or the like, so that the driver can experience an impact equivalent to that received by the passengers. The driver can usually only experience the behavior of the driver's seat of the leading vehicle. When a driver experiences how his or her driving operations are reflected in the vehicle's behavior and how the passengers feel, this motivates the driver to drive in a way that provides a comfortable ride.

[0210] For example, the sensor installed in the vehicle is a temperature sensor, and the image reproducing device synchronizes the detection result of the temperature sensor with the image captured by the camera 35 and displays them on the display screen at the same time. This makes it possible to know the relationship between temperature fluctuations and opening and closing of doors, raising and lowering of shades, increase and decrease in the number of passengers, etc. This information is useful for finding measures to maintain a more comfortable temperature inside the vehicle.

[0211] For example, it is advisable to include a temperature sensor among the sensors installed in the vehicle. Based on the detection result of the temperature sensor and the images taken by the camera 35, it is possible to know the relationship between temperature fluctuations and door opening / closing, raising / lowering of shades, increase / decrease in the number of passengers, etc. This information is useful for finding measures to keep the temperature inside the vehicle more comfortable.

[0212] For example, the detection result of the temperature sensor may be displayed in the driver's compartment so that the train driver can immediately see it. Based on the temperature inside the train, the driver can make decisions such as whether to make an announcement encouraging the driver to open or close the shades, or whether to change the temperature setting of the air conditioner. Based on the open / closed state of the doors and the temperature change, the driver can determine whether the temperature change is due to the opening or closing of the doors.

[0213] For example, the sensor installed in the vehicle is a thermal camera, and the image reproducing device has a function of identifying passengers who had a high fever by comparing the image taken by the camera 35 with the temperature distribution obtained by the thermal camera. Furthermore, the image reproducing device has a function of searching for, for example, the route taken by a passenger with a high fever in the vehicle based on the detection result of the thermal camera. This information is an important clue for searching for, for example, the route taken by a passenger with a high fever in the vehicle based on the detection result of the thermal camera. For example, it is possible to search for the route taken by a passenger with a high fever in the vehicle based on the detection result of the thermal camera. For example, it is possible to search for the route taken by a passenger with a high fever in the vehicle by comparing the image taken by the camera 35 with the temperature distribution obtained by the thermal camera. It is possible to identify passengers who had a high fever. This information is an important clue for searching for, for example, the route taken by a person infected with a dangerous virus.

[0214] For example, a gas sensor may be included in the sensors installed in the vehicle. Dangerous gases may be detected based on the detection results of the gas sensor. When a dangerous gas is detected, the processing device may immediately sound an alarm in the driver's cabin. When the driver notices the alarm, it becomes possible to take appropriate measures before the damage becomes too great.

[0215] An operator of the image replay device can easily select the camera 35 that captured the image he or she wants to see. The functions of the image replay device may be realized by an information terminal such as a personal computer and an application such as a viewer installed on the information terminal. The data of the images captured by the camera 35 may be acquired via an external storage medium such as an SD card or a communication network such as a wireless LAN. Information indicating the installation location of the camera 35 in the vehicle, the device ID of the camera 35, etc. may be recorded in the information for recording the image data in the external storage medium.

[0216] The viewer may have a function of storing the device ID of the camera 35 and information identifying the vehicle in which the camera 35 is installed, for example, information such as "car No. 7 of Nozomi 100," in association with each other. The terminal in which the viewer is installed may have a function of importing image data recorded in multiple external storage media, allowing the viewer to access any image data. When the operator of the viewer performs an operation to specify a vehicle, the viewer may display the installation locations of the multiple cameras 35 installed in the specified vehicle on the display screen. For example, the vehicle may be displayed as a plan view, and a mark representing the camera 35 may be displayed at the installation location of the camera 35 in the plan view.

[0217] The viewer may have a function for displaying an image captured by a camera 35 on the display screen when the viewer operator selects one of the multiple cameras 35 displayed. The image may be displayed with embedded text or graphics containing information about the time the image was captured, vehicle identification information, and the position of the camera 35 within the vehicle. The viewer may further have a function for saving the image being displayed as a single video file. This video file can be played back using a general video playback device, and may be used for a variety of purposes, such as employee training.

[0218] The viewer should have the function of performing skeletal analysis of passengers from images of passengers inside the vehicle and displaying their bones (also called skeletons). The bone display makes it easier to recognize the posture of passengers. For example, it becomes easier to recognize various postures, such as whether they are holding on to a strap, holding on to a bar, or not holding on to anything. This makes it possible to obtain information about the preferred position and height of the strap, the preferred position and orientation of the bar, etc.

[0219] Furthermore, by associating the image captured by the camera 35 with the sensor detection information, it is possible to obtain information that cannot be obtained only from the image captured by the camera 35 or only from the sensor detection information acquired by the sensor. The sensor detection information is preferably recorded in chronological order. The cameras 35 are preferably disposed near each of the multiple doors of the vehicle. Furthermore, the multiple cameras 35 are preferably disposed in a zigzag pattern in a plan view near the right door and the left door.

[0220] [R. Communication function with server] Next, a communication function between the photographing device 23 mounted on the train and an external server will be described. The photographing device 23 has a parent function, and the server has a child function. In other words, communication cannot be initiated from the photographing device 23 and not from the server. The photographing device 23 has a function of recording video captured by the camera 35 on an SD card, and of constantly transmitting video data to the server (hereinafter referred to as a "constant transmission function").

[0221] Furthermore, the photographing device 23 has a function (hereinafter referred to as a "specific time transmission function") of transmitting moving image data to the server only at specific times, such as in an emergency when recording to the SD card is no longer possible or when an instruction to upload moving image data is received from the server. The photographing device 23 has a function of enabling one of the constant transmission function and the specific time transmission function and disabling the other function.

[0222] As a modified example, the photographing device 23 may have a child function and the tablet terminal may have a parent function. The photographing device 23 constantly records moving images captured by the camera 35 on an SD card, and maintains a standby state for a connection start request regarding communication. When a connection start request is received from the tablet terminal, communication between the photographing device 23 and the tablet terminal becomes possible. When a request to send moving image data is received from the tablet terminal, the photographing device 23 reads the moving image data from the SD card and sends it to the tablet terminal. The operator of the tablet terminal can check whether moving images are normally recorded on the SD card without removing the SD card from the photographing device 23.

[0223] [S. Other Modifications] (S-1) The moving body includes vehicles that carry an unspecified number of passengers, such as trains and buses, vehicles that carry an unspecified number of passengers, such as taxis and hire cars, and elevators that carry an unspecified number of passengers up and down in a building. In particular, a camera may be placed on a moving body in which the same passengers are photographed by the camera for a certain period of time without being replaced, and the passengers are replaced after a certain period of time, and the attributes of the passengers change. For example, a camera may be placed on a moving body that runs through a number of stops, such as a station or a bus stop, in order, such as a train or a bus. When the passengers of the moving body are replaced at the station or bus stop, the selection of content based on the image may be performed again. The acquisition of evaluation information may be performed continuously, for example, at a certain time interval, regardless of the replacement of passengers. The moving body may also be a moving body in which a specified number of passengers or a specified number of passengers are boarded. If the moving object is a train, the stop corresponds to a station, if the moving object is a route bus, the stop corresponds to a bus stop, and if the moving object is an elevator, the stop corresponds to each floor.

[0224] (S-2) In the embodiment, the external storage medium is described as an SD card, but the external storage medium may be an external storage medium other than an SD card (for example, a USB memory). The external storage medium may be a storage medium realized by a cloud computing environment or a storage medium provided outside of other processing devices.

[0225] (S-3) Some of the functions of the processing device described in the above-mentioned embodiment may be realized by the processing device on the camera side, and the remaining functions may be realized by the processing device on the display device side. For example, a function of analyzing an image captured by a camera may be realized by the processing device on the camera side, and a function of selecting content based on the image analysis result may be realized by the processing device on the display device side. In this case, the image analysis result may be transferred from the processing device on the camera side to the processing device on the display device side. The analysis result may include information on the passenger attributes and the ratio of passengers having the attributes. In this case, the processing device on the display device side may select at least one content from multiple contents based on the passenger attributes, etc. Instead of these pieces of information, information on the type of content to be displayed may be transferred from the processing device on the camera side to the processing device on the display device side. In this case, the processing device on the display device side may select content according to the information on the type of content.

[0226] (S-4) Some or all of the functions of the processing device described in the above-mentioned embodiments may be realized by an external server. In this case, an image captured by the camera may be transmitted to the server via a communication network. The communication network may be a public communication network such as the Internet. The communication network may be wired, wireless, or a combination of these. The server may obtain information that is the basis for evaluating the content displayed on the display device, for example, based on the image captured by the camera.

[0227] The scope of the present invention is not limited to the configurations expressly described in the specification, but includes combinations of various aspects of the present invention disclosed herein. The configurations of the present invention for which a patent is sought are specified in the appended claims, but it is the intention of the present inventors to include in the claims any configurations disclosed in this specification that are not currently specified in the claims.

[0228] The present invention is not limited to the configurations described in the above-mentioned embodiments. The components of each of the above-mentioned embodiments and variations may be arbitrarily selected and combined. In addition, any component of each of the embodiments and variations may be arbitrarily combined with any component described in the means for solving the invention or any component that embodies any component described in the means for solving the invention. We intend to obtain rights to these as well through amendments to this application or divisional applications, etc.

[0229] In addition, we intend to obtain rights to the overall design or partial design by converting to a design registration application. The drawings depict the entire device in solid lines, but they include not only the overall design but also partial designs claimed for a part of the device. For example, some parts of the device may be partial designs, and the drawings include some parts of the device as partial designs regardless of the parts. A part of the device may be a part of the device, or a part of that part. We intend to obtain rights to the overall design, as well as partial designs in which any part of the solid line part of the drawing is shown as a broken line part. In addition, the modules, parts, and components inside the device's casing, all of which are shown in the drawings, are subject to trade independently, and we intend to obtain rights to them by converting to a design registration application. [Explanation of symbols]

[0230] 20 Vehicle body 21 Door 22 Display device 23 Imaging Equipment 31 Support part 32 Cover 33 Torx screws 34 Recording unit main body 35 Camera 36 Posture adjustment mechanism 36A Intermediate support member 36B, 36C Rotating shaft 37 Opening 38 Rotating Lid 40 Processing equipment 41 CPU 42 Reader / Writer 43 Storage device 44 Image data output terminal 45 Power Cable 46 Interlocking switch 50 Baggage 51 Handle 55, 56 Bar 60 Storage Location 61 Partition 62 Swing Door 63 Lock button 70 Train 71 Home 72 Station display devices 73 Station display device processing device 75 passengers

Claims

1. A function for acquiring image data taken by cameras mounted on each of a plurality of carriages of a train; a function of displaying information on the coupling configuration of a plurality of carriages of a train and the installation positions of the cameras in the carriages on a display screen, and allowing a user to select a particular camera from the information displayed on the display screen; and a function of displaying an image captured by the selected camera on a display screen. Image reproduction device.

2. A function of acquiring sensor detection information detected by a sensor installed in the vehicle and associated with image data captured by the camera; Further comprising a function of displaying the image captured by the camera and the sensor detection information on a display device in association with each other.

2. An image reproduction device according to claim 1.

3. A program for causing a computer to realize the functions of the image reproduction device described in claim 1 or 2.

Citation Information

Patent Citations

  • Train total information system

    JP2005231431A

  • Imaging device, management device, and video monitoring system

    JP2012217088A

  • Monitoring system and congestion rate calculation method

    JP2013025523A

  • Call registering device of elevator

    JP2013245096A

  • Car information control device

    WO2019150719A1