Teacher data creation system

Abstract

To provide a system which can efficiently create a large number of different patterns of moving images to be used as teacher data.SOLUTION: A teacher data creation system includes: a moving simulation device for generating a traffic flow of a human model; and a moving image creation device for creating a moving image of the human model. The moving simulation device includes: a structure model acquisition unit for acquiring a structure model; a human model acquisition unit for acquiring a human model; and a traffic flow generation unit for generating the traffic flow of the human model on the basis of the structure model. The moving image creation device includes: a traffic flow acquisition unit for acquiring the traffic model from the moving simulation device: a structure model appearance acquisition unit for acquiring the appearance of the structure model; a human model appearance acquisition unit for acquiring the appearance of the human model; and a video creation unit for creating a moving image on the basis of the appearance of the structure model, the appearance of the human model, and the traffic flow.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a system for creating teacher data for machine learning in video.

Background Art

[0002] Estimation techniques using learned models have been developed in various fields. For example, there is a technique for estimating objects and events shown in video.

[0003] Machine learning is performed by inputting a large amount of teacher data into a model. For example, for learning a model for estimating that an abnormality has occurred from video shown on a surveillance camera, it is necessary to collect a large number of videos in which the abnormality is shown. However, since it is difficult to predict when and where an abnormality will occur and the occurrence frequency is also low, it is very difficult to actually shoot scenes where an abnormality has occurred and collect a large number of teacher data.

[0004] In order to efficiently collect videos and images for teacher data, instead of actually shooting and collecting videos and images, a data augmentation technique is known in which videos and images are created by computer graphics (CG) and substituted (Patent Document 1). According to data augmentation, it is possible to efficiently create videos and images that are difficult to shoot in various patterns.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] For creating animations using CG, it is necessary to input various data such as backgrounds, the appearance of characters, the movements of characters and their timings. For teacher data, it is necessary to create a large number of videos in various patterns, and a great deal of effort is required for preparing and inputting the data for creating animations.

[0007] An object of the present disclosure is to provide a system capable of efficiently creating a large number of videos to be used as teacher data.

Means for Solving the Problem

[0008] A teacher data creation system according to the present disclosure is a teacher data creation system including a movement simulation device that generates a movement line of a person model, and an animation creation device that creates a video of the person model, wherein the movement simulation device includes a structure model acquisition unit that acquires a structure model, a person model acquisition unit that acquires a person model, and a movement line generation unit that generates a movement line of the person model based on the structure model, and the animation creation device includes a movement line acquisition unit that acquires the movement line from the movement simulation device, a structure model appearance acquisition unit that acquires the appearance of the structure model, a person model appearance acquisition unit that acquires the appearance of the person model, and an animation creation unit that creates a video based on the appearance of the structure model, the appearance of the person model, and the movement line.

Effects of the Invention

[0009] According to the teacher data creation system according to the present disclosure, since the movement line of a person in a structure is generated by the movement simulation device and a video is created by the animation creation device using the movement line, the movement line of the person input to the animation creation device can be efficiently generated, and a video in which a person moves in a structure can be efficiently created.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Embodiments for Carrying Out the Invention

[0011] Hereinafter, the teacher data creation system according to the embodiments of the present disclosure will be specifically described with reference to the drawings. Note that the embodiments described below are all examples showing comprehensive or specific examples. Also, each figure is a schematic diagram and is not necessarily drawn precisely. Furthermore, in each figure, the same reference numerals are assigned to substantially the same components, and duplicate explanations may be omitted or simplified.

[0012] In the following embodiments, the present disclosure will be described by taking as an example the case where teacher data used for learning an event estimation model is created by CG. This event estimation model estimates an event that occurred at a predetermined location based on an image captured by a camera at that location.

[0013] FIG. 1 is a diagram showing the functional configuration of a teacher data creation system 1 according to the present embodiment. The teacher data creation system 1 includes a movement simulation device 10 and a moving image creation device 20.

[0014] A structure model acquisition unit 12 of the movement simulation device 10 acquires a structure model that models a structure that is the object of moving image creation for teacher data. The structure model is, for example, a model of a railway vehicle, a movie theater, a store, a station, an underground shopping street, a road, etc., and includes information necessary for generating the movement lines of human models, such as walls, columns, entrances and exits, passageways, obstacles, seats, display shelves, etc. included in these structures.

[0015] The person model acquisition unit 14 acquires a person model that models the persons appearing in the video. The person model may include, as information, attributes such as age, gender, height, weight, current position, etc. Further, for example, when creating a video of a scene where an abnormality has occurred inside a railway vehicle, the person model may include attribute information such as a destination (getting-off station), an abnormal actor, a fallen person, etc. The number of person models to be acquired is determined according to the video to be created. For example, if it is a video inside a morning or evening commuter train, the number is large, and if it is a video inside a train during the day, the number is small, and it is automatically set based on predetermined conditions or arbitrarily set by the input of an operator. Further, the person model acquisition unit 14 may sequentially acquire person models over time.

[0016] The flow line generation unit 16 generates and outputs the flow line of a person inside the structure based on the acquired structure model and person model. The flow line generation unit 16 can also generate a flow line randomly based on the structure model and person model, or can generate it according to predetermined conditions. The flow line is generated so as to move at a predetermined speed while avoiding obstacles according to the walls, columns, entrances and exits, passageways, obstacles, seats, display shelves, etc. included in the structure model, and also according to the positions of other person models. For example, in a scene where passengers board a railway vehicle, the flow line is generated from the entrance and exit of the railway vehicle toward an empty seat without hitting other passengers or walls.

[0017] The flow line acquisition unit 24 of the video creation device 20 acquires the flow line of a person inside the structure from the movement simulation device 10. The flow line acquisition unit 24 may sequentially acquire in real time at the time when the movement simulation device 10 generates the flow line, or may collectively acquire the flow lines generated by the movement simulation device 10 during a predetermined period. Further, the generated flow lines may be accumulated in the storage device each time, and the flow lines generated in the past may be read from the storage device and acquired.

[0018] The structure model appearance acquisition unit 26 acquires the appearance of the structure that will be the location for creating the moving image. Here, the appearance includes not only the exterior of the structure but also the internal appearance. That is, it includes the appearance necessary for creating the moving image. For example, in the case of a railway vehicle, the appearance inside the railway vehicle including, for example, walls, seats, entrances and exits, etc. is acquired. Also, in the case of a store, the appearance inside the store including, for example, walls, display shelves, aisles, entrances and exits, etc. is acquired.

[0019] The person model appearance acquisition unit 22 acquires the appearance of the people who appear in the moving image. The appearance of a person is set according to age, gender, height, weight, clothing, etc. respectively. The appearance of a person may be set based on the attributes of the person model acquired by the person model acquisition unit 14, or may be appropriately set by an operator in the moving image creation device 20.

[0020] The moving image creation unit 28 creates a moving image in which a person moves along the traffic line inside the structure based on the appearance of the structure, the appearance of the person, and the traffic line. The moving image may be created by rendering it as an image from a camera (viewpoint) provided at a predetermined location of the structure.

[0021] As described above, since the traffic line of a person is generated by the movement simulation device 10 and the moving image creation device 20 creates a moving image using the generated traffic line, the information input by the operator to the moving image creation device 20 is reduced, and the creation of the moving image can be performed efficiently.

[0022] Figure 2 is a diagram showing an example of the system configuration of the teacher data creation system 1. The movement simulation device 10 and the moving image creation device 20 may be configured as different devices (such as PCs) to exchange information between the two devices via a network or the like, or the functions of the movement simulation device 10 and the moving image creation device 20 may be provided in one device.

[0023] When the movement simulation device 10 and the moving image creation device 20 are configured as different devices, they may be connected by a network to send information from the movement simulation device to the moving image creation device, or the information may be recorded on a recording medium and transferred.

[0024] When connecting via a network, it may be a network with a limited area such as a LAN, or it may be a network that connects remote locations such as the Internet. When using a recording medium, it is possible to use a recording medium such as a USB memory or a DVD-ROM.

[0025] 3 is a diagram showing the hardware configuration of the movement simulation device 10. A CPU 30 is connected to a memory 31, a storage device 32 such as an SSD, a DVD-ROM drive 33, a display 34, a keyboard / mouse 35, a communication circuit 36, and the like.

[0026] The communication circuit 36 ​​is a circuit for connecting to the Internet or a LAN. An operating system 37, a movement simulation program 38, and data 39 such as structure models and human models are recorded in the storage device 32. The movement simulation program 38 cooperates with the operating system 37 to fulfill its functions.

[0027] 4 is a diagram showing the hardware configuration of the moving image creation device 20. A memory 51, a storage device 52 such as an SSD, a DVD-ROM drive 53, a display 54, a keyboard / mouse 55, a communication circuit 56, etc. are connected to a CPU 50.

[0028] The communication circuit 56 is a circuit for connecting to the Internet or a LAN. An operating system 57, a video creation program 58, and data 59 such as structure models and human models are recorded in the storage device 52. The video creation program 58 cooperates with the operating system 57 to fulfill its functions.

[0029] Hereinafter, the present disclosure will be described taking as an example an anomaly detection system that detects the occurrence of an anomaly in a railway vehicle using the teacher data creation system according to the present embodiment. The anomaly detection system is constructed by installing a surveillance camera in the railway vehicle and using a trained model that estimates whether an abnormal state is included in the captured image.

[0030] The learned model is a model that estimates whether the movement of the projected person is different from normal. Inside a moving railway vehicle, it is normal for passengers to hardly move. Therefore, if multiple people start moving at the same time, it is determined that some abnormality has occurred. For example, if an abnormal actor (criminal) appears, it is conceivable that the people around will flee all at once. If a critically ill person collapses, it is conceivable that the people around will rush to help. Therefore, if it is detected that multiple passengers have started moving at the same time, it is determined that an abnormality has occurred. Also, as a model for estimating the occurrence of an abnormality, when a suspicious abandoned object is recognized inside the vehicle, the passengers take an avoidance action of detouring to avoid the abandoned object. In this case, it is possible to determine that an abnormality has occurred based on the frequency of the abandoned object and the avoidance action of the passengers.

[0031] For the learning of the model, a video in which a scene where multiple people start moving at the same time (move in a way different from normal) is projected is used as teacher data. The video is prepared by creating a video in which the movements of people in various patterns are projected using the teacher data creation system of the present embodiment.

[0032] Note that, unlike the present embodiment, when constructing a learned model using a video in which only normal scenes are projected as teacher data, it is possible to detect the occurrence of an abnormality by making an estimation that it does not correspond to a normal scene.

[0033] FIG. 5 is a block diagram of an abnormality detection system 300 using the teacher data creation system 1 and the video created thereby. The movement simulation device 10 simulates the movement of people in a structure, and the video creation device 20 creates a video of the movement of people in the structure using the result. The abnormality detection system 300 causes the model to learn using the video created by the video creation device 20 as teacher data, and detects the abnormality projected onto the camera 304 by the learned model.

[0034] Specifically, the video acquisition unit 301 of the anomaly detection system 300 acquires the video created by the video creation device 20 and stores it in the video database 302. Then, after the preprocessing unit 303 performs the necessary preprocessing on the video, the anomaly detection model learning unit 305 inputs the video into the model for learning.

[0035] When actually performing anomaly detection, the anomaly detection model inference unit 306 inputs the video captured by the camera 304 installed on the structure into the learned model to detect the occurrence of anomalies. If an anomaly is detected, the warning notification unit 307 issues a warning through screen display, alarm sound, etc.

[0036] Next, while referring to the detailed block diagram of the movement simulation device 10 shown in FIG. 6, the processing of the movement simulation device 10 will be described according to the flowcharts of FIGS. 8 to 10. The CPU 30 of the movement simulation device 10 (hereinafter, may be abbreviated as "movement simulation device 10") first acquires a vehicle model (step S1). The operator creates a vehicle model by inputting parameters such as a plan view and overall length of the vehicle into the movement simulation device 10 using a keyboard / mouse 35, etc., and the created vehicle model is stored in the vehicle information database 109.

[0037] FIG. 12 is a diagram showing an example of the input vehicle model. In the vehicle model 60, the space inside the vehicle is defined by the outer wall 61. On the outer wall 61, entrances and exits 62 are defined on the side surface. The position and number of the entrances and exits 62 are appropriately designed according to the vehicle model to be created. Doors 63 for moving to adjacent vehicles are defined at the front and rear ends of the outer wall 61. Seats 64 are defined inside the vehicle. The position and number of the seats are appropriately designed according to the vehicle model to be created. In the illustrated example, it is a six-seater long seat provided along the inside of the outer wall 61.

[0038] Next, the operator inputs the simulation conditions (step S2). FIG. 13 is a diagram showing an example of an input screen 80 for inputting simulation conditions and displaying simulation results. In the simulation area 81, a plan view of the defined vehicle model and the positions of the people inside it (black circles) are shown. Note that the people do not yet exist at the start of the simulation.

[0039] In the condition input area 82, areas for inputting various simulation conditions are set. The operator inputs or selects the train timetable, scenario, abnormal person occurrence rate, fallen person occurrence rate, seat occupancy rate, and male-female ratio of the railway vehicle, respectively.

[0040] FIG. 14 is a diagram showing an example of a timetable. In the timetable, the station name, arrival time, and the number of people boarding the vehicle at the station are set and stored in the timetable database 101. The timetable is selected from data in which the stations and their arrival times are pre-stored according to the route assumed in the simulation. The timetable may be the timetable data of an actual route or the timetable data of a fictional route. The timetable may be set such that the number of boarding people is different at different times at the same station.

[0041] FIG. 15 is a diagram showing an example of a scenario. The scenario is created by the scenario creation unit 102. In the scenario, the occurrence time of an abnormality, the type of abnormality, the attributes of the people involved in the abnormality, and the actions of each person at the time of the abnormality occurrence are set and stored in the scenario database 103.

[0042] In the "abnormal behavior", for example, it is set that an "abnormal person" moves in a predetermined direction from the current position, and a "normal passenger" moves to an adjacent vehicle so as to escape from the "abnormal person".

[0043] In the case of "falling", for example, it is set that "normal passengers" around the person who has "fallen" (within a predetermined range) move to the vicinity of the person who has "fallen".

[0044] Returning to Fig. 13, in the condition input area 82, the abnormal person occurrence rate is the probability that an abnormal person occurs during the running of the railway vehicle. If it is set to 100% here, the scenario and the attributes of the person model are set assuming that an abnormal person will surely occur during the running of the railway vehicle.

[0045] The fallen person occurrence rate is the probability that a fallen person occurs during the running of the railway vehicle. If it is set to 100% here, the scenario and the attributes of the person model are set assuming that a fallen person will surely occur during the running of the railway vehicle.

[0046] The seat occupancy rate is the probability that the person model who has boarded the train heads towards the seat. If it is set to 100% here, it is assumed that all person models will move to the seat as long as there are empty seats.

[0047] The male - female ratio is the ratio for setting the gender of the person model. If it is set to 100% here, all person models are set as male.

[0048] In the elapsed time display area 83, the time in the simulation and the arrival station at that time are displayed.

[0049] In the start time setting area 84 and the end time setting area 85, the start time and the end time of the simulation can be input.

[0050] After inputting various conditions, by operating the execution button 86, the simulation is started.

[0051] Returning to Fig. 8, when the simulation process is started (step S3), the event occurrence unit 104 of the movement simulation device 10 recognizes the occurrence of an event according to the passage of time (step S4). The event occurs at a time determined according to the timetable and the scenario, which is either the arrival at the station, the appearance of a suspicious person (abnormal person), or the occurrence of a fallen person.

[0052] Next, the movement simulation device 10 determines whether the generated event is arrival at a station or not (step S5). If Yes, it simulates the movement of passengers when arriving at the station and calculates the flow lines of boarding and alighting passengers.

[0053] The positioning unit 108 of the movement simulation device 10 starts processing regarding the person inside the vehicle (step S6), and obtains the flow line of the passengers getting off at the arrived station. Regarding the passengers located inside the vehicle, it determines whether they get off at the station based on the attribute information described later (step S7). If step S7 is Yes, it starts and executes a disembarkation thread (step S8). If step S7 is No, it is assumed that the person model does not move. Such determination is repeated until the processing regarding the persons inside the vehicle is completed (step S9).

[0054] FIG. 9 is a diagram showing the processing of the disembarkation thread. The positioning unit 108 of the movement simulation device 10 determines the flow line for the person model to move to the entrance / exit (step S31). When reaching the entrance / exit, the person model is regarded as having disembarked and the data is deleted (steps S32, 33).

[0055] Next, new person models are created according to the number of boarding passengers at the arrived station, and the boarding process is started (step S10). First, the attribute creation unit 105 of the movement simulation device 10 sets the attributes of the newly created person models and saves them in the passenger information database 110 (step S11).

[0056] FIG. 16 is a diagram showing an example of the attributes set for the person models. Each person model is set with an ID number, gender, age, build, type, position inside the vehicle, destination station, and status. Among these, the type indicates whether it is a suspicious person (abnormal person) or other general passengers (ordinary passengers). The position of a person indicates the position inside the vehicle model. The destination station indicates the station towards which the person heads to the entrance / exit when arriving. The status indicates whether the person is standing upright (is at a position other than a seat after boarding), sitting (is heading towards a position with a seat after boarding), fallen, or in need of rescue (is heading towards the fallen person).

[0057] Returning to FIG. 8, the position of the person model newly created in the boarding process is set at the entrance / exit. Also, based on the seat occupancy rate, an attribute is set for the state of the person model to be either "seated" or "standing" (step S11). Then, the seat selection unit 106 of the movement simulation device 10 determines whether there is an empty seat in the vehicle based on the seat information in the seat information database 107 (step S12).

[0058] If there is an empty seat and the state of the person model is "seated" (step S13 is Yes), the seat selection unit 106 of the movement simulation device 10 selects a seat (step S14), makes the person model wait at the seat (step S15), and determines the movement route of the person model from the entrance / exit to the seat.

[0059] FIG. 17 is a diagram showing an example of seat information. The seat information is set as a part of the railway vehicle model as information of a combination of ID, seat position, and whether it is an empty seat, and is stored in the storage device 32.

[0060] On the other hand, if the state of the person model is "standing" (step S13 is No), the positioning unit 108 of the movement simulation device 10 determines to move to a position where the person stands at a place other than the seat, that is, an arbitrary position (step S16), makes the person model wait at the arbitrary position (step S17), and determines the movement route of the person model from the entrance / exit to the position.

[0061] If step S12 is No, the positioning unit 108 of the movement simulation device 10 determines to move to an arbitrary position as standing at a position other than the seat in the vehicle regardless of whether the state of the person model is "seated" or "standing" (step S16), makes the person model wait at the arbitrary position (step S17), and determines the movement route of the person model from the entrance / exit to the position.

[0062] By repeatedly executing steps S11 to S17 for the number of person models boarding, the process for the number of boarding passengers is completed (step S18).

[0063] FIG. 18 is a diagram showing an outline of a simulation when the vehicle model 60 stops at a station and the person models 66 and 67 board from the entrance 62. For the person model 66 in the "seated" state, since there are empty seats in the vehicle, the movement simulation device 10 determines a flow line so that the boarding person model 66 moves toward the empty seat. On the other hand, for the person model 67 in the "standing" state, the movement simulation device 10 determines a flow line so that the person model 67 moves to an area other than the seats in the vehicle. The person models already on board do not move.

[0064] Returning to FIG. 8, when step S5 is No, the movement simulation device 10 starts processing related to the persons in the vehicle as an event other than arrival at the station (step S19), specifically, performs a simulation regarding the appearance of a suspicious person or the occurrence of a fallen person.

[0065] When a suspicious person appears, the movement simulation device 10 sets some of the person models as "suspicious persons" and the rest as "ordinary passengers". The setting may be made when creating the person models or when an event occurs.

[0066] When a fallen person occurs, the movement simulation device 10 sets some of the person models as "fallen persons" and sets the person models within a predetermined distance from the "fallen persons" among the other person models as "rescuers". The setting may be made when creating the person models or when an event occurs.

[0067] When an event occurs, the positioning unit 108 of the movement simulation device 10 determines whether each person model is a target of the event. If it is a target of the event (step S20 is Yes), the movement simulation device 10 starts an event action thread (step S21). If it is not a target of the event (step S20 is No), the processing related to the persons in the vehicle ends here (step S22), and the processing for the next person model is performed.

[0068] FIG. 10 is a diagram showing the processing of an event action thread. The positioning unit 108 of the movement simulation device 10 determines the movement route of the human model so that the human model acts according to the events generated by the human model and the attributes defined by itself (step S34). At this time, when the human model sitting on the seat (the state is "seated") moves, the information is updated assuming that the seat where the human model was located has become an empty seat (step S35).

[0069] The "abnormal person" is set to move from the current position on foot in a random direction. The "normal passenger" is set to run and move from the current position toward the door 63 (connection passage) of the adjacent vehicle.

[0070] FIG. 19 is a diagram showing an overview of the movement of the abnormal person 68 and normal passengers when the abnormal person 68 appears. All normal passengers move at high speed toward the door 63 of the adjacent vehicle so as to move away from the abnormal person 68. Here, high speed is assumed to be running away. The abnormal person 68 moves at low speed. Here, low speed is assumed to be walking.

[0071] The "fallen person" is set to not move on the spot, and the "rescue" is set to move to near the "fallen person".

[0072] FIG. 20 is a diagram showing an overview of the movement of the surrounding passengers when the fallen person 69 occurs. The passengers 71 located within a predetermined distance from the fallen person 69 move to the vicinity of the fallen person so as to run toward the fallen person. The passengers 72 located more than a predetermined distance away from the fallen person do not move as bystanders.

[0073] The movement routes determined in each of the above steps are sent to the animation creation device 20 in real time or in a batch for a certain period of time (step S23).

[0074] In this way, from the start of the simulation process (step S3) to the end of the simulation process (step S24) in FIG. 8, it is continued until the end time of the simulation.

[0075] Next, while referring to the detailed block diagram of the moving image creation device 20 shown in FIG. 7, the processing of the moving image creation device 20 will be described along the flowchart of FIG. 11. First, the CPU 50 of the moving image creation device 20 (hereinafter, may be abbreviated as the "moving image creation device 20") reads and acquires information on the vehicle model including the appearance of the vehicle model in the scene where the moving image is to be created (step S41). The information on the vehicle model may use the information on the vehicle model input to the movement simulation device 10, or may be directly input to the moving image creation device 20. The information on the vehicle model is stored in the vehicle information database 207.

[0076] When the vehicle model has been acquired, start the moving image creation process (step S42).

[0077] FIG. 21 is a diagram showing an example of the operation screen 90 of the moving image creation device 20. "Camera settings" are for setting the position, direction, field of view, etc. of the viewpoint of the moving image to be created, and the moving image is created from the viewpoint of shooting inside the railway vehicle by a camera provided inside the railway vehicle. The camera may be configured to be able to select a pre-prepared camera, or may be configured to be arbitrarily set.

[0078] "Lighting settings" are for setting the lighting inside the railway vehicle, and the moving image is generated with shadows assuming that light hits from the set lighting. The lighting may be configured to be able to select pre-prepared lighting, or may be configured to be arbitrarily set. Specifically, set the position where the lighting is provided, brightness, irradiation direction, etc.

[0079] The moving image creation process can select "real-time processing" that is performed in parallel with the route generation process of the movement simulation device 10 and "offline processing" that is performed after receiving the route after the route generation process of the movement simulation device 10 is completed. In real-time processing, the route generated by the movement simulation device 10 is sequentially received and a moving image is created in real-time. In offline processing, the routes for a predetermined period generated by the movement simulation device 10 are collectively received and a moving image is created offline.

[0080] In the overall display area 91, an image of a railway vehicle and the people inside it is displayed from an overhead view. The camera set in the "Camera Settings" and the lighting set in the "Lighting Settings" are also displayed in the overall display area. In conjunction with the operations of the camera addition button and the lighting addition button, cameras and lighting can be added, and the settings (deletion) of the cameras and lighting can be changed by selecting them in the overall display area 91.

[0081] In the rendering display area 92, a video rendered as if the railway vehicle interior and people were photographed with the set camera is displayed. By changing the camera settings and lighting settings while viewing the rendering display area 92, it may be possible to create a video while arbitrarily adjusting the camera and lighting settings.

[0082] The "start time" and "end time" set the period for which a video is to be created based on the traffic line obtained from the movement simulation device 10. If the "start time" and "end time" are not input, the video may be created for the entire period (the running time of the vehicle).

[0083] After the input of various settings is completed, click the "Execute CG Video Creation" button to execute the video creation process.

[0084] Returning to FIG. 11, the human model placement / deletion unit 201 of the video creation device 20 determines whether it is real-time processing in step S43. If Yes, passenger information including the movement line of people is received (acquired) from the movement simulation device 10 in real time (step S44). If No, the passenger information including the movement line generated by the movement simulation device 10 is collectively read and acquired (step S45).

[0085] Next, the process for each passenger character model is started (step S46). The ID management unit 202 starts a search process (matching process) of the ID list (see FIG. 22) stored in the personal information database 203 as the ID of the passenger information obtained as an already appeared character model, and based on the result, determines whether it is an already appeared character model or a new character model that has not appeared yet (steps S47, S48). In the case of a new character model (when step S48 is Yes), the character model placement / deletion unit 201 creates a new character model and places the character model in the vehicle (step S49). Then, the appearance reflection unit 204 acquires appearance information such as body build and clothing color from the character model database 205 based on the attribute information of the character model and reflects the appearance of the character model (step S50).

[0086] On the other hand, when step S48 is No, it is determined whether the character model has already gotten off (old passenger information) (step S51). If Yes, the character model is deleted (step S52), and the search process of the ID list is terminated (step S53). If No, the process for the ID is terminated.

[0087] Next, when the placement and appearance reflection of the character model are completed, the character model position reflection unit 206 places the character model in the vehicle model stored in the vehicle information database 207 (step S54). When the character model animation application unit 208 sends the information of the vehicle model in which the character model is placed and the information of the traffic line to the animation control unit 209, the animation control unit 209 selects an appropriate animation from the animation database 210 and applies (creates) an animation in which the character model moves (step S55). In this way, steps S47 to S55 are executed for all character models, and the process for each passenger character model is terminated (step S56).

[0088] Next, the camera placement unit 211 places a camera serving as a video viewpoint inside the railway vehicle based on the information stored in the camera parameter database 212 (step S57). Also, the lighting placement unit 213 places lighting necessary for shadow generation based on the information stored in the lighting parameter 214 (step S58). Then, the rendering unit 215 of the video creation device 20 creates a video by performing rendering based on the placement of the camera (step S59), and ends the video creation process (step S60).

[0089] FIG. 23 is a diagram showing a movement simulation created by the movement simulation device 10, an overall display created by the video creation device 20, and the rendered video. In the movement simulation device 10, the movement route of the human model inside the railway vehicle is generated along the time series.

[0090] In the video creation device 20, a video is created based on the movement route generated by the movement simulation device 10 and further based on the interior view of the railway vehicle and the appearance of each human model. The video is created by rendering the video captured by a camera installed at a predetermined position with respect to the created video.

[0091] According to the teacher data creation system of the present disclosure, since the movement route of a person is generated by the movement simulation device and a video is created by the video creation device based on this movement route, the video can be created efficiently. In addition, a large number of videos in the case where various events occur in the structure model can be created efficiently. By making the model learn using the created video as teacher data, a learned abnormal occurrence estimation model that can estimate that an abnormality has occurred can be created efficiently.

[0092] The above embodiment is an example. The structure model can be applied to various things such as stores, movie theaters, underground shopping malls, roads, etc. other than railway vehicles. The events that occur are set according to the target location, and the movement of the human model at that time can also be set arbitrarily.

[0093] The present disclosure includes the following aspects.

[0094] (1) A teacher data creation system having a movement simulation device that generates the movement route of a human model and a video creation device that creates a video of the human model, wherein the movement simulation device includes a structure model acquisition unit that acquires a structure model, a human model acquisition unit that acquires a human model, and a movement route generation unit that generates the movement route of the human model based on the structure model, and the video creation device includes a movement route acquisition unit that acquires the movement route from the movement simulation device, a structure model appearance acquisition unit that acquires the appearance of the structure model, a human model appearance acquisition unit that acquires the appearance of the human model, and a video creation unit that creates a video based on the appearance of the structure model, the appearance of the human model, and the movement route.

[0095] According to the above configuration, since the movement route of a person within a structure is generated by the movement simulation device and a video is created by the video creation device using the movement route, the movement route of the person input to the video creation device can be efficiently generated, and a video of a person moving within the structure can be efficiently created.

[0096] (2) In the teacher data creation system of (1) above, the movement simulation device further includes an event information acquisition unit that acquires event information regarding an event occurring within the structure model, and the movement route generation unit generates the movement route of the human model based on the event information.

[0097] According to the above configuration, since the movement route of a person is generated according to an event occurring within the structure, a video of a person moving in response to the occurrence of various events can be efficiently created.

[0098] (3) In the teacher data creation system of (2) above, the event is the occurrence of an abnormality or the stopping of a railway vehicle at a station.

[0099] According to the above configuration, since the movement route of a person is generated in response to the occurrence of an abnormality or a station stop, a video in which a person moves in response to the occurrence of a specific event can be efficiently created.

[0100] (4) In the teacher data creation system of (1) above, the movement route generation unit is a teacher data creation system that generates a movement route so as to head in a predetermined direction or an unspecified direction within the structure model.

[0101] According to the above configuration, since the movement route of a person within a structure is generated so as to head in a predetermined direction or an unspecified direction, a video in which a person moves in various directions can be efficiently created.

[0102] (5) In the teacher data creation system of (1) above, the person model is classified as an abnormal person or a general person, and the movement route generation unit is a teacher data creation system that generates the movement route of the general person based on the current position of the abnormal person.

[0103] According to the above configuration, since the movement route of a person within a structure is generated based on the current position of an abnormal person, for example, a video in which surrounding people escape in a direction away from the abnormal person can be efficiently created.

[0104] In addition, the present disclosure includes the following aspects.

[0105] (6) A movement simulation device including a structure model acquisition unit that acquires a structure model, a person model acquisition unit that acquires a person model, and a movement route generation unit that generates the movement route of the person model when an event occurs in the structure model based on the structure model and the person model.

[0106] According to the above configuration, since the movement route of the person model is determined based on the event occurring in the structure, the movement route of the person model corresponding to various events can be efficiently generated.

[0107] (7) In the movement simulation device of (6) above, the structure model is a railway vehicle model.

[0108] According to the above configuration, it is possible to efficiently generate the flow lines of the human model according to various events occurring inside the railway vehicle.

[0109] (8) In the movement simulation device of (7) above, the event is an abnormality that occurred inside the railway vehicle model, and the flow line generation unit is a movement simulation device that generates the flow line of the human model when the abnormality occurs.

[0110] According to the above configuration, it is possible to efficiently generate the flow line of the human model when an abnormality occurs.

[0111] (9) In the movement simulation device of (7) or (8) above, it has a boarding / alighting number acquisition unit that acquires the number of boarding passengers or alighting passengers set according to different times at a specific station, and the flow line generation unit is a movement simulation device that generates a flow line between the boarding / alighting opening of the railway vehicle model and a predetermined position inside the railway vehicle model based on the number of boarding passengers or the number of alighting passengers.

[0112] According to the above configuration, since the number of boarding and alighting passengers is set differently depending on the time zone at a specific station, it is possible to efficiently generate the flow line of the human model at the time of boarding and alighting according to the actual number of boarding and alighting passengers.

[0113] (10) In the movement simulation device of (9) above, a seat area and a standing area are defined in the railway vehicle model, and the flow line generation unit is a movement simulation device that generates a flow line from the boarding / alighting opening to the seat area or the standing area based on a preset seating rate.

[0114] According to the above configuration, it is possible to efficiently generate the flow line of the human model by setting the ratio of people sitting and standing during boarding according to the seating rate in various proportions.

[0115] (11) In the movement simulation device of (9) above, a getting-off station is defined in the person model, and the flow line generation unit is a movement simulation device that generates a flow line from the current position of the person model whose getting-off station is the specific station to the boarding and alighting opening.

[0116] According to the above configuration, the flow line of the person model getting off at the station can be efficiently generated.

[0117] (12) In the movement simulation device of (8) above, the person model acquisition unit is a movement simulation device that acquires a number of person models according to the set number of passengers or boarding rate.

[0118] According to the above configuration, the flow line of the person model when an abnormality occurs in various cases with different numbers of passengers or boarding rates can be efficiently generated.

[0119] (13) In the movement simulation device of (8) or (12) above, the flow line generation unit is a movement simulation device that generates a flow line so as to go in a predetermined direction or an unspecified direction inside the railway vehicle.

[0120] According to the above configuration, the flow line of the person model can be efficiently generated for various cases of the direction the person model is going.

[0121] (14) In the movement simulation device of (8) or (12) above, the person model is classified as a disabled person or a general person, and the flow line generation unit is a movement simulation device that generates the flow line of the general person based on the current position of the disabled person.

[0122] According to the above configuration, the flow line of the general person can be efficiently generated based on the disabled person.

[0123] (15) In the movement simulation device of (14) above, the railway vehicle model has a connection port to the adjacent vehicle, and the flow line generation unit is a movement simulation device that generates the flow line of the general person so as to go to the connection port.

[0124] According to the above configuration, when an abnormal actor appears, the movement route when a general person moves to escape from the abnormal actor can be efficiently generated.

[0125] (16) In the movement simulation device of (14) above, the general person is further classified into a rescuer or other standby person around the abnormal person, and the movement route generation unit is a movement simulation device that generates the movement route of the rescuer so as to head towards the abnormal person.

[0126] According to the above configuration, the movement route of the person model when the people around rescue a person with an abnormality can be efficiently generated.

[0127] (17) In the movement simulation device of (14) above, an acquisition unit that acquires an abnormality occurrence rate that is the probability of an abnormal person occurring, and an assignment unit that assigns the attributes of an abnormal person to the person model based on the abnormality occurrence rate.

[0128] According to the above configuration, the movement route of the person model can be efficiently generated by setting the abnormality occurrence rate in various ways.

[0129] (18) In the movement simulation device of (6), (7), (8) or (12) above, a teacher data creation device that creates a video based on the movement route acquired from the movement simulation device.

[0130] According to the above configuration, teacher data for machine learning can be efficiently created assuming various cases.

Industrial Applicability

[0131] It can be suitably used when creating a large number of videos.

Explanation of Signs

[0132] 10 Movement simulation device 20 Video creation device 60 Vehicle model 80 Input screen 90 Operation screen

Claims

1. A movement simulation device that generates a movement route of a person model, A video creation device that creates a video of the person model, A teacher data creation system having: The movement simulation device includes: A structure model acquisition unit that acquires a structure model, A person model acquisition unit that acquires a person model, A movement route generation unit that generates a movement route of the person model based on the structure model, and The video creation device includes: A movement route acquisition unit that acquires the movement route from the movement simulation device, A structure model appearance acquisition unit that acquires the appearance of the structure model, A person model appearance acquisition unit that acquires the appearance of the person model, A video creation unit that creates a video based on the appearance of the structure model, the appearance of the person model, and the movement route. A teacher data creation system.

2. The movement simulation device further includes an event information acquisition unit that acquires event information regarding an event occurring within the structure model, The movement route generation unit generates a movement route of the person model based on the event information. The teacher data creation system according to Claim 1.

3. The event is an occurrence of an abnormality or a station stop of a railway vehicle. The teacher data creation system according to Claim 2.

4. The movement route generation unit generates a movement route so as to head in a predetermined direction or an unspecified direction within the structure model. The teacher data creation system according to Claim 1.

5. The person model is classified as an abnormal person or a normal person, The movement route generation unit generates a movement route of the normal person based on the current position of the abnormal person. The teacher data creation system according to Claim 1.

Images