Information processing device, display method, and program

Abstract

This system helps users easily understand the relationships between data series of data detected while a moving object is in operation. [Solution] An information processing device comprising: a data acquisition unit that acquires multiple types of detection data, which are data detected by sensors while a moving object is in operation; and a graph output unit that outputs a symbol graph composed of symbols generated by combining the detection data.

Description

Technical Field

[0001] The technology of the present disclosure relates to an information processing technology for generating a graph using data detected in a mobile object.

Background Art

[0002] Conventionally, a technology for creating a graph using results of detecting behavior of a vehicle has been disclosed (Patent Document 1). Specifically, the operation management system described in Patent Document 1 uses results obtained by detecting behavior of a vehicle in time series with a sensor, and describes that values for each data sequence indicating dangerous behavior of the vehicle such as sudden start, braking at an intersection, acceleration at an intersection, sudden steering, and sudden braking are represented in a radar chart format. It is also described that values of driving operations such as accelerator operation and brake operation are represented by a line graph.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technology described in Cited Document 1 has a problem that, if there is some relationship between data sequences of data detected while a mobile object such as a vehicle is being driven, it is difficult for a user to grasp the relationship between the data sequences.

[0005] The present disclosure has been made to solve the above problem, and an object of the present disclosure is to support a user to easily grasp the relationship between data sequences of data detected while a mobile object is being driven.

Means for Solving the Problems

[0006] The information processing device disclosed herein is A data acquisition unit that acquires multiple types of detection data, which are data detected by sensors while a moving object is in operation, A graph output unit that outputs a symbol graph composed of symbols generated by combining multiple types of the aforementioned detection data, It is something that is provided. [Effects of the Invention]

[0007] This disclosure provides the benefit of enabling users to easily understand the relationships (such as correlations and causal relationships) between data series of data detected while a moving object is in motion. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a diagram showing an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 1 of the present disclosure. [Figure 2] Figure 2 shows an example of the configuration of the data acquisition unit in the information processing device according to Embodiment 1 of this disclosure. [Figure 3] Figure 3 is a diagram showing an example of the configuration included in the graph output unit of the information processing device according to Embodiment 1 of this disclosure. [Figure 4] Figure 4 shows an example of the configuration included in the symbol graph generation unit of the information processing device according to Embodiment 1 of this disclosure. [Figure 5] Figure 5 is a first symbol diagram for illustrating the symbols in the symbol graph generated by the information processing device of the present disclosure. [Figure 6] Figure 6 is a second symbol diagram illustrating the symbols in the symbol graph generated by the information processing device of the present disclosure. [Figure 7] Figure 7 shows a first example of the display of a symbol graph generated by the information processing device of the present disclosure. [Figure 8]Figure 8 shows a second example of the display of a symbol graph generated by the information processing device of the present disclosure. [Figure 9] Figure 9 shows a third example of the display of a symbol graph generated by the information processing device of the present disclosure. [Figure 10] Figure 10 is a flowchart showing an example of processing in an information processing device according to Embodiment 1 of this disclosure. [Figure 11] Figure 11 is a flowchart showing a detailed example of the processing of an information processing device according to Embodiment 1 of this disclosure. [Figure 12] Figure 12 is a flowchart showing an example of a case in which a symbol graph generation process is executed in the graph generation process of the information processing apparatus according to Embodiment 1 of this disclosure. [Figure 13] Figure 13 is a diagram showing an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 2 of this disclosure. [Figure 14] Figure 14 shows examples of modifiable arrangements in a symbol graph (circular packing type symbol graph) output by an information processing device according to Embodiment 2 of the present disclosure. Figure 14a is an example of a center-priority arrangement, Figure 14b is an example of a causal arrangement, and Figure 14c is an example of a four-quadrant arrangement. [Figure 15] Figure 15 shows examples of changes in the arrangement of symbols and changes in the display area according to the target data in a symbol graph (square packing type symbol graph) output by an information processing device according to Embodiment 2 of this disclosure. Figure 15a is an example of minimum area arrangement, Figure 15b is an example of causal relationship arrangement, and Figure 15c is an example of causal relationship arrangement when different data from Figure 15b is targeted. [Figure 16] Figure 16 is a flowchart showing an example of processing in an information processing device according to Embodiment 2 of this disclosure. [Figure 17] Figure 17 shows an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 3 of this disclosure. [Figure 18]FIG. 18 is a diagram showing an example of a functional image realized by the information processing apparatus according to Embodiment 3 of the present disclosure. [Figure 19] FIG. 19 is a flowchart showing an example of processing of the information processing apparatus according to Embodiment 3 of the present disclosure. [Figure 20] FIG. 20 is a diagram showing a configuration example (first configuration example) of an information processing apparatus according to Embodiment 4 of the present disclosure and an information processing system including the information processing apparatus. [Figure 21] FIG. 21 is a diagram showing an example of a functional image (first image example) realized by the information processing apparatus according to Embodiment 4 of the present disclosure. [Figure 22] FIG. 22 is a diagram showing an example of a functional image (second image example) realized by the information processing apparatus according to Embodiment 4 of the present disclosure. [Figure 23] FIG. 23 is a flowchart showing an example of processing of the information processing apparatus shown in FIG. 20. [Figure 24] FIG. 24 is a diagram showing a configuration example (second configuration example) of an information processing apparatus according to Embodiment 4 of the present disclosure and an information processing system including the information processing apparatus. [Figure 25] FIG. 25 is a flowchart showing an example of processing of the information processing apparatus shown in FIG. 24. [Figure 26] FIG. 26 is a diagram showing a configuration example of an information processing apparatus according to Embodiment 5 of the present disclosure and an information processing system including the information processing apparatus. [Figure 27] FIG. 27 is a diagram showing a configuration example of an information processing apparatus according to Embodiment 6 of the present disclosure and an information processing system including the information processing apparatus. [Figure 28] FIG. 28 is a diagram showing a configuration example of a video output unit in the information processing apparatus according to Embodiment 6 of the present disclosure. [Figure 29] FIG. 29 is a diagram showing an example of a functional image (first image example) realized by the information processing apparatus according to Embodiment 6 of the present disclosure. [Figure 30]Figure 30 is a diagram showing an example of the function (second example) realized by the information processing device according to Embodiment 6 of this disclosure. [Figure 31] Figure 31 is a flowchart showing an example of processing in an information processing device according to Embodiment 6 of this disclosure. [Figure 32] Figure 32 is a flowchart showing a detailed example of video output processing in an information processing device according to Embodiment 6 of this disclosure. [Figure 33] Figure 33 shows a first example of a hardware configuration for realizing the functions of the configuration of this disclosure. [Figure 34] Figure 34 shows a second example of a hardware configuration for realizing the functionality of the configuration described herein. [Modes for carrying out the invention]

[0009] To further illustrate this disclosure, embodiments of this disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1. Embodiment 1 describes an example of a configuration for realizing the basic functions of this disclosure.

[0011] An example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 1 of this disclosure will be described. Figure 1 is a diagram showing an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 1 of the present disclosure. Information processing system 1(1A) is a system that processes data detected while a moving object is in operation, and is, for example, a driver monitoring system (DMS). The information processing system 1(1A) shown in Figure 1 includes a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100(100A).

[0012] The sensor device 10 outputs detection data to be used in the information processing device 100A. The sensor device 10 is a sensor or a group of sensors that perform detection while the moving object is in motion. The sensor device 10 consists of sensors mounted on the mobile body, sensors built into terminals or the like carried by the occupants of the mobile body, or both. The sensor device 10 outputs detection data. The sensor device 10 outputs mobile body data, which is detection data that detects the state, surrounding conditions, or both of the behavior of the mobile body. The mobile body data may include multiple types of detection data. The sensor device 10 also outputs occupant data, which is data that detects the state, biological information, or both of the behavior of the occupants of the mobile body. The occupant data may include multiple types of detection data.

[0013] The storage device 20 stores information used for processing by the information processing device 100. The storage device 20 stores the detection data output by the sensor device 10. Specifically, the storage device 20 stores multiple types of detection data detected at the same time or within the same time period, in chronological order. If the sensor device 10 includes a camera that captures images of the outside from a moving object, a camera that captures images of the inside of the moving object's room, or both, the storage device 20 is configured to store the video output by the camera. Furthermore, when storing video, the storage device 20 may be configured to store video only when it determines that an event has occurred based on the detection data output by the sensor device 10. The storage device 20 may be configured to be included in the information processing device 100. In this case, the storage device 20 can be described as a data storage unit in the information processing device 100.

[0014] The input / output device 30 functions as an input / output interface between the information processing device 100 and the user. If the information processing system 1A is a DMS, the input / output device 30 is, for example, a mobile device. The input / output device 30 shown in Figure 1 is composed of an input source device 40 and an output destination device 50. The input source device 40 is an input device that receives operation input from the user to the information processing device 100 and issues commands to the information processing device 100. The output device 50 displays the graph output by the information processing device 100. The output device 50 is, for example, a display device. Alternatively, the output device 50 may perform further processing using the graph data output by the information processing device 100. In this embodiment, the input source device 40 and the output destination device 50 are integrated into a single input / output device 30, but the input source device 40 and the output destination device 50 may be separate devices.

[0015] The information processing device 100 (100A) has the function of acquiring detection data, which is data detected while the moving object is in operation, and generating a graph using multiple types of detection data. The detected data is data detected by the sensor device 10. The detection data includes mobile object data, occupant data, or mobile object data and occupant data. In this disclosure, "mobile object data" is used as a general term for data detected when the mobile object is in operation, including the state of the mobile object's behavior, the surrounding conditions of the mobile object, or both the state of the mobile object's behavior and the surrounding conditions of the mobile object. The information processing device 100A may acquire multiple types of mobile object data as detected data. In this disclosure, occupant data is used as a general term for data that detects the behavioral state, biological state, or behavioral state and biological state of the occupants of a mobile vehicle while the mobile vehicle is being driven. The information processing device 100A may acquire multiple types of occupant data as detected data. The detected data includes personal identification information that identifies individual crew members or organizational identification information that identifies the organization to which a crew member belongs. This allows the information processing device 100 to generate aggregated graphs, for example, by individual or by organization. Furthermore, the detection data includes detection date and time information. This allows the information processing device 100 to generate a graph that represents combinations of detection data in chronological order. Additionally, for example, it can generate a symbol graph that represents symbols, which are aggregated by combining multiple types of detection data for each time (date and time), in chronological order.

[0016] If the information processing system 1A is a DMS, the information processing device 100A performs information processing to generate a graph using detection data detected regarding the vehicle's behavior and detection data detected regarding the driver's state. Detection data regarding the vehicle's behavior includes information such as speed, acceleration, and surrounding conditions. Detection data indicating the driver's state includes information such as driving behavior information and biometric information.

[0017] The information processing device 100 (100A) shown in Figure 1 is configured to include a data acquisition unit 110 and a graph output unit 120. The data acquisition unit 110 acquires multiple types of detection data, which are data detected while the moving vehicle is in operation. Specifically, the data acquisition unit 110 acquires multiple types of detection data, which are data detected by the sensor device 10 while the moving vehicle is in operation. The graph output unit 120 outputs a graph generated using the detection data acquired by the data acquisition unit 110. The graph output unit 120 also has the function of outputting a symbol graph composed of symbols generated by combining the detection data. The detection data combined by the graph output unit 120 consists of multiple types of detection data from a specified period set in advance, which are accumulated in time series from multiple types of detection data detected at the same time or during times including the same time.

[0018] A detailed configuration example of the data acquisition unit 110 will be described. Figure 2 shows an example of the configuration of the data acquisition unit in the information processing device according to Embodiment 1 of this disclosure. The data acquisition unit 110 is configured to include, for example, a mobile data acquisition unit 111 and a crew data acquisition unit 112, as shown in Figure 2. The mobile object data acquisition unit 111 acquires the mobile object data output by the sensor device 10. The crew data acquisition unit 112 acquires crew data output by the sensor device 10.

[0019] A detailed example of the graph output unit 120 configuration will be described. Figure 3 is a diagram showing an example of the configuration included in the graph output unit of the information processing device according to Embodiment 1 of this disclosure. The graph output unit 120 shown in Figure 3 includes a symbol graph generation unit 121. The symbol graph generation unit 121 generates a symbol graph composed of symbols generated by combining multiple types of detection data. Specifically, the symbol graph generation unit 121 generates symbols for each data combination by combining multiple types of mobile object data, or mobile object data and crew data. The symbol graph generation unit 121 generates a graph composed of symbols as elements and outputs a symbol graph (symbol diagram), which is a graph containing symbols. The symbol graph generation unit 121 generates a symbol graph composed of symbols representing data that combines, for example, mobile body data, which is data detecting the state or surrounding conditions of a mobile body, and occupant data, which is data detecting the state of the occupants of the mobile body. The graph output unit 120 outputs graph data representing the symbol graph.

[0020] A detailed example of the symbol graph generation unit 121 will be described. Figure 4 shows an example of the configuration included in the symbol graph generation unit of the information processing device according to Embodiment 1 of this disclosure.

[0021] The symbol graph generation unit 121 is configured to include, for example, a symbol graph type setting unit 122, a data aggregation unit 123, a symbol generation unit 124, a symbol placement unit 125, and an iconization unit 126, as shown in Figure 4.

[0022] The symbol graph type setting unit 122 sets the graph type to be generated. The symbol graph type setting unit 122 selects, for example, generation conditions from among the pre-registered symbol graph types, which are the conditions for generating a symbol graph of the commanded symbol graph type. The symbol graph type setting unit 122 does not need to be included in the configuration of the symbol graph generation unit 121 if there is only one type of symbol graph type that has been registered in advance.

[0023] The data aggregation unit 123 extracts and aggregates multiple types of detected data based on pre-commanded and registered aggregation conditions. The data aggregation unit 123 aggregates data such as acceleration as moving object data and occupant data such as distraction, drowsiness, phone calls, near misses, etc. The data aggregation unit 123 may further aggregate non-driving behaviors such as sudden braking due to a person suddenly running out into the road. The data aggregation unit 123 aggregates detection data for individuals or organizations as instructed in advance. This allows for the generation of graphs based on the detection data for individuals or organizational units. Organizations are envisioned to be, for example, the company or department to which a crew member belongs. The data aggregation unit 123 aggregates the detected data for a specified period, which is a pre-determined period, as the aggregation period. The aggregation period is not particularly limited in this disclosure, but is envisioned to be a period such as a day, week, month, quarter, half a year, or year. This makes it possible to generate a graph based on the detected data for any period desired by the user.

[0024] The symbol generation unit 124 uses the aggregated detection data to combine multiple types of detection data and generates symbols corresponding to the combination of detection data. The symbol generation unit 124 determines the size, shape, and color of a symbol according to the elements of the combined detection data, and generates a symbol that is represented by that size, shape, and color. The symbol generation unit 124 generates symbols of different shapes for each type of first detection data among the combinations of detection data. In this case, the symbol generation unit 124 generates symbols of predetermined shapes according to the type of first detection data among the combinations of detection data. The symbol generation unit 124 generates symbols of different colors for each type of first detection data among the combinations of detection data. In this case, the symbol generation unit 124 generates symbols of predetermined colors according to the type of first detection data among the combinations of detection data. The symbol generation unit 124 generates symbols of different sizes, for example, according to the value of the second detection data in the combination of detection data. In this case, the symbol generation unit 124 generates symbols so that they are of different sizes according to the value of the second detection data in the combination of detection data. Specifically, the symbol generation unit 124 generates symbols so that their size is proportional to the value of the second detection data in the combination of detection data. For example, it generates symbols whose size is proportional to a value such as acceleration. The symbol generation unit 124 can generate a symbol for the other detection data (first detection data) even if the value of one of the detection data (second detection data) in the combination of detection data is 0. In this case, the elements of the symbol graph generated by the symbol graph generation unit 121 include the symbol for the first detection data for which the value of the second detection data is 0. The size of the circumference of the symbols generated by this symbol generation unit 124 is inevitably related to the importance and number of occurrences of the element. The more occurrences of an element there are, the larger the circle will be. Also, even if the number of occurrences of an element is small, the circle will be large if there are elements with high importance.

[0025] The symbol placement unit 125 places the generated symbols as elements of the symbol graph based on placement conditions that have been commanded and registered in advance. The symbol placement unit 125 also places symbols as elements of the symbol graph according to the detection data indicated by the symbols. For example, the symbol placement unit 125 derives the placement position of the symbols and places them to satisfy placement conditions such as minimizing the area of ​​the symbol graph.

[0026] The iconization unit 126 converts the elements of the symbol graph into icons corresponding to the detection data for each element. For example, the iconization unit 126 determines the shape, color, or shape and color of an icon that is pre-set for each type of detection data in one of the combinations of detection data, and determines the size according to the value of the other detection data, thereby converting the symbol into an icon. The icon is, for example, a shape that illustrates the type of driving behavior shown in the occupant data. For example, if the type of driving behavior is dozing off, the icon is a shape that illustrates the face of a sleeping person. If the type of driving behavior is talking on the phone, the icon is a shape that illustrates a telephone receiver. In this case, the elements of the symbol graph output by the graph output unit 120 are represented by icons of shape, color, and size corresponding to the type of detected data for each element, which helps the user to easily understand what the combination of detected data indicates. Furthermore, the information processing device 100 may not be configured to display symbols representing elements of the symbol graph as icons, and if it is configured not to display symbols as icons, it may not be equipped with an iconization unit 126.

[0027] Here, we will explain the symbol graph and examples of symbols using diagrams. Figure 5 is a first symbol diagram for illustrating the symbols in the symbol graph generated by the information processing device of the present disclosure. The information processing device 100 can, for example, represent mobile data (vehicle data) acquired from various sensors mounted on a mobile object (vehicle) and driving behavior data (occupant data) acquired from the driver's biosignals as a single symbol. The information processing device 100 can represent multiple types of detection data by changing the size, shape, and color of the symbol.

[0028] The information processing device 100 can generate symbols 1110, 1120, 1130, and 1140 as shown in Figure 5. Symbol 1110 shows an example of a symbol representing the first combination in terms of shape, size, and numerical value. The first example of a combination is a combination of occupant data and moving object data. Symbol 1110 represents "distraction," a type of driving behavior, as occupant data using a circular shape, and "acceleration," as moving object data, using size and numerical value. Symbol 1120 shows an example of a symbol representing the second combination using shape, size, and numerical values. The second example of a combination is a combination of occupant data and moving object data. Symbol 1110 represents "drowsiness," a type of driving behavior, as occupant data using a rectangle, and "acceleration," as moving object data, using size and numerical values. Symbol 1130 shows an example of a symbol that represents the first combination using color, size, and numerical values. Symbol 1130 represents "distraction," a type of driving behavior, as occupant data, in a dark color, and "acceleration," a moving object data, as size and numerical values. Symbol 1140 shows an example of a symbol that represents the second combination using color, size, and numerical values. Symbol 1130 represents "distraction," a type of driving behavior, as occupant data using a light color, and "acceleration," as moving object data, using size and numerical values. Using such symbols, for example, acceleration values ​​as moving object data can be represented by the size (and numerical value) of the symbol, and types of driving behaviors such as distracted driving, dozing off, talking on the phone, and near misses as occupant data can be represented by the shape, color, or shape and color of the symbol. The symbols have different shapes for each type of first detection data among the detection data represented by the symbol. In this case, the symbols are created by the information processing device 100 in a shape predetermined according to, for example, the type of first detection data among the detection data represented by the symbol. The symbols have different colors for each type of first detection data among the detection data shown by the symbol. In this case, the symbols are created by the information processing device 100 using colors predetermined according to the type of first detection data among the detection data shown by the symbol. The symbols have different sizes depending on the value of the second detection data among the detection data shown by the symbol. In this case, the symbols are created by the information processing device 100 so that they have different sizes depending on the value of the second detection data among the detection data shown by the symbol. For example, the symbols are created by the information processing device 100 so that their size is proportional to the value of the second detection data among the detection data shown by the symbol. The size levels are set to arbitrary sizes as appropriate, either in advance or in advance by user command.

[0029] Figure 6 is a second symbol diagram illustrating the symbols in the symbol graph generated by the information processing device of the present disclosure. Figure 6 shows examples of symbols 1210, 1220, 1230, 1240, and 1290 when the symbols represent moving object data and occupant data, where the moving object data is acceleration and the occupant data is distraction, drowsiness, phone calls, near misses, and non-driving behavior. Figure 6 also shows examples of symbols generated when the symbol generation unit 124 of the symbol graph generation unit 121 is configured to generate a symbol for the other detection data (first detection data) even when the value of one of the detection data (second detection data) in the combination of detection data is 0. As shown by the symbol on the far left of Figure 6, symbols 1210, 1220, 1230, and 1240 for each data series include symbols that represent the occupant data detected when the acceleration is zero. As shown in Figure 6, non-driving behaviors, which are not included in driving behaviors (the behavior of occupants while driving), can be included in the symbol graph (symbol 1290). Non-driving behaviors include, for example, sudden braking when a person suddenly jumps out into the road.

[0030] Next, we will explain a specific example of a symbol graph. Figure 7 shows a first example of the display of a symbol graph generated by the information processing device of the present disclosure. Figure 8 shows a second example of the display of a symbol graph generated by the information processing device of the present disclosure. Figure 9 shows a third example of the display of a symbol graph generated by the information processing device of the present disclosure. The information processing device 100 generates and outputs a symbol graph in the format shown in Figures 7, 8, and 9. The symbol graph type may be pre-set or commanded during creation. It may also be configured to be changeable, as in the embodiment described later. The linear symbol graph 1300 shown in Figure 7 is a graph in which symbols are arranged on a straight line. In the linear symbol graph 1300, if the straight line represents, for example, the time axis, it becomes easy to understand the detection data (combinations of detection data) detected over time. The circular packing type symbol graph 1400 shown in Figure 8 and the square packing type symbol graph 1500 shown in Figure 9 are graphs in which the elements of the corresponding aggregation period are packed together in order to minimize the area. The circular packing type symbol graph 1400 shown in Figure 8 is a graph with a circular shape. The square packing type symbol graph 1500 shown in Figure 9 is a graph with a rectangular outline.

[0031] In addition to the above configuration, the information processing device 100 includes a control unit (not shown), a storage unit (not shown), and a communication unit (not shown). A control unit (not shown) controls the entire information processing device 100 and its individual components. For example, the control unit starts the information processing device 100 according to an external command. The control unit also controls the state of the information processing device 100 (operating state = state such as startup, shutdown, or sleep). A storage unit (not shown) stores the data used by the information processing device 100. For example, the storage unit stores the output (output data) from each component of the information processing device 100 and outputs the requested data to the requesting component. The communication unit (not shown) communicates with external devices. For example, it communicates between the information processing device 100 (100A) and peripheral devices (for example, if the information processing system 1 is a DMS, sensors on a mobile device, mobile device-mounted devices, etc.). For example, if the information processing device 100 and the peripheral devices are not connected by wire, the communication unit (not shown) has the function of communicating between the information processing device 100 and the peripheral devices. The communication unit (not shown) also has the function of communicating with an external device, such as a server device. The control unit (not shown), storage unit (not shown), and communication unit (not shown) are the same in the embodiments described later.

[0032] An example of processing by the information processing device according to Embodiment 1 of this disclosure will be described. Figure 10 is a flowchart showing an example of processing in an information processing device according to Embodiment 1 of this disclosure. The process shown in Figure 10 is an information processing method or display method by an information processing device. The information processing device 100 (100A) according to this embodiment starts the processing shown in Figure 10 ("Start") when the processing start conditions are met. The processing start conditions are, for example, when an execution command is received from a user who intends to perform an operation analysis. The information processing device 100A then performs data acquisition processing (step ST1010 "data acquisition"). In the data acquisition processing, the data acquisition unit 110 of the information processing device 100A acquires multiple types of detection data, which are data detected by sensors while the moving object is in operation. The data acquisition unit 110 outputs the acquired detection data to the graph output unit 120. The information processing device 100A then performs graph output processing (step ST1020 "Graph Output"). In the graph output processing, when the graph output unit 120 of the information processing device 100A receives the detection data output by the data acquisition unit 110, it outputs graph data showing a symbol graph composed of symbols generated by combining multiple types of detection data to the output destination device 50. When the output destination device 50 receives graph data output by the graph output unit 120 of the information processing device 100A, it displays a symbol graph. When the information processing device 100A executes the graph output process, it terminates the process shown in Figure 10 ("terminate").

[0033] A detailed example of the process described above will be explained. Figure 11 is a flowchart showing a detailed example of the processing of an information processing device according to Embodiment 1 of this disclosure. When the information processing device 100A starts the process shown in Figure 11 ("Start"), it then executes the generation condition setting process (step ST1110 "Set conditions to be acquired based on the graph to be generated"). In the generation condition setting process, the information processing device 100A sets the conditions to be acquired based on the graph to be generated. The information processing device 100A then performs detection data acquisition processing (step ST1120 "Acquire multiple types of detection data"). In the detection data acquisition processing, the data acquisition unit 110 of the information processing device 100A acquires the detection data output by the sensor device 10 via the storage device 20. The data acquisition unit 110 acquires multiple types of detection data. For example, the data acquisition unit 110 acquires multiple types of mobile object data, multiple types of occupant data, or mobile object data and occupant data, which are detection data. The information processing device 100A then performs detection data output processing (step ST1130 "Output acquired detection data to graph output unit"). In the detection data output processing, the data acquisition unit 110 of the information processing device 100A passes the acquired detection data to the graph output unit 120. The information processing device 100A then performs a graph generation process (step ST1140 "Generate a graph by combining multiple types of detection data"). In the graph generation process, the graph output unit 120 of the information processing device 100A receives the detection data acquired by the data acquisition unit 110, combines and aggregates multiple types of detection data, and generates a graph using the aggregated data. In the graph generation process, the information processing device 100A performs, for example, a symbol graph generation process. In the symbol graph generation process, the symbol graph generation unit 121 in the graph output unit 120 of the information processing device 100A uses a combination of multiple types of detection data to generate symbols that represent each element of the aggregated data with a shape, color, or shape and color corresponding to the detection data indicated by the element, and a size corresponding to different detection data. The symbol graph generation unit 121 generates a symbol graph by arranging the generated symbols according to pre-set arrangement conditions. The information processing device 100A then performs graph data output processing (step ST1150 "Output graph to output destination device"). In graph data output processing, the graph output unit 120 of the information processing device 100A outputs graph data representing a symbol graph to the output destination device 50. When the information processing device 100A executes the graph data output process, it terminates the process shown in Figure 11 ("terminate").

[0034] Here, we will explain an example of the process when executing symbol graph generation during graph generation. Figure 12 is a flowchart showing an example of a case in which a symbol graph generation process is executed in the graph generation process of the information processing apparatus according to Embodiment 1 of this disclosure. In the graph output unit 120 of the information processing device 100A, the symbol graph generation unit 121 executes the symbol graph generation process in the graph generation process of step ST1140, for example, the process shown in Figure 12. When the symbol graph generation unit 121 starts the process shown in Figure 12 ("Start"), it then executes the graph type setting process (step ST1210 "Graph Type Setting"). In the graph type setting process, the symbol graph type setting unit 122 of the symbol graph generation unit 121 sets the graph type of the symbol graph to be generated to a pre-set graph type. Alternatively, the unit may accept a command for the graph type of the symbol graph and set the commanded graph type. The symbol graph generation unit 121 then performs data aggregation processing (step ST1220 "Data Aggregation"). In the data aggregation processing, the data aggregation unit 123 of the symbol graph generation unit 121 acquires multiple types of detection data from the storage device 20 according to the set conditions and aggregates the detection data. The data aggregation unit 123 outputs the aggregated data to the symbol graph generation unit 121. The symbol graph generation unit 121 then performs a symbol generation process (step ST1230 "Symbol Generation"). In the symbol generation process, the symbol generation unit 124 of the symbol graph generation unit 121 generates a symbol for each combination of detection data using aggregated data obtained by combining and aggregating multiple types of detection data. The symbol graph generation unit 121 then performs a symbol placement process (step ST1240 "Symbol Placement"). In the symbol placement process, the symbol placement unit 125 of the symbol graph generation unit 121 generates a symbol graph with symbols placed according to pre-set placement conditions. The symbol graph generation unit 121 then performs an iconization process (step ST1250 "iconization"). In the iconization process, the iconization unit 126 of the symbol graph generation unit 121 changes the symbols for each element in the symbol graph to icons corresponding to the detection data for each element. The iconization unit 126 outputs graph data showing the symbol graph with the symbols converted into icons. The symbol graph generation unit 121 outputs the generated symbol graph to the output destination device 50 and then terminates the process shown in Figure 12 ("termination").

[0035] The above configuration allows for the implementation of functions such as the following: (1) Display the type of driver behavior using the shape or color of a symbol. (2) The size of the symbolized elements (the shape of the symbols) shall be expressed in proportion to the values ​​of various on-vehicle sensors (such as acceleration) detected after the driving behavior occurs. The number of steps may be arbitrary. (3) The number of symbolized elements represents the number of detected driving behaviors. *Even if only driving behavior is detected with a value of 0, indicating no change in vehicle data, it will be symbolized, with a symbol of 0 (e.g., no sudden acceleration or deceleration after distraction). (4) The symbols will be aggregated by individual drivers or organizations. (5) The aggregation period for symbols can be set to days / weeks / months / years or a specified period. (6) The collection of aggregation periods may or may not be grouped together. If grouped together, the rectangular shape is not required. (7) Vehicle data elements not related to driving behavior may be included (e.g., sudden braking due to a person suddenly appearing). This makes it easier for users to understand the driving characteristics of occupants that are not reflected in mobile vehicle data. For example, it becomes easier to identify if a driver frequently becomes distracted or falls asleep at the wheel. Furthermore, it becomes possible to compare and reference individual drivers and organizations. According to the above configuration, for example, it is possible to visualize the relationship between vehicle behavior and the driver's driving characteristics (such as frequent distractions), and to provide a user interface that can support safe driving by the driver. Furthermore, the above configuration, when applied to vehicles, makes it easier to understand what kind of driving by the driver caused dangerous vehicle behavior. In addition, even if dangerous vehicle behavior did not occur, it becomes possible to easily identify the driver's driving characteristics (such as frequent distractions), enabling preventive guidance for safe driving.

[0036] This embodiment shows an example of a configuration that can be expressed as follows. [1] A data acquisition unit that acquires multiple types of detection data, which are data detected while a moving object is in operation, A graph output unit outputs a symbol graph composed of symbols generated by combining the aforementioned detection data, Equipped with an information processing device. This disclosure has the effect of providing an information processing device that can help users easily understand the relationships (such as correlations and causal relationships) between data series of data detected while a moving object is in motion. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, helps users easily understand the relationships between data series of data detected while the moving object is in operation.

[0037] This embodiment shows an example of a configuration that can be expressed as follows.

[11] A display method that uses an information processing device to display information on a display device, The aforementioned information processing device acquires multiple types of detection data, which are data detected by sensors while the moving object is in operation. The information processing device outputs a symbol graph, which is composed of symbols generated by combining the detected data, to the display device. A method of display characterized by the following features. This disclosure has the effect of providing a display method that enables users to easily understand the relationships (such as correlations and causal relationships) between data series of data detected while a moving object is in motion. Furthermore, by applying the above display method to the DMS, it is possible to provide a support method in the DMS that enables users to easily understand the relationships between data series of data detected while the moving object is in operation, similar to the above.

[0038] This embodiment shows an example of a configuration that can be expressed as follows.

[12] Computers, A data acquisition unit that acquires multiple types of detection data, which are data detected by sensors while a moving object is in operation, A graph output unit outputs a symbol graph composed of symbols generated by combining the aforementioned detection data, A program characterized by operating as an information processing device equipped with [a specific feature]. This disclosure has the effect of providing a program that can help users easily understand the relationships (such as correlations and causal relationships) between data series of data detected while a moving object is in motion. Furthermore, by applying the above program to the DMS program, it becomes possible to provide a DMS program that, similarly to the above, helps users easily understand the relationships between data series of data detected while the moving object is in motion.

[0039] This embodiment further illustrates an example of a configuration that can be expressed as follows. [3] The elements of the symbol graph output by the aforementioned graph output unit are represented by icons corresponding to the detected data for each element. The information processing apparatus according to [1][2], characterized in that This disclosure has the effect of providing an information processing device that can help users more easily understand the relationships between data sequences of data detected while a moving object is in motion, thereby assisting in visual comprehension. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0040] This embodiment further illustrates an example of a configuration that can be expressed as follows. [4] The graph output unit outputs a symbol graph composed of symbols representing data that combines mobile body data, which is data detecting the state or surrounding conditions of the mobile body, and occupant data, which is data detecting the state of the occupants of the mobile body. An information processing device according to any one of [1][2][3], characterized in that This disclosure has the effect of providing an information processing device that makes it easier for users to understand the relationships between data series of data detected while a moving object is in motion, especially when there is a strong relationship between the data series of moving object data and occupant data. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0041] Embodiment 2. In the above-described embodiment 1, an example of a configuration was explained in which a symbol graph is generated by combining multiple types of detection data and representing them with symbols. Embodiment 2 describes an example of a configuration that allows for the transformation of the packing (arrangement) of symbolized elements (symbols). In the information processing device according to Embodiment 2, for example, a pre-set packing method is a method of packing the elements for the relevant aggregation period in the smallest possible area, but it is made possible to change the position of the elements according to the purpose at the user's command. Furthermore, in the information processing device according to Embodiment 2, when the purpose is to correlate with vehicle data, the system is divided into four axes: sudden acceleration and sudden deceleration (vertical direction), sudden left steering and sudden right steering (horizontal direction), and each element is placed in a nearby quadrant. In the information processing device according to Embodiment 2, the area of ​​the outer perimeter of the graph described above is configured to change depending on the maximum value of the element. In Embodiment 2, components of Embodiment 2 that are the same as those of Embodiment 1 already described will be given the same reference numerals or similar reference numerals, and redundant explanations may be omitted as appropriate.

[0042] Next, an example of the configuration of the information processing device according to Embodiment 2 of this disclosure will be described. Figure 13 is a diagram showing an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 2 of this disclosure. The information processing system (DMS) 1(1B) shown in Figure 13 comprises a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100(100B). The input / output device 30 is configured to include an input source device (input device) 40 and an output destination device (display device) 50.

[0043] The information processing device 100 (100B) according to Embodiment 2 receives a commanded arrangement condition while a graph is being output, and changes the arrangement of symbols, which are elements of the graph, according to the commanded arrangement condition. The information processing device 100 (100B) shown in Figure 13 includes a data acquisition unit 110, a graph output unit 120, a placement condition command reception unit 131, and a symbol placement change unit 132. The data acquisition unit 110 has the same functions as the data acquisition unit 110 described earlier.

[0044] The placement condition command receiving unit 131 receives the commanded placement conditions, which are the commanded placement conditions. The placement condition command receiving unit 131 receives and acquires the commanded placement conditions that have been commanded by the user's operation input via the input source device 40.

[0045] The symbol placement modification unit 132 changes the placement of symbols in the symbol graph according to the command placement conditions.

[0046] The graph output unit 120 has the same functions as the graph output unit 120 already described, and further outputs graph data of a symbol graph in which the arrangement of symbols has been changed according to the command arrangement conditions.

[0047] The image of the functions of the information processing device 100 (100B) according to Embodiment 2 will be explained. Figure 14 shows examples of modifiable arrangements in a symbol graph (circular packing type symbol graph) output by an information processing device according to Embodiment 2 of the present disclosure. Figure 14a is an example of a center-priority arrangement, Figure 14b is an example of a causal arrangement, and Figure 14c is an example of a four-quadrant arrangement. The center-priority circular packing type symbol graph 2100 shown in Figure 14a is, for example, an arrangement that has been pre-set as the normal arrangement in the information processing device 100B. The center-priority circular packing type symbol graph 2100 is generated by placing a large element at the center of the circle. The circular packing symbol graph 2200 with causal relationships shown in Figure 14b is generated by adjusting the distance between elements based on their causal relationships. In the circular packing symbol graph 2200, elements are arranged so that the closer they are to each other, the stronger the causal relationship between them. The circular packing type symbol graph 2300, shown in Figure 14c with a four-quadrant arrangement, is generated by arranging the symbols in the four quadrants of rapid acceleration / deceleration and sharp steering immediately after occurrence. For example, by changing the arrangement from the one shown in Figure 14a to the one shown in Figure 14b or Figure 14c, different relationships become easier for the user to understand.

[0048] Figure 15 shows examples of changes in the arrangement of symbols and changes in the display area according to the target data in a symbol graph (square packing type symbol graph) output by an information processing device according to Embodiment 2 of this disclosure. Figure 15a is an example of minimum area arrangement, Figure 15b is an example of causal relationship arrangement, and Figure 15c is an example of causal relationship arrangement when different data from Figure 15b is targeted. The square-packed symbol graph 2400 with the minimum area arrangement shown in Figure 15a is, for example, an arrangement that has been pre-set as the normal arrangement in the information processing device 100B. In the square-packed symbol graph 2400, the elements for the relevant aggregation period are packed in the minimum area. The square-packing type symbol graph with a four-quadrant arrangement (first example) 2500 shown in Figure 15b and the square-packing type symbol graph with a four-quadrant arrangement (second example) 2600 shown in Figure 15c are both four-quadrant arrangement symbol graphs in which symbols are arranged in the four quadrants of sudden acceleration / deceleration (acceleration axis, deceleration axis) and sudden steering (left steering axis, right steering axis) immediately after occurrence. In this case, the area of ​​the outer rectangle changes according to the arrangement of elements. Figure 14b shows the case where the maximum value is 2, and Figure 14c shows the case where the maximum value is only 1. In Figure 14b, the area of ​​the outer rectangle of the graph is created to be larger than in Figure 14c. By changing the arrangement from that shown in Figure 14a to that shown in Figure 14b or Figure 14c, users can more easily understand the differences in relationships based on the detected data.

[0049] An example of processing by the information processing device according to Embodiment 2 of this disclosure will be described. Figure 16 is a flowchart showing an example of processing in an information processing device according to Embodiment 2 of this disclosure. The information processing device 100 (100B) according to this second embodiment starts the processing shown in Figure 16 ("Start") when the processing start conditions are met. The processing start conditions are, for example, when the information processing device 100C outputs graph data, or when the output graph is displayed on the output destination device 50. The information processing device 100B then performs a sorting change command reception determination process (step ST2110 "Sorting change command received?"). In the sorting change command reception determination process, the sorting condition command reception unit 131 of the information processing device 100B determines whether a sorting condition command has been received. If the placement condition command receiving unit 131 determines that it has not received a placement condition command (step ST2110 "Placement change command received?" "NO"), the process proceeds to the termination determination process in step ST2150. If the information processing device 100B determines that the placement condition command receiving unit 131 has received a placement condition command (step ST2110 "Placement change command received?" "YES"), it then executes a placement condition acquisition process (step ST2120 "Placement condition acquisition"). In the placement condition acquisition process, the placement condition command receiving unit 131 of the information processing device 100B acquires the placement conditions indicated in the placement change command. The information processing device 100B then performs a symbol arrangement change process (step ST2130 "Symbol arrangement change"). In the symbol arrangement change process, the symbol arrangement change unit 132 of the information processing device 100B changes the arrangement of symbols in the symbol graph according to the command arrangement conditions. The information processing device 100B then performs graph data output processing (step ST2140 "Output graph to output destination device"). In graph data output processing, the graph output unit 120 of the information processing device 100B outputs a symbol graph in which the arrangement of symbols has been changed according to the command arrangement conditions. When the information processing device 100B executes graph data output processing, it then executes termination determination processing (step ST2150 "Termination?"). In the termination determination processing, a control unit (not shown) of the information processing device 100B determines that processing should be terminated if conditions are met, for example, such as the graph output by the information processing device 100B being finished displaying on the output destination device 50. If the information processing device 100B determines that it does not want to terminate the process (step ST2150 "Termination?" "NO"), it returns to the process in step ST2110. If the information processing device 100B determines in the termination determination process that it should terminate (step ST2150 "Termination?" "YES"), it then terminates the process shown in Figure 16 ("Termination").

[0050] The above configuration makes it easier for users to understand the relationships between data series while rearranging the symbol placement.

[0051] This embodiment further illustrates an example of a configuration that can be expressed as follows. [2] A placement condition command receiving unit that receives commanded placement conditions, which are the commanded placement conditions, A symbol arrangement changing unit that changes the arrangement of symbols in the symbol graph according to the aforementioned command arrangement conditions, Equipped with, The graph output unit outputs a symbol graph in which the arrangement of symbols has been changed according to the command arrangement conditions. The information processing apparatus according to [1], characterized in that This disclosure provides an information processing device that makes it easier for users to understand the relationships between data series while a moving object is in motion, by allowing users to change the arrangement of symbols and thus assisting users in more easily understanding the relationships between data series of data detected while the object is in motion. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0052] Embodiment 3. In the above-described embodiment 2, a configuration example was described in which the arrangement of symbols, which are elements of the symbol graph, can be changed. Embodiment 3 describes an example configuration that allows the type of graph to be changed. In Embodiment 3, components related to Embodiment 3 that are the same as those related to Embodiment 1 or Embodiment 2 already described are denoted by the same reference numerals or similar numerals, and redundant explanations may be omitted as appropriate.

[0053] Next, an example of the configuration of the information processing device according to Embodiment 3 of this disclosure will be described. Figure 17 shows an example configuration of an information processing device and an information processing system including the information processing device according to Embodiment 3 of this disclosure. The information processing system (DMS) 1(1C) shown in Figure 17 comprises a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100(100C). The input / output device 30 is configured to include an input source device (input device) 40 and an output destination device (display device) 50.

[0054] The information processing device 100 (100C), while a graph is being output, receives a commanded graph type and changes the graph to one corresponding to the received graph type. The information processing device 100 (100C) shown in Figure 17 includes a data acquisition unit 110, a graph output unit 120, a graph type command reception unit 141, and a graph type changing unit 142.

[0055] The graph type command receiving unit 141 receives the commanded graph type, which is the commanded graph type. The graph type command receiving unit 141 receives the commanded graph type, which is the commanded graph type, from among the graph types (graph categories) that have been registered in advance.

[0056] The graph type changing unit 142 uses the detection data shown in the output graph to change the graph to a graph type corresponding to the command graph type and output it. For example, the graph type changing unit 142 uses the detection data shown in the output graph (symbol graph) to change the graph to a graph type corresponding to the command graph type and output it.

[0057] The graph output unit 120 has the same functions as any of the graph output units 120 already described, and further outputs graph data of the graph that has been changed by the graph type change unit 142.

[0058] The image of the functions realized by the information processing device according to Embodiment 3 will be explained. Figure 18 is a diagram illustrating an example of the functions realized by the information processing device according to Embodiment 3 of this disclosure. The circular packing type symbol graph 3100 is, for example, a graph set to the normal type, and is generated by placing elements (symbols) on circular packing. The acceleration type graph (Sankey diagram format graph) 3200 is a graph generated to show the relationship between driving behavior and vehicle behavior (acceleration) using a Sankey diagram. In the figure, the graph includes "other" data, which is generated by supplementing data from other sources. The time-series type graph 3300 is a graph generated by arranging each element in a time series, showing the time period of occurrence and the speed at which it occurs. In the graph shown in the figure, the level of drowsiness at the time of occurrence is shown. The information processing device 100C can, for example, change the graph type. Specifically, it can change the graph type from, for example, the circular packing representation. This allows the user to be provided with different representations depending on the perspective they are analyzing. For example, it is envisioned that a significant event can be identified in the packing graph, and then the graph type can be changed to show the speed and time when that event occurred.

[0059] An example of processing by the information processing device according to Embodiment 3 of this disclosure will be described. Figure 19 is a flowchart showing an example of processing in an information processing device according to Embodiment 3 of this disclosure. The information processing device 100 (100C) according to this third embodiment starts the processing shown in Figure 19 ("Start") when, for example, the processing start conditions are met. The processing start conditions are, for example, when the information processing device 100C outputs graph data, or when the output graph is displayed on the output destination device 50. The information processing device 100C then performs a graph type change command reception determination process (step ST3110 "Graph type change command received?"). In the graph type change command reception determination process, the graph type command reception unit 141 of the information processing device 100C determines whether a graph type change command has been received. If the graph type command receiving unit 141 determines that it has not received a graph type change command (step ST3110 "Graph type change command received?" "NO"), the process proceeds to the termination determination process in step ST3150. If the graph type command receiving unit 141 of the information processing device 100C determines that it has received a graph type change command (step ST3110 "Graph type change command received?" "YES"), it then executes a command graph type acquisition process (step ST3120 "Command graph type acquisition"). In the command graph type acquisition process, the graph type command receiving unit 141 of the information processing device 100C receives the command graph type, which is the commanded graph type. The information processing device 100C then performs a graph type change process (step ST3130 "Graph Type Change"). In the graph type change process, the graph type change unit 142 of the information processing device 100C uses the detection data shown in the output graph to change the graph to a graph type corresponding to the command graph type. For example, the graph type change unit 142 uses the detection data shown in the output graph (symbol graph) to change the graph to a graph type corresponding to the command graph type. The information processing device 100C then performs graph data output processing (step ST3140 "Output graph to output destination device"). In the graph data output processing, the graph output unit 120 of the information processing device 100C outputs the graph data of the graph changed by the graph type change unit 142 to the output destination device 50. The information processing device 100C then performs a termination determination process (step ST3150 "Termination?"). In the termination determination process, a control unit (not shown) of the information processing device 100C determines that the process should be terminated if conditions are met, for example, such as the graph output by the information processing device 100C being finished displaying on the output destination device 50. If the information processing device 100C determines that it does not want to terminate the process (step ST3150 "Termination?" "NO"), it returns to the process in step ST3110. If the information processing device 100C determines that it has finished processing (step ST3150 "Finish?" "YES"), it finishes the processing shown in Figure 19 ("Finish").

[0060] With the configuration described above, the information processing device can change the type of graph. The information processing device can, for example, change the graph type from a circular packing representation. Using this device, users can switch graph types from the perspective of the starting data point, enabling them to analyze the subject from various viewpoints. Users can flexibly change graph types for different purposes, allowing for data analysis from diverse perspectives.

[0061] This embodiment further illustrates an example of a configuration that can be expressed as follows. [5] A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, A graph type changing unit that uses the detection data shown in the output graph to change the graph to a graph type corresponding to the command graph type and output it, Equipped with, An information processing device according to any one of [1][2][3][4], characterized in that This disclosure provides an information processing device that makes it easier for users to understand the relationships between data series while a moving object is in motion, by allowing users to change the graph type and thus assisting users in more easily understanding the relationships between data series of data detected while the object is in motion. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0062] Embodiment 4. In the embodiments described above, when there were many elements in the graph, they sometimes overlapped, making it difficult for the user to grasp the elements. Embodiment 4 further describes an example of a configuration in which elements are selected and emphasized. Furthermore, we will explain an example of a configuration in which a set of elements can be extracted and selected from a chart in which elements are associated. In Embodiment 4, components of Embodiment 4 that are the same as those of Embodiment 1, Embodiment 2, or Embodiment 3 already described are denoted by the same reference numerals or similar numerals, and redundant explanations may be omitted as appropriate.

[0063] Next, a first configuration example of the information processing device according to Embodiment 4 of this disclosure will be described. Figure 20 shows an example configuration (first configuration example) of an information processing device and an information processing system including the information processing device according to Embodiment 4 of the present disclosure. The information processing system (DMS) 1 (1D) shown in Figure 20 comprises a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, an input source device (input device) 40, an output destination device (display device) 50, and an information processing device 100 (100D). The input / output device 30 is configured to include an input source device (input device) 40 and an output destination device (display device) 50.

[0064] The information processing device 100 (100D) according to Embodiment 4 receives elements selected by the user and highlights those elements in the graph. The information processing device 100 (100D) shown in Figure 20 is composed of a data acquisition unit 110, a graph output unit 120, a selected element receiving unit 151, and an element emphasis unit 152. The selection element receiving unit 151 receives the selection element, which is an element selected in the output graph. The element emphasis unit 152 emphasizes the selected element in the output graph. The graph output unit 120 outputs a graph that includes the selected elements highlighted by the element highlighting unit 152.

[0065] The image of the functions realized by the information processing device 100D according to Embodiment 4 will be explained. Figure 21 is a diagram showing an example of the function (first example) realized by the information processing device according to Embodiment 4 of this disclosure. Figure 22 is a diagram showing an example of the function (second example) realized by the information processing device according to Embodiment 4 of this disclosure. Figure 21 shows a square-packing type symbol graph 4100 arranged in four quadrants. The square-packing type symbol graph 4100 shown in Figure 21 is processed so that when element 4110 is selected, the selected element 4110 is emphasized by the element emphasis unit 152 relative to the other elements. Figure 22 shows a Sankey diagram-style graph 4200. The Sankey diagram-style graph 4200 shown in Figure 22 is processed so that when element 4210 "0.2G" is selected, the selected element 4210 is emphasized by the element emphasis unit 152 relative to the other elements.

[0066] An example of processing using the information processing device (first configuration example) according to Embodiment 4 of this disclosure will be described. Figure 23 is a flowchart showing an example of the processing performed by the information processing device shown in Figure 20. The information processing device 100 (100D) according to this fourth embodiment starts the processing shown in Figure 21 ("Start") when, for example, the processing start conditions are met. The processing start conditions are, for example, when the information processing device 100D outputs graph data, or when the output graph is displayed on the output destination device 50. The information processing device 100D then performs an element selection determination process (step ST4110 "Is an element of the graph selected?"). In the element selection determination process, the selected element receiving unit 151 of the information processing device 100D determines whether it has received a selected element, which is an element selected in the output graph. If the selection element receiving unit 151 determines that it has not received a selected element in the output graph (step ST4110 "Is an element in the graph selected?" "NO"), the information processing device 100D proceeds to the termination determination process in step ST4150. When the information processing device 100D determines that the selection element receiving unit 151 has received a selection element, which is an element selected in the output graph (step ST4110 "Is an element in the graph selected?" "YES"), it then executes the selection element acquisition process (step ST4120 "Acquire selection element"). In the selection element acquisition process, the selection element receiving unit 151 of the information processing device 100D acquires information indicating the received selection element and outputs it to the element emphasis unit 152. The information processing device 100D then performs element emphasis processing (step ST4130 “element emphasis”). In element emphasis processing, the element emphasis unit 152 of the information processing device 100D emphasizes the selected element in the output graph. The element emphasis unit 152 processes the selected element to emphasize it relative to other elements based on the information indicating the selected element. The information processing device 100D then performs graph data output processing (step ST4140 "Output graph to output destination device"). In the graph data output processing, the graph output unit 120 of the information processing device 100D outputs graph data showing a graph in which the selected elements have been highlighted by the element highlighting unit 152 to the output destination device 50. The information processing device 100D then performs a termination determination process (step ST4150 "Termination?"). In the termination determination process, a control unit (not shown) of the information processing device 100D determines that the process should be terminated if conditions are met, for example, such as the graph output by the information processing device 100D being finished displaying on the output destination device 50. If the information processing device 100D determines that it does not want to terminate the process (step ST4150 "Termination?" "NO"), it returns to the process in step ST4110. If the information processing device 100D determines that it has finished processing (step ST4150 "Finish?" "YES"), it finishes the processing shown in Figure 23 ("Finish").

[0067] Next, a second configuration example of the information processing device according to Embodiment 4 of this disclosure will be described. Figure 24 is a diagram showing an example configuration (second configuration example) of an information processing device and an information processing system including the information processing device according to Embodiment 4 of the present disclosure. The information processing system (DMS) 1 (1D') shown in Figure 24 comprises a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100 (100D'). The input / output device 30 is configured to include an input source device (input device) 40 and an output destination device (display device) 50. The information processing device 100 (100D') according to Embodiment 4 receives an element selected by the user, selects related elements in addition to the selected element, and highlights the element in the graph. The information processing device 100 (100D') shown in Figure 24 includes a data acquisition unit 110, a graph output unit 120, a selected element receiving unit 156, a related element extraction unit 157, and an element emphasis unit 158. The selection element receiving unit 156 receives the selection element, which is an element selected in the output graph. When the selected element is received, the related element extraction unit 157 extracts related elements from the elements in the graph that are related to the information shown in the selected element, and adds the related elements to the selected element. As a result, the related elements are treated as selected elements in subsequent processing. The element emphasis unit 158 ​​processes the output graph to emphasize the selected elements, including the extracted related elements. The graph output unit 120 outputs a graph containing the elements emphasized by the element emphasis unit 158 ​​to the output destination device 50.

[0068] The image of the functions realized by the information processing device 100D' according to Embodiment 4 will be explained. The square packing type symbol graph 4100 with a four-quadrant arrangement shown in Figure 21 is processed so that when element 4110 is selected, in addition to the selected element 4110, related element 4120 is emphasized by the element emphasis unit 152 relative to the other elements. Furthermore, the quadrant region (related region 4130) in which the selected element 4110 is located is processed to emphasize it in relation to the regions of other testimonies. When the information processing device 100D' receives a selected element, it outputs a graph that has been processed to highlight related elements or related regions associated with the selected element. In this way, the information processing device 100D' can select elements linked to a single selected element. For example, in the case of a circular packing arrangement, the information processing device 100D' can add elements of the same size as the selected element as related elements to the selected element and select them as a group of elements. For example, in the case of a four-quadrant arrangement, the information processing device 100D' can add elements of the same phenomenon to the selection sheet as related elements and select them as a group of elements. The Sankey diagram-style graph 4200 shown in Figure 22 is processed so that when element 4210 "0.2G" is selected, the selected element 4210 is emphasized by the element emphasis unit 152 relative to the other elements. Furthermore, it is possible to select the risk factors related to the selectable element 4210 "Acceleration of 0.2G or more," namely "Distraction," "Napping," and "Telephone Call." In other words, when the information processing device 100D' receives a selected element, it outputs a graph that has been processed to highlight related elements or related regions associated with the selected element.

[0069] An example of processing using the information processing device (second configuration example) according to Embodiment 4 of this disclosure will be described. Figure 25 is a flowchart showing an example of the processing performed by the information processing device shown in Figure 24. The information processing device 100 (100D') according to this fourth embodiment starts the process shown in Figure 25 ("Start"), for example, in the case of... The information processing device 100D' then performs an element selection determination process (step ST4210 "Is an element of the graph selected?"). In the element selection determination process, the selected element receiving unit 156 of the information processing device 100D' determines whether it has received a selected element, which is an element selected in the output graph. If the selection element receiving unit 156 determines that it has not received a selected element in the output graph (step ST4210 "Is an element in the graph selected?" "NO"), the information processing device 100D' proceeds to the termination determination process in step ST4270. When the information processing device 100D' determines that the selection element receiving unit 156 has received a selection element, which is an element selected in the output graph (step ST4210 "Is an element in the graph selected?" "YES"), it then executes the selection element acquisition process (step ST4220 "Acquire selection element"). In the selection element acquisition process, the selection element receiving unit 156 of the information processing device 100D' acquires information indicating the received selection element and outputs it to the element highlighting unit 152. The information processing device 100D' then performs a related element extraction determination process (step ST4230 "Related element extraction?"). In the related element extraction determination process, the related element extraction unit 157 of the information processing device 100D' determines whether to extract related elements. If the related element extraction unit 157 determines that it does not extract related elements (step ST4230 "Related element extraction?" "NO"), the process proceeds to the element emphasis processing in step ST4250. If the information processing device 100D' determines that the related element extraction unit 157 should extract related elements (step ST4230 "Related element extraction?" "YES"), it then executes the related element extraction process (step ST4240 "Related element extraction"). In the related element extraction process, when the selected element is received, the related element extraction unit 157 of the information processing device 100D' extracts related elements from the elements in the graph that are related to the information shown in the selected element, and adds the related elements to the selected element. The information processing device 100D' then performs element emphasis processing (step ST4250 “Element Emphasis”). In element emphasis processing, the element emphasis unit 158 ​​of the information processing device 100D' processes the output graph to emphasize the selected elements, including the extracted related elements. The information processing device 100D' then performs graph data output processing (step ST4260 "Output graph to output destination device"). In the graph data output processing, the graph output unit 120 of the information processing device 100D' outputs a graph containing elements highlighted by the element highlighting unit 158 ​​to the output destination device 50. The information processing device 100D' then performs a termination determination process (step ST4270 "Termination?"). In the termination determination process, the control unit (not shown) of the information processing device 100D' determines to terminate the process if conditions are met, for example, such as the graph output by the information processing device 100D' being finished displaying on the output destination device 50. If the information processing device 100D' determines that it does not want to terminate the process (step ST4270 "Termination?" "NO"), it returns to the process in step ST4210. If the information processing device 100D' determines that it has finished processing (step ST4270 "Finish?" "YES"), it finishes the processing shown in Figure 25 ("Finish").

[0070] According to the above configuration, highlighting specific relevant elements makes it easier for users to explore the data. Furthermore, the system can be visualized based on the user's actions, allowing them to identify related elements and extract and select those elements. For example, in the four-quadrant arrangement described in Embodiment 2, selecting one element will extract related elements (the attributes to be extracted can be pre-registered). Furthermore, the extracted elements are presented in a way that links them to the selected location. For example, in a Sankey diagram, selecting 0.2G or higher will extract elements related to the relevant driving behavior (distracted driving, near misses, etc.). Furthermore, it is possible to extract only the selected elements and change the way they are represented in the graph. Furthermore, methods for understanding relationships include highlighting selected elements (highlights, drop shadows, etc.). Alternatively, it's possible to manipulate elements in a chain-like fashion to understand their relationships.

[0071] This embodiment further illustrates an example of a configuration that can be expressed as follows. [6] A selection element receiving unit that receives the selected element in the output graph, An element emphasis expression unit that highlights the selected element in the output graph, Equipped with, An information processing device according to any one of [1][2][3][4][5], characterized in that... This disclosure provides an information processing device that makes it easier for users to understand the relationships between data series of data detected while a moving object is in motion, by highlighting elements in the graph. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0072] This embodiment further illustrates an example of a configuration that can be expressed as follows. [7] A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. An element emphasis expression unit processes the output graph to highlight the selected elements, including the extracted related elements. Equipped with, An information processing device according to any one of [1][2][3][4][5][6], characterized in that... This disclosure provides an information processing device that makes it easier for users to understand the relationships between data series of data detected while a moving object is in motion, by highlighting related elements along with the selected element when the user selects an element in the graph. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0073] Embodiment 5. In the embodiments described above, a form that emphasizes the selected element was explained. Embodiment 5 further describes a configuration example in which selected elements can be extracted and used as target elements for graph plotting. In Embodiment 5, components related to Embodiment 5 that are the same as those related to Embodiments 1, 2, 3, or 4 already described are denoted by the same reference numerals or similar numerals, and redundant explanations may be omitted as appropriate.

[0074] Next, an example of the configuration of the information processing device according to Embodiment 5 of this disclosure will be described. Figure 26 is a diagram showing an example configuration of an information processing device according to Embodiment 5 of this disclosure and an information processing system including said information processing device. The information processing system (DMS) 1(1E) shown in Figure 26 comprises a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100(100E). The input / output device 30 shown in Figure 26 is composed of an input source device (input device) 40 and an output destination device (display device) 50.

[0075] The information processing device 100 (100E) generates a graph using selected elements or related elements associated with the selected elements. The information processing device 100 (100E) shown in Figure 26 includes a data acquisition unit 110, a graph output unit 120, a selection element receiving unit 161, a related element extraction unit 162, a graph type command receiving unit 163, an element information acquisition unit 164, and a graph type changing unit 165. The data acquisition unit 110 has the same functions as the data acquisition unit 110 described earlier. The selection element receiving unit 161 receives the selection element, which is an element selected in the output graph. When the selected element is received, the related element extraction unit 162 extracts related elements from the elements in the graph that are related to the information shown in the selected element, and adds the related elements to the selected element. The graph type command receiving unit 163 receives the commanded graph type, which is the commanded graph type. The graph type command receiving unit 163 receives the commanded graph type, which is the commanded graph type, from among the graph types (graph categories) that have been registered in advance. The element information acquisition unit 164 acquires selected element information, including detection data for each selected element. The graph type changing unit 165 uses the selected element information to change and generate a graph of the graph type corresponding to the commanded graph type, and outputs the generated graph. The graph output unit 120 outputs the graph generated by changing the graph type using the graph type change unit 165 to the output destination device 50.

[0076] An example of processing by the information processing device according to Embodiment 5 of this disclosure will be described. The information processing device 100 (100E) according to Embodiment 5 executes a process that combines a part of the processing of the information processing device 100D according to Embodiment 4 or a part of the processing of the information processing device 100D' with the processing of the information processing device 100C according to Embodiment 3. The information processing device 100E starts processing when, for example, the processing start conditions are met (see "Start" in Figure 23). The processing start conditions include, for example, when the information processing device 100E outputs graph data, or when the output graph is displayed on the output destination device 50. The information processing device 100E then performs an element selection determination process (see step ST4110 “Has a graph element been selected?” in Figure 23). In the element selection determination process, the selected element receiving unit 161 of the information processing device 100E determines whether it has received a selected element, which is an element selected in the output graph. If the selection element receiving unit 161 determines that it has not received a selected element in the output graph (see step ST4110 "Is an element in the graph selected?" "NO"), the information processing device 100E proceeds to the termination determination process in step ST4150. If the information processing device 100E determines that the selection element receiving unit 156 has received a selection element, which is an element selected in the output graph (see step ST4110 "Is an element in the graph selected?" "YES"), it then executes the selection element acquisition process (see step ST4120 "Acquire selection element" in Figure 23). In the selection element acquisition process, the selection element receiving unit 161 of the information processing device 100E receives the selection element, which is an element selected in the output graph. Here, if related elements are to be included in the selected elements, the related element extraction unit 162, upon receiving the selected elements, extracts related elements from the elements in the graph that are related to the information shown in the selected elements, and adds the related elements to the selected elements (see step ST4240 in Figure 25). The information processing device 100E then performs a graph type change command reception determination process (see step ST3110 “Graph type change command reception?” in Figure 19). In the graph type change command reception determination process, the graph type command reception unit 163 of the information processing device 100E determines whether it has received the commanded graph type. If the information processing device 100E determines that the graph type command receiving unit 163 has not received a command graph type (see step ST3110 “Graph type change command received?” “NO” in Figure 19), it proceeds to the termination determination process in step ST3150. If the information processing device 100E determines that the graph type command receiving unit 163 has received a command graph type (see step ST3110 “Graph type change command received?” “YES” in Figure 19), it then executes the command graph type acquisition process (see step ST3120 “Command graph type acquisition” in Figure 19). In the command graph type acquisition process, the graph type command receiving unit 163 of the information processing device 100E receives the command graph type, which is the commanded graph type. The graph type command receiving unit 163 receives the command graph type input by the user via the input source device 40. The information processing device 100E then performs an element information acquisition process. In the element information acquisition process, the element information acquisition unit 164 acquires selected element information, including detection data for each selected element received in the process of step ST4240. The information processing device 100E then performs a graph type change process (see step ST3130 “Graph Type Change” in Figure 19). In the graph type change process, the graph type change unit 165 of the information processing device 100E uses the detection data of the selected elements shown in the output graph to change the graph to the graph type corresponding to the command graph type. For example, the graph type change unit 165 uses the detection data shown in the output graph (symbol graph) to change the graph to the graph type corresponding to the command graph type. The information processing device 100E then performs graph data output processing (see step ST3140 “Output graph to output destination device” in Figure 19). In the graph data output processing, the graph output unit 120 of the information processing device 100E outputs the graph data of the graph that has been changed by the graph type change unit 165 to the output destination device 50. The information processing device 100E then performs a termination determination process (see step ST3150 “Termination?” in Figure 19). In the termination determination process, a control unit (not shown) of the information processing device 100E determines to terminate the process when conditions are met, such as when the graph output by the information processing device 100E has finished being displayed on the output destination device 50. If the information processing device 100E determines that it does not want to terminate the process (see step ST3150 "Termination?" "NO" in Figure 19), it returns to the element selection determination process of step ST4110 in this embodiment. If the information processing device 100E determines that it is time to terminate the process (see step ST3150 "Termination?" "YES" in Figure 19), it terminates the process ("Termination").

[0077] This embodiment further illustrates an example of a configuration that can be expressed as follows. [8] A selection element receiving unit that receives the selected element in the output graph, A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, An element information extraction unit that acquires selected element information including detection data for each of the selected elements, A graph type changing unit that uses the selected element information to change the graph to a graph type corresponding to the command graph type and outputs it, Equipped with, An information processing device according to any one of [1][2][3][4][5][6], characterized in that... This disclosure provides an information processing device that enables users to more easily understand the relationships between data series of data detected while a moving object is in motion, by allowing them to view the data in a commanded format using elements selected by the user in the graph. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0078] This embodiment further illustrates an example of a configuration that can be expressed as follows. [9] A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, A selection element information extraction unit that acquires selection element information including detection data for each of the aforementioned selection elements, A graph type changing unit that uses the selected element information to change the graph to a graph type corresponding to the command graph type and outputs it, Equipped with, An information processing device according to any one of [1][2][3][4][5][6][7], characterized in that... This disclosure provides an information processing device that enables users to more easily understand the relationships between data series of data detected while a moving object is in motion, by allowing them to view the elements selected by the user in the graph and the elements related to the selected elements in a graph in a commanded format. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0079] Embodiment 6. In contrast to the embodiments described above, Embodiment 6 further describes a configuration example in which, when video is stored as detection data, the video is played back in an order that takes into account the type of graph representing the selected element. In Embodiment 6, for example, a configuration is envisioned in which, in response to a user operation (such as a right-click operation), supplementary information corresponding to the specified element or multiple elements is displayed, and the display or video playback is performed according to the characteristics of the graph type. Furthermore, in Embodiment 6, for example, if multiple elements are selected, the configuration is envisioned such that if it is a time-series type, the video is played in the order of the timeline, and if it is a circular packing type symbol graph (normal type), the video is played in order of increasing importance. In Embodiment 6, components related to Embodiment 6 that are the same as those related to Embodiments 1, 2, 3, or 4 already described are denoted by the same reference numerals or similar numerals, and redundant explanations may be omitted as appropriate.

[0080] Next, an example of the configuration of the information processing device according to Embodiment 6 of this disclosure will be described. Figure 27 is a diagram showing an example configuration of an information processing device according to Embodiment 6 of this disclosure and an information processing system including said information processing device. The information processing system (DMS) 1(1F) shown in Figure 27 is composed of a sensor device (sensor group) 10, a storage device (data storage unit) 20, an input / output device 30, and an information processing device 100(100F). The sensor device (sensor group) 10 includes an imaging device. The imaging device includes an imaging device that images the area around the moving object, an imaging device that images the interior of the room where the moving object is located, or both. The memory device (data storage unit) 20 stores the video footage captured by the camera. The video footage is stored along with the time and other information, and is linked to and stored with other detection data. The input / output device 30 shown in Figure 27 is composed of an input source device (input device) 40 and an output destination device (display device) 50. The information processing device 100 (100F) further plays back the video footage stored in association with the selected elements in a playback order corresponding to the graph type. The information processing device 100 (100F) shown in Figure 27 includes a data acquisition unit 110, a graph output unit 120, a selection element receiving unit 171, a related element extraction unit 172, a supplementary information output unit 180, and a video output unit 190. The selection element receiving unit 171 receives the selection element, which is an element selected in the output graph. When the selected element is received, the related element extraction unit 172 extracts related elements from the elements in the graph that are related to the information shown in the selected element, and adds the related elements to the selected element. As a result, in subsequent processing, the related elements are treated as selected elements. The supplementary information output unit 180, when an element is selected, adds supplementary information to the graph that indicates the selected element, related elements, or both, and outputs it. The video output unit 190 acquires the graph type of the graph containing the selected elements and outputs the video associated with each selected element in an order according to the conditions based on the graph type. The video footage consists of images taken from a moving vehicle showing the outside, images taken from inside the vehicle, or images taken from a moving vehicle showing the outside and images taken from inside the vehicle.

[0081] Figure 28 is a diagram showing an example of the configuration of the video output unit in the information processing device according to Embodiment 6 of the present disclosure. The video output unit 190 shown in Figure 28 includes a graph type acquisition unit 191, a playback order determination unit 192, a time-series playback unit 193, and a severity-order playback unit 194. The graph type acquisition unit 191 acquires the graph type of the graph containing the selected element. The playback order determination unit 192 determines the playback order according to the graph type based on the graph type. For example, the playback order determination unit 192 determines the playback order of the video using playback order information corresponding to a pre-set graph type. The time-series playback unit 193 outputs images related to the elements in the order of the timeline if the graph containing the selected elements is a time-series graph. The severity-order playback unit 194 outputs images related to the elements in descending order of severity, if the graph containing the selected elements is a symbol graph.

[0082] The image of the functions realized by the information processing device according to Embodiment 6 will be explained. Figure 29 is a diagram showing an example of the function (first example) realized by the information processing device according to Embodiment 6 of this disclosure. Figure 29 shows a symbol graph (circle-packed symbol graph) 6100 and a video playback order image (ordered by severity) 6200. In the case of the symbol graph (circle-packed symbol graph) 6100, for example, as shown in the video playback order image (ordered by severity) 6200, the video is played in order of magnitude (highest severity) of the elements. Figure 30 is a diagram showing an example of the function (second example) realized by the information processing device according to Embodiment 6 of this disclosure. Figure 30 shows a time-series graph 6300 and a video playback order image (time-series order) 6400. When the time-series graph 6300 is displayed, the video is played back in the order of the event occurrence times, as shown in the video playback order image (time-series order) 6400.

[0083] An example of processing by the information processing device according to Embodiment 6 of this disclosure will be described. Figure 31 is a flowchart showing an example of processing in an information processing device according to Embodiment 6 of this disclosure. The information processing device 100 (100F) according to this embodiment 6 starts the processing shown in Figure 31 ("Start") when, for example, the processing start conditions are met. The processing start conditions are, for example, when the information processing device 100F receives a request to select an element for a graph that is displayed on the output destination device 50 based on the graph data output by the information processing device 100F. The information processing device 100F then performs a supplementary information output determination process (step ST6110 "Supplementary information output?"). In the supplementary information output determination process, the supplementary information output unit 180 of the information processing device 100F determines whether to add supplementary information to the selected element or the selected element including related elements and output it. If the information processing device 100F determines that the supplemental information output unit 180 should output supplemental information (step ST6110 "Supplemental information output?" "YES"), it then executes a supplemental information acquisition process (step ST6120 "Supplemental information acquisition"). In the supplemental information acquisition process, the supplemental information output unit 180 of the information processing device 100F acquires supplemental information that indicates a selected element, a related element, or both. The information processing device 100F then performs supplemental information output processing (step ST6130 “Supplemental Information Output”). In supplemental information output processing, the supplemental information output unit 180 of the information processing device 100F adds supplemental information to the graph and outputs it. The graph output unit 120 outputs the graph data with the added supplemental information to the output destination device 50. The information processing device 100F then performs a video output determination process (step ST6140 "Video output?"). In the video output determination process, the video output unit 190 of the information processing device 100F determines whether to output video depending on whether the setting for video output is enabled. If the information processing device 100F determines that it should output video (step ST6140 "Video Output?" "YES"), it then executes a video acquisition process (step ST6150 "Video Acquisition"). In the video acquisition process, the video output unit 190 of the information processing device 100F acquires video stored in the storage device 20 based on element information related to the selected element. The information processing device 100F then performs video output processing (step ST6160 "Video Output"). In the video output processing, the video output unit 190 of the information processing device 100F outputs the acquired video in playback order according to the graph type.

[0084] Here, we will explain a detailed example of video output processing. Figure 32 is a flowchart showing a detailed example of video output processing in an information processing device according to Embodiment 6 of this disclosure. In the video output unit 190 of the information processing device 100 (100F) according to this embodiment 6, for example, in step ST6160 shown in Figure 31, the process shown in Figure 32 is started ("start"). The video output unit 190 of the information processing device 100F then performs a graph type acquisition process (step ST6210 "Graph Type Acquisition"). In the graph type acquisition process, the graph type acquisition unit 191 of the video output unit 190 acquires the graph type of the graph containing the selected elements. The graph type acquisition unit 191 outputs graph type information indicating the graph type to the playback order determination unit 192. The video output unit 190 of the information processing device 100F then performs a playback order determination process (step ST6220 "Determine the playback order according to the graph type"). In the playback order determination process, the playback order determination unit 192 of the video output unit 190 determines the playback order of the video using playback order information corresponding to a pre-set graph type. The playback order determination unit 192 outputs playback order information indicating the playback order of the video to the playback order function unit that determined the playback order. In this embodiment, it will be explained assuming that the order is determined in chronological order or in order of severity. If the playback order determination unit 192 of the information processing device 100F decides to play the video in chronological order (step ST6220 "Determine playback order according to graph type" "Chronological order"), the video output unit 190 then executes the chronological playback process (step ST6230 "Play in chronological order"). In the chronological playback process, if the graph containing the selected elements is a chronological graph, the chronological playback unit 193 of the video output unit 190 outputs the video related to the elements in the order of the timeline. If the playback order determination unit 192 of the information processing device 100F determines to play the images in order of severity (step ST6220 "Determine playback order according to graph type" "Severity order"), the video output unit 190 then executes the severity order playback process (step ST6240 "Play in order of severity"). In the severity order playback process, if the graph containing the selected elements is a symbol graph, the severity order playback unit 194 of the video output unit 190 outputs images related to the elements in order of decreasing severity. For example, the severity order playback unit 194 plays the images in descending order of the detected data values ​​shown for the elements of the graph. When the video output unit 190 of the information processing device 100F plays back a video, it then terminates the process shown in Figure 32 ("terminate"). Returning to the explanation of Figure 31. The information processing device 100F then performs a termination determination process (step ST6170 "Termination?"). In the termination determination process, a control unit (not shown) of the information processing device 100F determines that the process should be terminated if conditions are met, for example, such as the graph output by the information processing device 100F being finished displaying on the output destination device 50. If the information processing device 100F determines that it does not want to terminate the process (step ST6170 "Termination?" "NO"), it returns to the process in step ST6110. If the information processing device 100F determines that it has finished processing (step ST6170 "Finish?" "YES"), it finishes the processing shown in Figure 31 ("Finish").

[0085] According to the configuration described above, the video is played in a playback order corresponding to the graph type, eliminating the need for the user to consider or instruct the video playback order. This helps the user more easily understand the relationships between data series of data detected while the moving object is in motion.

[0086] This embodiment further illustrates an example of a configuration that can be expressed as follows.

[10] A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. A video output unit that obtains the graph type of the graph containing the selected elements and outputs the video associated with each selected element in an order according to the conditions based on the graph type, Equipped with, An information processing device according to any one of [1][2][3][4][5][6][7][8][9], characterized in that... This disclosure has the effect of providing an information processing device that enables users to view images related to elements selected in a graph, thereby helping users more easily understand the relationships between data series of data detected while a moving object is in motion. Furthermore, by applying the above-mentioned information processing device to the DMS, it is possible to provide a DMS that, similarly to the above, can help users more easily understand the relationships between data series of data detected while the moving object is in motion. Furthermore, by applying the above configuration to a system including an information processing device, the above display method, or the above program, the same effects as described above can be achieved.

[0087] Here, we will explain the hardware configuration required to realize the functions of this disclosure described above. Figure 33 shows a first example of a hardware configuration for realizing the functions of the configuration of this disclosure. Figure 34 shows a second example of a hardware configuration for realizing the functionality of the configuration described herein. Each of the information processing devices 100 (100A, 100B, 100C, 100D, 100D', 100E, 100F) in this disclosure is implemented by hardware as shown in Figure 33 or Figure 34.

[0088] The information processing device 100 (100A, 100B, 100C, 100D, 100D', 100E, 100F) is composed of, for example, a processor 10001, a memory 10002, an input / output interface 10003, and a communication circuit 10004, as shown in Figure 33. The processor 10001 and memory 10002 are, for example, components installed in a computer. The memory 10002 contains the following components for the computer: data acquisition unit 110, mobile data acquisition unit 111, crew data acquisition unit 112, graph output unit 120, symbol graph generation unit 121, symbol graph type setting unit 122, data aggregation unit 123, symbol generation unit 124, symbol placement unit 125, iconization unit 126, placement condition command reception unit 131, symbol placement change unit 132, graph type command reception unit 141, graph type change unit 142, and selection element reception unit. Programs for the following functions are stored: attachment unit 151, element emphasis unit 152, selected element receiving unit 156, related element extraction unit 157, element emphasis unit 158, selected element receiving unit 161, related element extraction unit 162, graph type command receiving unit 163, element information acquisition unit 164, graph type change unit 165, supplementary information output unit 180, video output unit 190, graph type acquisition unit 191, time-series playback unit 193, severity-order playback unit 194, and a control unit (not shown). The processor 10001 reads and executes the program stored in memory 10002, which enables the following functions: data acquisition unit 110, mobile object data acquisition unit 111, crew data acquisition unit 112, graph output unit 120, symbol graph generation unit 121, symbol graph type setting unit 122, data aggregation unit 123, symbol generation unit 124, symbol placement unit 125, iconization unit 126, placement condition command reception unit 131, symbol placement change unit 132, graph type command reception unit 14 1. The functions of the graph type change unit 142, the selected element reception unit 151, the element emphasis unit 152, the selected element reception unit 156, the related element extraction unit 157, the element emphasis unit 158, the selected element reception unit 161, the related element extraction unit 162, the graph type command reception unit 163, the element information acquisition unit 164, the graph type change unit 165, the supplementary information output unit 180, the video output unit 190, the graph type acquisition unit 191, the time-series playback unit 193, the severity-order playback unit 194, and the control unit (not shown) are realized. Furthermore, a data storage unit (storage device) and a memory unit (not shown) are realized by memory 10002 or other memory (not shown). Furthermore, a communication unit (not shown) is realized by the communication circuit 10004.

[0089] Processor 10001 uses, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a microprocessor, a microcontroller, or a DSP (Digital Signal Processor). Memory 10002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. The processor 10001 and the memory 10002 or the communication circuit 10004 are connected in a way that allows them to transmit data to each other. Furthermore, the processor 10001, the memory 10002, and the communication circuit 10004 are connected to other hardware via the input / output interface 10003 in a way that allows them to transmit data to each other.

[0090] Alternatively, in the information processing device 100 (100A, 100B, 100C, 100D, 100D', 100E, 100F), the data acquisition unit 110, the mobile data acquisition unit 111, the crew data acquisition unit 112, the graph output unit 120, the symbol graph generation unit 121, the symbol graph type setting unit 122, the data aggregation unit 123, the symbol generation unit 124, the symbol placement unit 125, the iconization unit 126, the placement condition command reception unit 131, the symbol placement change unit 132, the graph type command reception unit 141, and the graph type change unit 14 2. The functions of the selected element receiving unit 151, the element emphasis unit 152, the selected element receiving unit 156, the related element extraction unit 157, the element emphasis unit 158, the selected element receiving unit 161, the related element extraction unit 162, the graph type command receiving unit 163, the element information acquisition unit 164, the graph type change unit 165, the supplementary information output unit 180, the video output unit 190, the graph type acquisition unit 191, the time-series playback unit 193, the severity-order playback unit 194, and the control unit (not shown) may be implemented by a dedicated processing circuit 20001, as shown in Figure 34.

[0091] The processing circuit 20001 may utilize, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), an FPGA (Field-Programmable Gate Array), a SoC (System-on-a-Chip), or a system LSI (Large-Scale Integration). Furthermore, a data storage unit (storage device) and a memory unit (not shown) are realized by memory 20002 or other memory (not shown). Memory 20002 may be a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable Read Only Memory), or flash memory; it may be a magnetic disk such as a hard disk or flexible disk; it may be an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc); or it may be a magneto-optical disk. Furthermore, a communication unit (not shown) is realized by the communication circuit 20004. The processing circuit 20001 and the memory 20002 or the communication circuit 20004 are connected in a way that allows them to transmit data to each other. Furthermore, the processing circuit 20001, the memory 20002, and the communication circuit 20004 are connected in a way that allows them to transmit data to other hardware via the input / output interface 20003. Furthermore, in the information processing device 100 (100A, 100B, 100C, 100D, 100D', 100E, 100F), the data acquisition unit 110, the mobile data acquisition unit 111, the crew data acquisition unit 112, the graph output unit 120, the symbol graph generation unit 121, the symbol graph type setting unit 122, the data aggregation unit 123, the symbol generation unit 124, the symbol placement unit 125, the iconization unit 126, the placement condition command reception unit 131, the symbol placement change unit 132, the graph type command reception unit 141, and the graph type change unit 1 42. The functions of the selected element receiving unit 151, the element emphasis unit 152, the selected element receiving unit 156, the related element extraction unit 157, the element emphasis unit 158, the selected element receiving unit 161, the related element extraction unit 162, the graph type command receiving unit 163, the element information acquisition unit 164, the graph type change unit 165, the supplementary information output unit 180, the video output unit 190, the graph type acquisition unit 191, the time-series playback unit 193, the severity-order playback unit 194, and the control unit (not shown) may be implemented by separate processing circuits or by a single processing circuit.

[0092] Alternatively, in the information processing device 100 (100A, 100B, 100C, 100D, 100D', 100E, 100F), the data acquisition unit 110, the mobile data acquisition unit 111, the crew data acquisition unit 112, the graph output unit 120, the symbol graph generation unit 121, the symbol graph type setting unit 122, the data aggregation unit 123, the symbol generation unit 124, the symbol placement unit 125, the iconization unit 126, the placement condition command reception unit 131, the symbol placement change unit 132, the graph type command reception unit 141, the graph type change unit 142, the selected element reception unit 151, the element Some functions of the emphasis unit 152, selection element receiving unit 156, related element extraction unit 157, element emphasis unit 158, selection element receiving unit 161, related element extraction unit 162, graph type command receiving unit 163, element information acquisition unit 164, graph type change unit 165, supplementary information output unit 180, video output unit 190, graph type acquisition unit 191, time-series playback unit 193, severity-order playback unit 194, and control unit (not shown) may be implemented by the processor 10001 and memory 10002, while the remaining functions are implemented by the processing circuit 20001.

[0093] Within the scope of this disclosure, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component of each embodiment.

[0094] This disclosure makes it possible to help users easily understand the relationships between data series of data detected while a moving object is in motion, and is therefore suitable for use in, for example, driving analysis devices that analyze data detected while a moving object is in motion, driving support devices that use the analysis results to support driving, DMS, etc. Furthermore, in the field of web services for connected dashcams, it is suitable for use in order to prevent accidents among drivers within an organization by clearly visualizing trends in vehicle behavior (such as sudden acceleration and deceleration) and driver risk factors (such as drowsiness and distraction). [Explanation of symbols]

[0095] 1(1A,1B,1C,1D,1D',1E,1F) Information Processing System (DMS), 10 Sensor Device (Sensor Group), 20 Storage Device (Data Storage Unit), 30 Input / Output Device, 40 Input Source Device (Input Device), 50 Output Destination Device (Display Device), 100(100A,100B,100C,100D,100D',100E,100F) Information Processing Device, 110 Data Acquisition Unit, 111 Mobile Object Data Acquisition Unit, 112 Crew Data Acquisition Unit, 120 Graph Output Unit, 121 Symbol Graph Generation Unit, 122 Symbol Graph Type Setting Unit, 123 Data Aggregation Unit, 124 Symbol Generation Unit, 125 Symbol Placement Unit, 126 Iconization Unit, 131 Placement Condition Command Reception Unit, 132 Symbol Placement Change Unit, 141 Graph Type Command Reception Unit, 142 Graph type change unit, 151 Selected element reception unit, 152 Element emphasis unit, 156 Selected element reception unit, 157 Related element extraction unit, 158 Element emphasis unit, 161 Selected element reception unit, 162 Related element extraction unit, 163 Graph type command reception unit, 164 Element information acquisition unit, 165 Graph type change unit, 171 Selected element reception unit, 172 Related element extraction unit, 180 Supplementary information output unit, 190 Video output unit, 191 Graph type acquisition unit, 192 Playback order determination unit, 193 Time series playback unit, 194 Severity order playback unit, 1110 Symbol (Example of a symbol representing the first combination with shape and numerical value), 1120 Symbol (Example of a symbol representing the second combination with shape and numerical value), 1130 Symbol (Example of a symbol representing the first combination with shape and color), 1140 Symbols (examples of symbols representing the second combination with shape and color), 1210, 1220, 1230,1240 Symbol (Example of a symbol including the representation of zero values), 1290' Symbol (Example of a symbol not including the representation of zero values), 1300 Linear symbol graph, 1400 Circle packing symbol graph, 1500 Square packing symbol graph, 2100 Center-first arrangement circle packing symbol graph, 2200 Causal arrangement circle packing symbol graph, 2300 Four-quadrant arrangement circle packing symbol graph, 2400 Minimum area arrangement square packing symbol graph, 2500 Four-quadrant arrangement square packing symbol graph (Example 1), 2600 Four-quadrant arrangement square packing symbol graph (Example 2), 3100 Circle packing symbol graph, 3200 Acceleration type graph (Sankey diagram type graph), 3300 Time series type graph, 4100 Four-quadrant arrangement square packing symbol graph, 4110 Elements (selected elements), 4120 Elements (related elements), 4130 Related range (example of supplementary information representation), 4200 Sankey diagram format graph, 4210 Elements (selected elements), 4220 Elements (related elements), 6100 Symbol graph (circle packing format symbol graph), 6200 Video playback order image (by severity), 6300 Time series format graph, 6400 Video playback order image (by time series), 10001 Processor, 10002 Memory, 10003 Input / Output interface, 10004 Communication circuit, 20001 Processing circuit, 20002 Memory, 20003 Input / Output interface, 20004 Communication circuit.

Claims

1. A data acquisition unit that acquires multiple types of detection data, which are data detected by sensors while a moving object is in operation, A graph output unit that outputs a symbol graph composed of symbols generated by combining multiple types of the aforementioned detection data, Equipped with an information processing device.

2. A placement condition command receiving unit that receives commanded placement conditions, which are the commanded placement conditions, A symbol arrangement changing unit that changes the arrangement of symbols in the symbol graph according to the aforementioned command arrangement conditions, Equipped with, The graph output unit outputs a symbol graph in which the arrangement of symbols has been changed according to the command arrangement conditions. The information processing apparatus according to feature 1.

3. The elements of the symbol graph output by the aforementioned graph output unit are represented by icons corresponding to the detected data for each element. The information processing apparatus according to claim 1 or 2.

4. The graph output unit outputs a symbol graph consisting of symbols representing data that combines mobile data, which is data detecting the state or surrounding conditions of the mobile body, and occupant data, which is data detecting the state of the occupants of the mobile body. The information processing apparatus according to claim 1 or 2.

5. A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, A graph type changing unit that uses the detection data shown in the output graph to change the graph to a graph type corresponding to the command graph type and output it, Equipped with, The information processing apparatus according to claim 1 or 2.

6. A selection element receiving unit that receives the selected element in the output graph, An element emphasis unit that highlights the selected element in the output graph, Equipped with, The information processing apparatus according to claim 1 or 2.

7. A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. The output graph includes an element emphasis unit that highlights the selected elements, including the extracted related elements, Equipped with, The information processing apparatus according to claim 1 or 2.

8. A selection element receiving unit that receives the selected element in the output graph, A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, An element information extraction unit that acquires selected element information including detection data for each of the selected elements, A graph type changing unit that uses the selected element information to change the graph to a graph type corresponding to the command graph type and outputs it, Equipped with, The information processing apparatus according to claim 1 or 2.

9. A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. A graph type command receiving unit that receives the commanded graph type, which is the commanded graph type, An element information extraction unit that acquires selected element information including detection data for each of the selected elements, A graph type changing unit that uses the selected element information to change the graph to a graph type corresponding to the command graph type and outputs it, Equipped with, The information processing apparatus according to claim 1 or 2.

10. A selection element receiving unit that receives the selected element in the output graph, When the aforementioned selection element is accepted, the related element extraction unit extracts related elements from the elements in the graph that are related to the information shown in the selection element, and adds the related elements to the selection element. A video output unit that obtains the graph type of the graph containing the selected elements and outputs the video associated with each selected element in an order according to the conditions based on the graph type, Equipped with, The information processing apparatus according to claim 1 or 2.

11. A display method that uses an information processing device to display information on a display device, The aforementioned information processing device acquires multiple types of detection data, which are data detected by sensors while the moving object is in operation. The information processing device outputs a symbol graph, which is composed of symbols generated by combining multiple types of the detection data, to the display device. A method of display characterized by the following features.

12. Computers, A data acquisition unit that acquires multiple types of detection data, which are data detected by sensors while a moving object is in operation, A graph output unit that outputs a symbol graph composed of symbols generated by combining multiple types of the aforementioned detection data, A program characterized by operating as an information processing device equipped with [a specific feature].

Images