In-vehicle devices and systems

Abstract

To provide a technique for collecting information related to an operation feeling of an on-vehicle device for providing multi-media information to an occupant of a vehicle at suitable timing.SOLUTION: An on-vehicle device provides multi-media information to an occupant of a vehicle. When first voice indicating discomfort of the occupant is included in voice uttered from the occupant during the operation of the on-vehicle device by the occupant, the on-vehicle device transmits an operation log of the on-vehicle device during a predetermined period including a point of time when the voice is acquired to a server device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to in-vehicle devices and systems that provide multimedia information to vehicle occupants. [Background technology]

[0002] Patent Document 1 discloses a vehicle display system. The vehicle display system described in Patent Document 1 includes a display and a multimedia information unit. The display is provided in the vehicle so that it can be seen by the driver of the vehicle. The multimedia information acquisition unit acquires multimedia information. The multimedia information is information in the form of text or images. The display displays the multimedia information so that it overlaps with other vehicles included in information representing the external environment of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-133260 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide a technology that can collect, at an appropriate timing, information about the operational feel of an in-vehicle device that provides multimedia information to vehicle occupants. [Means for solving the problem]

[0005] An in-vehicle device according to a first aspect of the present disclosure includes: An in-vehicle device for providing multimedia information to a vehicle occupant, comprising: The vehicle control unit is provided with a control unit that, when the voice emitted by the occupant while the occupant is operating the vehicle-mounted device includes a first voice indicating the occupant's discomfort, transmits to a server device an operation log of the vehicle-mounted device for a predetermined period including the time when the voice was acquired.

[0006] A system according to a second aspect of the present disclosure includes: A system including an in-vehicle device that provides multimedia information to a vehicle occupant, and a server device that communicates with the in-vehicle device via a network, If the voice emitted by the occupant while the occupant is operating the in-vehicle device includes a first voice indicating the occupant's discomfort, the in-vehicle device transmits to the server device an operation log of the in-vehicle device for a predetermined period including the time when the voice was acquired.

[0007] Another aspect of the present disclosure is a method executed by the above-described in-vehicle device, a program for causing a computer to execute the method, or a non-transitory storage medium storing the program. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to collect information about the operational feel of an in-vehicle device that provides multimedia information to vehicle occupants at an appropriate timing. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an information management system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of an operation unit in an in-vehicle device that is operated by a passenger. [Figure 3] FIG. 3 is a diagram for explaining the flow of information collection by the information management system. [Figure 4] FIG. 4 is a block diagram illustrating an example of a functional configuration of the in-vehicle device. [Figure 5] FIG. 5 is a diagram showing log information stored in the log information database. [Figure 6] FIG. 6 is a flowchart showing the flow of information processing executed in the in-vehicle device. [Figure 7]FIG. 7 is a flowchart showing the flow of information processing executed in the management server. DETAILED DESCRIPTION OF THE INVENTION

[0010] The in-vehicle device according to the first aspect of the present disclosure provides multimedia information to vehicle occupants. The multimedia information may include, for example, map information, information about a route to a destination, information about the vehicle status, information about the vehicle's surroundings, traffic information, news, SNS (Social Networking Service), email, advertisements, or entertainment information (music, television, etc.). The multimedia information is information including images (still images or moving images), text, or audio within a vehicle in which an in-vehicle device is installed.

[0011] The in-vehicle device is a device operated by a vehicle occupant. The occupant may operate the in-vehicle device via a touch panel display or a hardware button, for example. The occupant may operate the in-vehicle device by speaking. The in-vehicle device can communicate with a server device via a network. The server device is a device installed outside the vehicle in which the in-vehicle device is installed.

[0012] The in-vehicle device includes a control unit. When a first voice is included in the voice uttered by a vehicle occupant while the vehicle occupant is operating the in-vehicle device, the control unit transmits an operation log of the in-vehicle device for a predetermined period to a server device. Here, the first voice is a voice indicating discomfort of the occupant. The predetermined period is a period including the time when the voice including the first voice was acquired.

[0013] When a vehicle occupant operates an in-vehicle device, for example, if the occupant finds it difficult to perform an intended operation or if it is difficult to understand how to operate the device, the occupant may feel uncomfortable operating the in-vehicle device. Furthermore, if the occupant feels uncomfortable operating the in-vehicle device, the occupant may emit a first voice. The in-vehicle device acquires voices including the first voice emitted in such cases.

[0014] If the cause of the first sound indicating discomfort from the vehicle occupant is an operation of the in-vehicle device, it is highly likely that the operation log for the predetermined period contains a log of the operation of the in-vehicle device that made the occupant feel uncomfortable. Therefore, by analyzing the operation log transmitted from the in-vehicle device to the server device, it is possible to identify the operation of the in-vehicle device that made the occupant feel uncomfortable.

[0015] According to the present disclosure, when a vehicle occupant feels uncomfortable operating an in-vehicle device, a log of the operation can be collected in real time in a server device, and collection of logs unrelated to the operation of the in-vehicle device that the occupant felt uncomfortable can be avoided.

[0016] Specific embodiments of the present disclosure will be described below with reference to the drawings. The dimensions, materials, shapes, and relative positions of the components described in the present embodiments are not intended to limit the technical scope of the present disclosure unless otherwise specified.

[0017] First Embodiment (System Configuration) FIG. 1 is a diagram showing a schematic configuration of an information management system according to this embodiment. The information management system 1 is a system that collects information related to the operational feel of an in-vehicle device. The in-vehicle device 100 is mounted on a vehicle 10. The in-vehicle device 100 is a device that provides multimedia information to an occupant (driver or passenger) of the vehicle 10.

[0018] In the information management system 1, the in-vehicle device 100 is connected to the management server 300 via a network N1. Here, the network N1 may be, for example, a WAN (Wide Area Network), which is a global public communication network such as the Internet, or a telephone communication network such as a mobile phone.

[0019] The in-vehicle device 100 includes a computer. The computer that constitutes the in-vehicle device 100 has a processor 101, a main memory unit 102, an auxiliary memory unit 103, and a communication interface (communication I / F) 104.

[0020] Here, the processor 101 may be, for example, a CPU (Central Processing Unit) or a D The main storage unit 102 is, for example, a RAM (Random Access Memory). The auxiliary storage unit 103 is, for example, a ROM (Read Only Memory), a HDD (Hard Disk Drive), or a flash memory. may include a removable medium (portable recording medium), where the removable medium is, for example, a USB memory, an SD card, or a disc recording medium such as a CD-ROM, a DVD disc, or a Blu-ray disc.

[0021] The auxiliary storage unit 103 stores an operating system (OS), various programs, various information tables, etc. The processor 101 loads the programs stored in the auxiliary storage unit 103 into the main storage unit 102 and executes them, thereby realizing various processes as described below.

[0022] The communication I / F 104 is an interface for performing wireless communication with the management server 300 via the network N1 according to a predetermined wireless communication standard. Examples of the predetermined wireless communication standard include 3G (3rd Generation), LTE (Long Term Evolution), and 5G (5th Generation). The communication I / F 104 is an interface for performing wireless communication with the management server 300 via the network N1 according to a predetermined wireless communication standard. It is composed of a road.

[0023] The in-vehicle device 100 does not necessarily have to be realized by a single physical configuration, but may be realized by multiple computers working together. Also, some or all of the functions of the in-vehicle device 100 may be realized by hardware circuits such as ASICs and FPGAs.

[0024] The in-vehicle device 100 is operated by an occupant of the vehicle 10. FIG. 2 is a diagram showing an example of the configuration of an operation unit in the in-vehicle device 100 that is operated by the occupant. The in-vehicle device 100 is provided with a touch panel display 20 as an operation unit. The touch panel display 20 displays information to be provided to the occupant, as well as one or more buttons 20a and levers 20b that are touch-operated by the occupant. Furthermore, the in-vehicle device 100 is provided with one or more hardware buttons 21 as an operation unit in addition to the touch panel display 20. When the operation unit is operated by the occupant, the in-vehicle device 100 stores an operation log as log information.

[0025] A microphone 200 is also provided in the cabin of the vehicle 10. The microphone 200 collects sounds emitted by the occupants of the vehicle 10. When the occupant operates an operation unit of the in-vehicle device 100, if the occupant feels uncomfortable with the operation, the occupant may emit a first sound indicating the discomfort. At this time, the first voice uttered by the passenger is also picked up by the microphone 200 in the same manner as other voices.

[0026] The information management system 1 then uses the voice picked up by the microphone 200 to collect information related to the operational feel of the in-vehicle device 100. Fig. 3 is a diagram for explaining the flow of information collection by the information management system 1. When the voice uttered by an occupant in the vehicle 10 is picked up by the microphone 200, the voice information is transmitted from the microphone 200 to the in-vehicle device 100.

[0027] When the in-vehicle device 100 receives voice information from the microphone 200 while the in-vehicle device 100 is being operated by an occupant, it is determined whether the voice information includes a first voice indicating the occupant's discomfort. If the voice information includes the first voice, the in-vehicle device 100 transmits an operation log for a predetermined period including the time when the voice information was received to the management server 300 via the network N1. As a result, when an occupant of the vehicle 10 utters the first voice while operating the in-vehicle device 100, an operation log related to the operation at that time is collected in the management server 300.

[0028] (Functional configuration) Next, the functional configuration of the in-vehicle device 100 will be described with reference to Fig. 4. Fig. 4 is a block diagram schematically showing an example of the functional configuration of the in-vehicle device 100. In this embodiment, the in-vehicle device 100 corresponds to the in-vehicle device according to the present disclosure.

[0029] The in-vehicle device 100 has, as functional units, a communication unit 110, a control unit 120, and a log information database (log information DB) 130. The communication unit 110 has a function of connecting the in-vehicle device 100 to a network N1. The communication unit 110 can be realized by a communication I / F 104.

[0030] The control unit 120 has a function of performing arithmetic processing for controlling the in-vehicle device 100. The control unit 120 can be realized by the processor 101. The control unit 120 has a log generation unit 1201, a voice information acquisition unit 1202, a discrimination unit 1203, and an extraction unit 1204 as functional units.

[0031] The log generating unit 1201 generates an operation log when the in-vehicle device 100 is operated by an occupant of the vehicle 10. The operation log generated by the log generating unit 1201 is stored as log information in the log information DB 130. FIG. 5 is a diagram showing log information (operation log) stored in the log information DB 130. As shown in FIG. 5, the operation log stored in the log information DB 130 includes the operation date and time and the operation content. Every time the in-vehicle device 100 is operated by an occupant of the vehicle 10, log information is added to the log information DB 130. The log information DB 130 is constructed in the auxiliary storage unit 103 when the processor 101 executes a program of a database management system.

[0032] The voice information acquisition unit 1202 acquires voice information transmitted from the microphone 200. Every time a voice uttered by an occupant of the vehicle 10 is picked up by the microphone 200, the voice information acquisition unit 1202 acquires the voice information. Then, when the voice information acquisition unit 1202 acquires voice information while the occupant is operating the in-vehicle device 100, the determination unit 1203 determines whether the acquired voice information includes a first voice.

[0033] Here, there are various patterns of voices that indicate discomfort of the occupant. The in-vehicle device 100 may store in advance in the auxiliary storage unit 103 a plurality of patterns of voice texts corresponding to the first voice. Then, the determination unit 1203 adds the pre-stored voice texts corresponding to the first voice to the voice text in the voice information acquired by the voice information acquisition unit 1202. It may be determined whether any of the following items is included.

[0034] Furthermore, the determination of whether the voice information includes the first voice may be performed based on the voice tone. In this case, the in-car device 100 may store a plurality of patterns of voice tones corresponding to the first voice in the auxiliary storage unit 103 in advance. Then, the determination unit 1203 may determine whether the voice in the voice information acquired by the voice information acquisition unit 1202 includes any of the pre-stored plurality of patterns of voice tones corresponding to the first voice.

[0035] Furthermore, the discrimination unit 1203 may determine whether or not the first voice is included in the voice information by using a trained machine learning model generated by machine learning. In this case, the machine learning model has one or more calculation parameters for executing calculations of the inference process. The values ​​of the calculation parameters of the machine learning model may be adjusted as appropriate by machine learning using training data so that the inference process of determining whether or not the first voice is included in the given voice information can be performed. Note that the type of machine learning model may be selected as appropriate depending on the embodiment. The machine learning model may be, for example, a neural network.

[0036] When the determination unit 1203 determines that the voice information acquired by the voice information acquisition unit 1202 includes the first voice, the extraction unit 1204 extracts operation logs for a predetermined period including the time point at which the voice information was acquired from the log information DB 130. Here, the predetermined period is determined in advance as a period during which it is considered that there is a possibility that the operation of the in-vehicle device 100 indicated by the operation log and the voice of the occupant indicated by the voice information are related to each other.

[0037] Then, when the extraction unit 1204 extracts the operation log, the control unit 120 executes a process of transmitting the extracted operation log to the management server 300 via the communication unit 110.

[0038] The management server 300 identifies the operation of the in-vehicle device 100 that made the occupant feel uncomfortable by analyzing the operation log received from the in-vehicle device 100. If the cause of the occupant of the vehicle 10 emitting the first voice is an operation of the in-vehicle device 100, it is highly likely that the operation log for the predetermined period includes a log of the operation of the in-vehicle device 100 that made the occupant feel uncomfortable. Therefore, by analyzing the operation log received from the in-vehicle device 100, it is possible to identify the operation of the in-vehicle device 100 that made the occupant feel uncomfortable.

[0039] (Information Processing) Next, the flow of information processing executed in the in-vehicle device 100 for collecting log information will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of information processing executed in the in-vehicle device 100. This flow is executed by the control unit 120 of the in-vehicle device 100.

[0040] In this flow, first, in S101, audio information transmitted from the microphone 200 is acquired. Next, in S102, it is determined whether the in-vehicle device 100 is being operated by an occupant. If a negative determination is made in S102, that is, if the in-vehicle device 100 is not being operated, the execution of this flow is terminated.

[0041] On the other hand, if a positive determination is made in S102, the process proceeds to S103. In S103, it is determined whether the first voice is included in the voice information acquired in S101. If a negative determination is made in S103, that is, if the first voice is not included in the voice information, the execution of this flow is terminated. In this case, the operation log is not sent to the management server 300.

[0042] On the other hand, if the determination in S103 is affirmative, the process proceeds to S104. In S104, operation logs for a predetermined period including the time when the voice information was acquired in S101 are extracted from the log information DB 130. Next, in S105, the operation logs extracted in S104 are transmitted to the management server 300. Thereafter, the execution of this flow is terminated.

[0043] According to the information management system 1 of this embodiment, when an occupant of the vehicle 10 finds an operation of the in-vehicle device 100 unpleasant, a log of the operation can be collected in real time in the management server 300. In addition, it is possible to avoid collecting logs unrelated to the operation of the in-vehicle device 100 that the occupant found unpleasant.

[0044] In the above embodiment, the in-vehicle device 100 is operated using the touch panel display 20 or the hardware buttons 21 provided as an operation unit, but the in-vehicle device 100 may also be operated by the voice of the occupant. Even in this case, when the in-vehicle device 100 is operated by the voice uttered by the occupant, an operation log thereof is stored in the log information DB 130.

[0045] Second Embodiment The schematic configuration of the information management system according to this embodiment is the same as the schematic configuration of the information management system according to the first embodiment. Also, the functional configuration of the in-vehicle device according to this embodiment is the same as the functional configuration of the in-vehicle device according to the first embodiment. However, in this embodiment, in addition to the operation log, logs other than the operation log are also stored as log information in the log information DB 130 in the in-vehicle device 100.

[0046] Specifically, in the in-vehicle device 100, a log of processing for providing multimedia information to the occupants (hereinafter, also referred to as a "processing log") is generated by the log generation unit 1201. Then, the generated processing log is stored as log information in the log information DB 130. Furthermore, in the in-vehicle device 100, a log of communication between the in-vehicle device 100 and other devices (hereinafter, also referred to as a "communication log") is generated by the log generation unit 1201. Here, the other devices communicating with the in-vehicle device 100 may be, for example, a terminal carried by the occupant, a device (such as an ECU) mounted on the vehicle 10, or a communication device located outside the vehicle 10. Then, the generated communication log is stored as log information in the log information DB 130. The generation of the processing log and the communication log in the in-vehicle device 100 is executed at a predetermined resolution.

[0047] Even if an occupant emits the first voice indicating discomfort while operating the in-vehicle device 100, the cause of the discomfort may be something other than the operation of the in-vehicle device 100. For example, the occupant may emit the first voice when a process that should have been executed in the in-vehicle device 100 to provide the occupant with multimedia information is not executed due to a malfunction of the in-vehicle device 100. Furthermore, the occupant may emit the first voice when communication with another device that should have been executed in the in-vehicle device 100 is not executed due to a malfunction or communication failure of the in-vehicle device 100.

[0048] Therefore, in the in-vehicle device 100 according to the present embodiment, when a first voice is included in the voice information acquired while an occupant is operating the in-vehicle device 100, in addition to the operation log for a predetermined period, a processing log and a communication log for a predetermined period are extracted from the log information DB 130. Then, the extracted operation log, processing log, and communication log are transmitted to the management server 300.

[0049] As a result, if an occupant of the vehicle 10 feels uncomfortable with the operation, processing, or communication of the in-vehicle device 100 while operating the in-vehicle device 100, a log indicating the cause of the discomfort can be collected in real time in the management server 300. The management server 300 then analyzes the operation log, processing log, and communication log received from the in-vehicle device 100 to determine the cause of the discomfort. Identify the factors that make you feel that way.

[0050] However, there are cases where the cause of the occupant's discomfort cannot be identified even if the management server 300 analyzes various logs. For example, even if the cause of the occupant's discomfort is a malfunction in the processing for providing multimedia information to the occupant or a malfunction in communication with other devices, if a log indicating the malfunction is not stored in the log information DB 130, the cause of the occupant's discomfort cannot be identified even if the management server 300 analyzes the processing log and the communication log.

[0051] Therefore, if the management server 300 cannot identify the cause of the occupant's discomfort even after analyzing various logs, the management server 300 sends a command to the in-vehicle device 100 to temporarily increase the resolution at which the log generation unit 1201 generates the processing logs and communication logs above a predetermined resolution. Increasing the resolution at which the log generation unit 1201 generates the processing logs and communication logs in the in-vehicle device 100 may generate a processing log or communication log indicating a malfunction that could not be detected at the predetermined resolution and store the log information DB 130. Then, if the processing log or communication log indicating the malfunction is stored in the log information DB 130, the management server 300 can analyze the logs to identify the cause of the occupant's discomfort.

[0052] (Information Processing) Here, the flow of information processing executed by the management server 300 when analyzing logs will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of information processing executed by the management server 300.

[0053] In this flow, first, in S201, an analysis of various logs received from the in-vehicle device 100 is executed. Next, in S202, it is determined whether or not the cause of the occupant's discomfort has been identified by the analysis of various logs in S201. If the determination in S202 is affirmative, that is, if the cause of the occupant's discomfort has been identified, execution of this flow is terminated.

[0054] On the other hand, if the determination in S202 is negative, the process proceeds to S203. In S203, a command to temporarily increase the resolution for generating the processing log and the communication log above the predetermined resolution is transmitted to the in-vehicle device 100. Thereafter, the execution of this flow is terminated. When the in-vehicle device 100 receives the command transmitted from the management server 300, the resolution for generating the processing log and the communication log is temporarily increased above the predetermined resolution. Then, the processing log and the communication log generated at the resolution temporarily increased above the predetermined resolution are stored in the log information DB 130.

[0055] In the above embodiment, in the in-vehicle device 100, both the processing log and the communication log are generated as logs other than the operation log, and are stored as log information in the log information DB 130. Then, both the processing log and the communication log are transmitted from the in-vehicle device 100 to the management server 300 together with the operation log. However, the logs other than the operation log that are generated in the in-vehicle device 100 and transmitted to the management server 300 may be only either the processing log or the communication log.

[0056] <Other embodiments> The above-described embodiment is merely an example, and the present disclosure may be modified as appropriate within the scope of the present disclosure. Furthermore, the processes and means described in the present disclosure may be freely combined and implemented as long as no technical contradiction occurs.

[0057] In addition, the processing described as being performed by one device may be shared and executed by multiple devices. Alternatively, processes that have been described as being performed by different devices may be executed by a single device. In a computer system, the hardware configuration (server configuration) by which each function is realized can be flexibly changed.

[0058] The present disclosure can also be realized by providing a computer program implementing the functions described in the above embodiments to a computer, and having one or more processors in the computer read and execute the program. Such a computer program may be provided to the computer via a non-transitory computer-readable storage medium connectable to the computer's system bus or via a network. Non-transitory computer-readable storage media include any type of medium suitable for storing electronic instructions, such as any type of disk, including magnetic disks (e.g., floppy disks, hard disk drives (HDDs), etc.), optical disks (e.g., CD-ROMs, DVDs, Blu-ray disks), read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards, flash memory, or optical cards. [Explanation of symbols]

[0059] 1. Information Management System 10. Vehicle 100...In-vehicle equipment 110··Communications Department 120 Control unit 130 Log information database 200··Microphone 300 Management Server

Claims

1. An in-vehicle device for providing multimedia information to a vehicle occupant, comprising: a control unit that, when a first voice indicating discomfort of the occupant is included in a voice uttered by the occupant while the occupant is operating the in-vehicle device, transmits to a server device a log of communications between the in-vehicle device and other devices during a predetermined period including a time point when the voice was acquired, together with an operation log of the in-vehicle device during the predetermined period. In-vehicle device.

2. The control unit determining whether the speech includes the first speech based on the text of the speech; The in-vehicle device according to claim 1 .

3. The control unit determining whether the first voice is included in the voice based on the tone of the voice; The in-vehicle device according to claim 1 .

4. The control unit transmitting to the server device a log of the process for providing multimedia information executed in the in-vehicle device during the predetermined period, together with an operation log of the in-vehicle device during the predetermined period and a log of the communication between the in-vehicle device and the other device during the predetermined period; The in-vehicle device according to claim 1 .

5. A system including an in-vehicle device that provides multimedia information to a vehicle occupant, and a server device that communicates with the in-vehicle device via a network, When the voice uttered by the occupant while the occupant is operating the in-vehicle device includes a first voice indicating the occupant's discomfort, the in-vehicle device stores a log of operations of the in-vehicle device during a predetermined period including the time when the voice was acquired, as well as a log of processes for providing multimedia information executed in the in-vehicle device during the predetermined period, or transmitting a log of communications between the in-vehicle device and other devices during a fixed period to the server device; The server device transmitting, to the in-vehicle device, a command to increase the resolution of generation of the processing log or the communication log based on the analysis result of the log received from the in-vehicle device; system.

6. The in-vehicle device determining whether the speech includes the first speech based on the text of the speech; The system of claim 5.

7. The in-vehicle device determining whether the first voice is included in the voice based on the tone of the voice; The system of claim 5.

8. The server device If a log related to the discomfort of the occupant cannot be identified by analyzing the log received from the in-vehicle device, a command is sent to the in-vehicle device to increase the resolution of generation of the processing log or the communication log. A system according to any one of claims 5 to 7.

Images