Information processing apparatus, mobile body, computer-readable storage medium, and information processing method

Abstract

This invention relates to an information processing apparatus, a mobile body, a computer-readable storage medium, and an information processing method. The information processing apparatus, which performs voice recognition processing via a communication network using a first voice recognition function provided by an external voice recognition device, comprises: a trigger detection unit that detects trigger information indicating that a user wishes to begin voice recognition processing; a communication state prediction unit that predicts future communication states; and a state notification unit that, when the trigger detection unit detects the trigger information, notifies the user of the state of the voice recognition processing based on the future communication state predicted by the communication state prediction unit.

Description

Technical Field

[0001] This invention relates to information processing apparatus, mobile body, computer-readable storage medium, and information processing method. Background Technology

[0002] Patent Document 1 discloses a method for estimating the timing of communication resumption when communication becomes unavailable during a conversation, and for changing the information displayed to the user based on the waiting time until resumption. Patent Document 2 discloses a method for obtaining recognition results corresponding to the input speech from past recognition results accumulated by a speech recognition server, even when communication is unavailable. Patent Document 3 discloses a method for utilizing a recognition engine on the terminal's speech when speech recognition results cannot be received from a server.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2014-174485

[0004] Patent Document 2: Japanese Patent Application Publication No. 2014-115936

[0005] Patent Document 3: Japanese Patent Application Publication No. 2006-003686 Summary of the Invention

[0006] In a first aspect, the present invention provides an information processing apparatus. The information processing apparatus performs speech recognition processing, for example, via a communication network using a first speech recognition function provided by an external speech recognition device. The information processing apparatus includes, for example, a trigger detection unit for detecting trigger information indicating that a user wishes to begin speech recognition processing. The information processing apparatus also includes, for example, a communication state prediction unit for predicting future communication states. Furthermore, the information processing apparatus includes, for example, a status notification unit that, when the trigger detection unit detects the trigger information, notifies the user of the status of the speech recognition processing based on the future communication state predicted by the communication state prediction unit.

[0007] In the information processing apparatus, the communication status can be bandwidth or throughput. In the information processing apparatus, (a) when the communication status prediction unit can predict the future communication status, the status notification unit can (i) determine, based on the future communication status predicted by the communication status prediction unit, a period during which speech recognition processing using the first speech recognition function can be performed, and (ii) notify the user of the period during which speech recognition processing using the first speech recognition function can be performed as the status of speech recognition processing. In the information processing apparatus, (b) when the communication status prediction unit cannot predict the future communication status, the status notification unit can notify the user of (i) the situation that the communication status prediction unit cannot predict the future communication status, and / or (ii) the current communication status.

[0008] The information processing device may include a communication control unit that controls data communication between the information processing device and an external device via a communication network. The information processing device may include a communication discrimination unit that determines the type of data communication. The information processing device may include a priority setting unit that sets communication priorities for multiple data communications based on the type determined by the communication discrimination unit. In the information processing device, the communication control unit may determine the priority of data communication with an external speech recognition device based on future communication states predicted by a communication state prediction unit and the communication priorities set by the priority setting unit. Furthermore, the communication control unit may delay data communication with a lower priority than that with the external speech recognition device, or limit the bandwidth of such lower-priority data communication.

[0009] The information processing apparatus may include a first speech recognition unit that performs speech recognition processing using a first speech recognition function provided by an external speech recognition device. The information processing apparatus may also include a second speech recognition unit that performs speech recognition processing without using the first speech recognition function provided by the external speech recognition device. The information processing apparatus may include a recognition device determination unit that determines whether to use the first or second speech recognition unit to perform speech recognition processing based on a future communication state predicted by a communication state prediction unit. In the information processing apparatus, (i) if the communication state prediction unit predicts that the future communication state is worse than a predetermined first state, or (ii) if the current communication state is worse than a predetermined second state, the recognition device determination unit may decide to use the second speech recognition unit to perform speech recognition processing.

[0010] In the information processing apparatus, (i) if the communication state prediction unit predicts that the future communication state will worsen than a predetermined first state, or (ii) if the current communication state worsens than a predetermined second state, the status notification unit may notify the user that speech recognition processing is being performed using the second speech recognition unit as the status of speech recognition processing. In the information processing apparatus, (i) if the communication state prediction unit predicts that the future communication state will worsen than a predetermined first state, or (ii) if the current communication state worsens than a predetermined second state, the status notification unit may notify the user that at least a portion of one or more instructions available using the second speech recognition unit is available as the status of speech recognition processing. In the information processing apparatus, (i) if the communication state prediction unit predicts that the future communication state will worsen than a predetermined first state, or (ii) if the current communication state worsens than a predetermined second state, when the user inputs an instruction that is unavailable using the second speech recognition unit, the status notification unit may notify the user that the user's instruction input was not accepted and the period during which speech recognition processing using the first speech recognition function is possible as the status of speech recognition processing.

[0011] In a second aspect of the invention, a mobile body is provided. The mobile body, for example, includes the information processing apparatus described in the first aspect.

[0012] In a third aspect of the invention, an information processing method is provided. This information processing method, for example, relates to a method of performing speech recognition processing via a communication network using a first speech recognition function provided by an external speech recognition device. The information processing method, for example, includes a trigger detection step for detecting trigger information indicating that a user wishes to begin speech recognition processing. The information processing method, for example, includes a communication state prediction step for predicting future communication states. The information processing method, for example, includes a state notification step for notifying the user of the state of speech recognition processing based on the future communication state predicted in the communication state prediction step when trigger information is detected in the trigger detection step.

[0013] In a fourth aspect of the invention, a program is provided. The program is, for example, a program for causing a computer to function as an information processing apparatus according to the first aspect. The program is, for example, a program for causing a computer to execute the information processing method according to the third aspect. A computer-readable medium for storing the program may also be provided. The computer-readable medium may also be a non-volatile computer-readable medium. The computer-readable medium may also be a computer-readable recording medium.

[0014] Furthermore, the summary of the invention does not list all the essential features of the invention. Additionally, sub-combinations of these features can also constitute an invention. Attached Figure Description

[0015] Figure 1 The usage mode of vehicle 50 according to one embodiment is illustrated schematically.

[0016] Figure 2 The functional configuration of vehicle 50 is shown schematically.

[0017] Figure 3 It is a table that shows the priority of data communication.

[0018] Figure 4 An example of a data structure for data communication information stored by the information processing device 200 is shown.

[0019] Figure 5 Conceptually, the communication control unit 230 performs communication throughput control based on priority.

[0020] Figure 6 The data transmission between functional blocks of the information processing device 200 is illustrated schematically.

[0021] Figure 7 A flowchart is shown involving the information processing method performed by the information processing device 200.

[0022] Figure 8 A flowchart is shown illustrating the communication throughput control method executed by the information processing device 200.

[0023] Figure 9 An implementation example of the control system 1000 within the vehicle 50 is shown.

[0024] Figure 10 An example of the internal structure of information system device 1041 is shown schematically.

[0025] Figure 11 An example of the internal structure of the voice recognition control unit 276 is shown schematically.

[0026] Figure 12 An example of the internal structure of the speech recognition system 1074 is shown schematically.

[0027] Figure 13 An example of information processing for vehicle 50 is illustrated schematically.

[0028] Figure 14 An example of information processing for vehicle 50 is illustrated schematically.

[0029] Figure 15 An example of a computer 3000 is shown. Detailed Implementation

[0030] The present invention will now be described through embodiments thereof, but these embodiments do not limit the scope of the claimed invention. Furthermore, not all combinations of features described in the embodiments are essential to the solutions provided by the present invention.

[0031] [Summary of Vehicle 50]

[0032] Figure 1 The illustration schematically depicts a usage mode of vehicle 50 according to one embodiment. Vehicle 50 is, for example, an automobile. Vehicle 50 can be an example of a transportation device used for transporting people or goods. Examples of vehicles 50 include automobiles, autonomous two-wheelers, standing vehicles with power units, electric trains, and construction machinery. Examples of automobiles include automobiles with internal combustion engines, electric vehicles, fuel cell vehicles (FCVs), hybrid vehicles, small commutator vehicles, electric trolleys, and buses. Examples of autonomous two-wheelers include motorcycles, tricycles, and electric bicycles. Examples of electric trains include various vehicles used in rail transit systems such as railways, trams, LRTs (Light Rail Transit), and monorails. Examples of construction machinery include heavy machinery and agricultural machinery. Examples of agricultural machinery include tillers, lawnmowers, and weeders. Furthermore, the use of vehicle 50 is not particularly limited. Vehicle 50 can be used for private purposes, commercial purposes, or as a public transportation vehicle.

[0033] Vehicle 50 is equipped with an information processing device 200. The information processing device 200 performs data communication with external devices 30a and 30b. In this embodiment, the information processing device 200 sends and receives information with external devices 30a, 30b, and 30c via a communication network 90 and a wireless communication system 92. For example, the information processing device 200 exchanges data 32 with external device 30a. For example, the information processing device 200 exchanges data 34 with external device 30b. For example, the information processing device 200 exchanges data 36 with external device 30c. Furthermore, in this embodiment, external devices 30a, 30b, and 30c are sometimes collectively referred to as "external device 30".

[0034] External device 30a includes, for example, a server for providing services to the occupants of vehicle 50. For example, external device 30a includes a server for storing content data such as animated images, a server for providing SNS (Social Networking Service), etc. Information processing device 200 receives animated image data from external device 30a according to instructions from the occupants of vehicle 50. Furthermore, information processing device 200 receives text messages, voice messages, image messages, animated image messages, etc., as SNS messages from external device 30a according to instructions from the occupants of vehicle 50. Furthermore, information processing device 200 sends text messages, voice messages, image messages, animated image messages, etc., as SNS messages from the occupants of vehicle 50 to external device 30a according to instructions from the occupants of vehicle 50.

[0035] In another embodiment, the external device 30a can also provide online dashcam services. For example, the external device 30a receives image data captured by a camera (not shown) configured in the vehicle 50 and stores it in any storage device (not shown). The images can be still images or moving images. The external device 30a can also receive voice data collected by a voice input device (not shown) configured in the vehicle 50 and store it in any storage device (not shown). Furthermore, the external device 30a can also retrieve image or voice data stored in the storage device according to instructions from the occupants of the vehicle 50 and send the retrieved data to the vehicle 50.

[0036] External device 30b is, for example, a server for providing services related to the control system of vehicle 50. External device 30b may include, for example, a server that collects information related to the control system of vehicle 50. Examples of control-related information collected by external device 30b include LIDAR (Light Detection and Ranging) data used for autonomous driving of vehicle 50. External device 30b may include, for example, a server that provides control system-related information to vehicle 50. As control system-related information provided by external device 30b, it may include map data used for autonomous driving of vehicle 50. Information processing unit 200 sends LIDAR data acquired for autonomous driving control to external device 30b. Information processing unit 200 receives map data from external device 30b, regardless of instructions from the occupants of vehicle 50.

[0037] External device 30c provides voice recognition functionality via communication network 90. ​​For example, information processing unit 200 sends user voice data to external device 30c. External device 30c analyzes the user's voice data and infers the content of the user's instruction or command. External device 30c sends information representing the content of the instruction or command to information processing unit 200. Thus, voice recognition functionality is provided via communication network 90. ​​Examples of data 36 include user voice data sent from vehicle 50 to external device 30c, and data representing the voice recognition results of external device 30c.

[0038] In this embodiment, the communication network 90 includes IP networks such as the Internet, P2P networks, dedicated lines including VPNs, and virtual networks. In this embodiment, the wireless communication system 92 is a mobile communication network connected to the communication network 90. ​​For example, the wireless communication system 92 includes a wireless access network and a core network.

[0039] In this embodiment, the information processing device 200 controls the communication throughput with the external device 30 based on the priority of data communication. For example, data communication such as SNS data has a lower priority than data communication related to the control system of the vehicle 50. For example, data communication for data used by the information processing device 200 to perform voice recognition processing using the external device 30c has a lower priority than data communication related to the control system of the vehicle 50.

[0040] In this embodiment, the information processing device 200 predicts future communication states based on time-series data of past communication states. For example, the information processing device 200 predicts future communication throughput based on time-series data of past communication throughput. When the predicted future communication throughput is low, the information processing device 200 delays data communication with external devices 30a that has a lower priority than vehicle 50 control-related information to a level that can maintain a minimum quality of service. This maintains the communication throughput of data communication related to vehicle 50 control information.

[0041] Furthermore, when it is predicted that the communication throughput will become lower and that the minimum quality of service cannot be maintained with the external device 30a, the information processing unit 200 stops data communication with the external device 30a and maintains the communication throughput of data communication related to the control system of the vehicle 50. In this way, by limiting the communication of low-priority data, the information processing unit 200 can increase the possibility of continuing high-priority data communication.

[0042] As described above, according to this embodiment, the information processing device 200 prioritizes data communication of data 34, including information related to the control system of vehicle 50, over data communication of data 32, including (i) content data such as moving images, music, voice, and games, and (ii) SNS data such as SNS messages. Therefore, when passengers (sometimes referred to as users) of transportation equipment or mobile bodies such as vehicle 50 utilize services provided through the transmission and reception of data 32, changes in the communication status may lead to data loss or communication delays. As a result, for example, there may be a significant reduction in the quality of service provided by external device 30a, or the possibility of repeated interruptions and resumptions of service provision.

[0043] Furthermore, as described above, according to this embodiment, the information processing device 200 prioritizes data communication of data 34, including information related to the control system of the vehicle 50, over data communication of data 36. Therefore, there is a possibility that the information processing device 200 cannot perform voice recognition processing using the external device 30c due to changes in the communication state. Furthermore, when the information processing device 200 performs voice recognition processing using the external device 30c, there is a possibility that the time for the information processing device 200 to respond to the user increases due to changes in the communication state. Therefore, it is desirable to develop a technology that can suppress a significant decrease in availability even when the communication state changes during vehicle 50 movement.

[0044] Therefore, in this embodiment, when the information processing device 200 performs voice recognition processing using the voice recognition function provided by the external device 30c via a communication network, it notifies the user of the status of the voice recognition processing. Specifically, the information processing device 200 first detects trigger information indicating that the user wishes to start the voice recognition processing. Examples of trigger information include saying the keyword "WakeUp," pressing the start button for voice recognition processing, or selecting an icon containing the start command for voice recognition processing.

[0045] Next, the information processing device 200 predicts the future communication status. Furthermore, based on the predicted future communication status, the information processing device 200 determines the status of speech recognition processing for a specific future period. The information processing device 200 notifies the user of the determined status of speech recognition processing. Examples of speech recognition processing status include (i) a status where speech recognition processing can be performed using the speech recognition function provided by the external device 30c, (ii) a status where speech recognition processing cannot be performed using the speech recognition function provided by the external device 30c, and (iii) a status where the execution time of speech recognition processing increases when the speech recognition function provided by the external device 30c is used. In addition, notifying the user of the status of speech recognition processing may include notifying the user of the actual status of speech recognition processing, or it may include deciding to notify the user of the status of speech recognition processing.

[0046] According to this embodiment, the user can anticipate changes in the speech recognition processing status due to changes in the communication status. Therefore, depending on the status of the speech recognition processing, the user can (i) input instructions or commands without using the speech recognition processing of the information processing device 200, (ii) change the content of their speech so that they can input instructions or commands using the speech recognition processing of the information processing device 200 even without using the speech recognition function provided by the external device 30c, or (iii) delay the input period of instructions or commands. As a result, even if the communication status changes, a significant decrease in availability can be suppressed.

[0047] Furthermore, for example, if the current communication status is poor, the information processing device 200 may not be able to predict the future communication status. Examples of the current communication status include current throughput, latency, and packet loss. For example, through... Figure 2 The throughput measurement unit 210 described herein measures the current communication status.

[0048] Therefore, in the above embodiment, when the information processing device 200 is able to predict the future communication state, the information processing device 200 can (i) determine the state of speech recognition processing in a specific future period based on the predicted future communication state, and (ii) notify the user of the determined state of speech recognition processing. On the other hand, when the information processing device 200 cannot predict the future communication state, the information processing device 200 can notify the user that the future communication state and / or the current communication state cannot be predicted. Thus, information is provided to the user regardless of whether the information processing device 200 is able to predict the future communication state.

[0049] [Specific configuration of each unit of the information processing device 200]

[0050] The units of the information processing device 200 can be implemented in hardware, in software, or in both hardware and software. At least a portion of the units of the information processing device 200 can be implemented by a single server or by multiple servers. At least a portion of the units of the information processing device 200 can also be implemented on a virtual machine or cloud system. At least a portion of the units of the information processing device 200 can also be implemented by a personal computer or a mobile terminal. Examples of mobile terminals include mobile phones, smartphones, PDAs, tablets, laptops or notebook computers, wearable computers, etc. The units of the information processing device 200 can also utilize distributed ledger technology such as blockchain or distributed networks to store information. The information processing device 200 will be described in detail later.

[0051] External device 30c may be an example of an external voice recognition device. The voice recognition function provided by external device 30c may be an example of a first voice recognition function. Vehicle 50 may be an example of a moving body.

[0052] [Another example of an implementation]

[0053] In this embodiment, the information processing device 200 will be described in detail using the example of an information processing device 200 installed on a vehicle 50, which is an example of a transport device or mobile body. However, transport devices or mobile bodies equipped with the information processing device 200 are not limited to this embodiment. Other examples of transport devices or mobile bodies include aircraft that move in the air and ships that move on or underwater. Examples of aircraft include airplanes, airships, hot air balloons, balloons, helicopters, and drones. Examples of ships include motorboats, jet skis, submarines, submersibles, and underwater scooters.

[0054] In this embodiment, the details of an example of the information processing apparatus 200 will be described using the case where the information processing apparatus 200 predicts future communication states based on time-series data of past communication states. However, the method by which the information processing apparatus 200 predicts future communication states is not limited to this embodiment.

[0055] In another embodiment, the information processing device 200 predicts future communication states based on (i) information related to communication states acquired by other vehicles traveling ahead, and (ii) big data constructed based on information related to communication states acquired by multiple vehicles. The information related to communication states acquired by other vehicles traveling ahead and the aforementioned big data may also be stored in the external device 30.

[0056] The information processing device 200 can obtain information related to the communication status of a specific location on the driving route of the vehicle 50 on which the information processing device 200 is installed from the external device 30. Furthermore, the information processing device 200 can also obtain information related to the communication status obtained by other vehicles traveling ahead, for example, through vehicle-to-vehicle communication.

[0057] In another embodiment, the external device 30 may also estimate the future communication status related to a specific location on the travel route of the vehicle 50 on which the information processing device 200 is installed. In this case, the information processing device 200 can obtain information from the external device 30 representing the estimated value of the future communication status on the travel route of the vehicle 50.

[0058] More specifically, the information processing device 200 can use various information (sometimes called map information) representing the geographical distribution of communication states to predict future communication states. The information processing device 200 obtains, for example, the various information representing the geographical distribution of communication states from an external device 30.

[0059] The information processing device 200 can also predict future communication conditions in advance using map information indicating the location of each area in one or more areas where the possibility of poor communication conditions is high. The map information can also be stored in a corresponding manner for each area in one or more areas, with information indicating the location of that area and information indicating the communication conditions in that location or area.

[0060] In one embodiment, the information processing device 200 acquires information indicating the self-position of the vehicle 50 from a self-position estimation device (not shown) configured in the vehicle 50 or in the information processing device 200. Based on the vehicle 50's movement history as indicated by the information indicating its self-position, the information processing device 200 predicts the future location or area of ​​the vehicle 50. The information processing device 200 is able to predict future communication status by comparing the prediction results with the map information.

[0061] In another embodiment, the information processing device 200 obtains information indicating the movement route of the vehicle 50 from a path search device (not shown) configured in the vehicle 50 or the information processing device 200. Furthermore, the information processing device 200 obtains information indicating the vehicle 50's own position from a self-position estimation device (not shown) configured in the vehicle 50 or the information processing device 200. Based on the information indicating the vehicle 50's own position and the information indicating the vehicle 50's movement route, the information processing device 200 predicts the future location or area of ​​the vehicle 50. The information processing device 200 can predict future communication status by comparing the prediction results with the map information.

[0062] Figure 2 The functional configuration of vehicle 50 is illustrated schematically. In this embodiment, vehicle 50 includes, for example, an information processing device 200, control devices 24a, 24b, and 24c, equipment 25a, equipment 25b, equipment 25c, and an in-vehicle network 29. In this embodiment, information processing device 200 includes, for example, a communication unit 202, a communication control unit 230, a throughput measurement unit 210, a communication status prediction unit 220, a communication discrimination unit 240, a priority setting unit 250, a quality calculation unit 260, and a voice recognition control unit 276. In this embodiment, equipment 25c includes, for example, a content execution unit 282 and a display unit 284.

[0063] In this embodiment, the information processing device 200, control device 24a, control device 24b, and control device 24c are interconnected via an in-vehicle network 29. The in-vehicle network 29 may include an Ethernet (registered trademark) network. The in-vehicle network 29 may include a CAN (Controller Area Network).

[0064] [Summary of Control Device 24 and Equipment 25]

[0065] Control devices 24a and 24b control equipment 25a and equipment 25b, respectively. Similarly, control device 24c controls equipment 25c. Control devices 24a, 24b, and 24c can each be an ECU (Electronic Control Unit). Equipment 25a, 25b, and 25c include, for example, drive system equipment, information communication system equipment, etc. Examples of drive system equipment include engines and electric motors.

[0066] Furthermore, the term "control device 24" is sometimes used as a general term for control devices 24a, 24b, and 24c. Similarly, the term "device 25" is sometimes used as a general term for devices 25a, 25b, and 25c. For one embodiment of control devices 24 and 25, see [reference needed]. Figure 9 Provide specific examples.

[0067] In combination Figure 2 In the described embodiments, examples of device 25c are given, assuming that device 25c includes an information system device or a video / audio system device. In one embodiment, device 25c prompts the user with the execution result of data 32 received by information processing device 200 from external device 30a. In another embodiment, device 25c prompts the user with the speech recognition result received by information processing device 200 from external device 30c.

[0068] In this embodiment, the content execution unit 282 acquires data 32 from the external device 30a. Furthermore, the content execution unit 282 executes the data 32. As described above, the data 32 is content data published by the external device 30a, and executing the data 32 plays moving images, audio, etc. The content execution unit 282 causes the display unit 284 to display the image (sometimes called a content image) generated by executing the data 32. The content image can be any one of more images included in the moving image content. The content image can be an example of the execution result of the data 32.

[0069] In this embodiment, the display unit 284 displays an image. The image can be a moving image or a still image. Examples of display units 284 include monitors and projectors. The display unit 284 may also include input devices such as touch panels, indicator devices, switches, voice recognition systems, and gesture recognition systems. The display unit 284 may also include voice output devices such as speakers.

[0070] For example, the display unit 284 displays the image output (sometimes referred to as playback) by the content execution unit 282. When the display unit 284 is equipped with a voice output device, the display unit 284 can also output the voice output (sometimes referred to as playback) by the content execution unit 282.

[0071] Furthermore, the display unit 284 performs voice recognition processing. The display unit 284 (i) acquires the user's voice and (ii) determines the content of the instruction or command that the user inputs into the information processing device 200 via voice. Thus, the display unit 284 is able to accept user input.

[0072] In one embodiment, the display unit 284 performs voice recognition processing using the voice recognition function provided by the external device 30c. In another embodiment, the display unit 284 performs voice recognition processing without using the voice recognition function provided by the external device 30c. The number of instructions or commands that can correspond to the case where the display unit 284 performs voice recognition processing without using the voice recognition function provided by the external device 30c may be less than the number of instructions or commands that can correspond to the case where the display unit 284 performs voice recognition processing using the voice recognition function provided by the external device 30c.

[0073] [Specific configuration of each unit of control device 24 and equipment 25]

[0074] Each unit of the control device 24 can be implemented in hardware, software, or both. Each unit of the device 25 can be implemented in hardware, software, or both.

[0075] [Overview of each unit of the information processing device 200]

[0076] The communication unit 202 communicates with the external device 30 via a mobile communication network. For example, the communication unit 202 sends and receives data 32 with the external device 30a according to instructions from the communication control unit 230. The communication unit 202 sends and receives data 34 with the external device 30b according to instructions from the communication control unit 230. Furthermore, the communication unit 202 sends and receives data 36 with the external device 30c according to instructions from the communication control unit 230.

[0077] Throughput measurement unit 210 measures the communication throughput with external device 30. Furthermore, throughput measurement unit 210 can measure the communication throughput in the upload direction from communication unit 202 to external device 30. Throughput measurement unit 210 can also measure the communication throughput in the download direction from external device 30 to communication unit 202.

[0078] The communication status prediction unit 220 uses at least the communication throughput measured by the throughput measurement unit 210 to predict future communication status. For example, the communication status prediction unit 220 predicts future communication throughput. Other communication statuses include throughput, latency, and packet loss.

[0079] The communication control unit 230 controls data communication with the external device 30 based on the future communication state predicted by the communication state prediction unit 220. Specifically, the communication control unit 230 controls data communication with the external device 30 by controlling the communication unit 202. For example, when the future communication state is worse than the predetermined state, the communication control unit 230 limits the bandwidth or communication throughput of data communication with lower priority set by the priority setting unit 250 compared to data communication with higher priority set by the priority setting unit 250.

[0080] Furthermore, if a predetermined condition is met in the future communication state (e.g., future communication throughput), the communication control unit 230 controls data communication with the external device 30a such that the amount of data communication related to data 36 when the vehicle 50's speed is a first speed is less than the amount of data communication related to data 36 when the vehicle 50's speed is a second speed. The second speed may be lower than the first speed. The second speed may also be a speed less than a predetermined threshold. The second speed may also be substantially zero. Examples of predetermined conditions include a future communication state worsening than a predetermined state and a future communication throughput being less than a predetermined value.

[0081] Therefore, when the vehicle speed of 50 is sufficiently low, the bandwidth or throughput allocated to data communication for data 36 becomes relatively large. As a result, the availability associated with speech recognition processing is improved.

[0082] In one implementation, when the vehicle 50 is traveling at a second speed, the amount of data communication in the control system is reduced compared to when the vehicle 50 is traveling at a first speed. As mentioned above, the second speed can be lower than the first speed.

[0083] For example, when the information processing device 200 downloads map data for navigation, the higher the speed of the vehicle 50, the more extensive the map data is required. Furthermore, the higher the speed of the vehicle 50, the more frequently map data is requested.

[0084] Furthermore, while vehicle 50 is moving, various data collection and transmission can be performed for the purpose of collecting large amounts of data. On the other hand, when vehicle 50 is stationary, the collection and transmission of this data can be stopped.

[0085] For example, external device 30b uses data uploaded from one or more vehicles 50 to generate various maps. There are no particular restrictions on the types of maps. In each map, for each of the more than one location, information representing the location of each location and various information at each location are correspondingly stored.

[0086] More specifically, the external device 30b acquires information from each of the more than one vehicle 50 that corresponds to the measurement results (sometimes referred to as LiDAR data) of the LiDAR (light detection and ranging) sensors mounted on each of the more than one vehicle 50, and establishes a correspondence between this information and the location information indicating the location where the LiDAR data was acquired. The external device 30b can also acquire image data from each of the more than one vehicle 50 that corresponds to the image data captured by a camera mounted on each of the more than one vehicle 50, and establish a correspondence between this information and the location information indicating the location where the image was captured. The external device 30b stores this information in any storage device. The external device 30b can also perform cleaning processing on this information and store the cleaned data in any storage device. Thus, for example, the external device 30b can generate a map relating to the shape of a road and / or buildings located near the road in approximately real-time.

[0087] Thus, even when LiDAR data or image data is uploaded from vehicle 50 to external device 30b, the higher the speed of vehicle 50, the more frequently data uploads are required. Conversely, the lower the speed of vehicle 50, the less frequently data is uploaded; if vehicle 50 stops, the collection and transmission of data may cease.

[0088] As a result, when the vehicle 50's speed is the second speed, compared to when the vehicle 50's speed is the first speed, the proportion of time spent transmitting and receiving data 34 between the information processing device 200 and the external device 30b within a predetermined period (sometimes called a unit period) is smaller. On the other hand, the proportion of time spent transmitting and receiving data 36 between the information processing device 200 and the external device 30c within a unit period is larger. Therefore, the amount of data communication associated with data 36 when the vehicle 50's speed is the first speed may be less than the amount of data communication associated with data 36 when the vehicle 50's speed is the second speed.

[0089] In another embodiment, the priority of data communication is set to change dynamically according to the speed of vehicle 50. According to this embodiment, for example, when the speed of vehicle 50 is a second speed, the priority of a portion of the control system data becomes lower than the priority of data 36. As a result, the amount of data communication associated with data 36 when the speed of vehicle 50 is a first speed may be less than the amount of data communication associated with data 36 when the speed of vehicle 50 is a second speed.

[0090] The communication discrimination unit 240 determines the type of data communication with the external device 30. The priority setting unit 250 sets the communication priority for multiple data communications based on the type determined by the communication discrimination unit 240.

[0091] The priority setting unit 250 sets the communication priority of multiple data communications based on the type and quality of the data communication determined by the communication discrimination unit 240. The priority setting unit 250 can set the communication priority of multiple data communications based on the type determined by the communication discrimination unit 240 and the quality of the data communication calculated by the quality calculation unit 260.

[0092] The quality calculation unit 260 calculates the quality of data communication based on the communication throughput predicted by the communication status prediction unit 220. For example, the quality calculation unit 260 can calculate the multimedia quality (MMq) (e.g., the MMq specified in ITU-T Recommendation G.1070). In addition to multimedia quality, the quality calculation unit 260 can also calculate any indicator representing the quality of service as the quality of data communication.

[0093] In this embodiment, the voice recognition control unit 276 controls the voice recognition processing of the information processing device 200. For example, the voice recognition control unit 276 controls the voice recognition processing of the device 25c by outputting a signal to the control device 24c for controlling the voice recognition processing of the information processing device 200.

[0094] Specifically, the voice recognition control unit 276 determines whether to use the external device 30c to perform voice recognition processing based on the future communication state predicted by the communication state prediction unit 220. The voice recognition control unit 276 also determines the future state of the voice recognition processing based on the future communication state predicted by the communication state prediction unit 220. Furthermore, the voice recognition control unit 276 determines whether to notify the user of the future state of the voice recognition processing based on the future communication state predicted by the communication state prediction unit 220. The voice recognition control unit 276 can output signals indicating these decision results to the control device 24c. The voice recognition control unit 276 will be described in detail later.

[0095] Furthermore, as described above, when the communication state prediction unit 220 is able to predict future communication states, the voice recognition control unit 276 can determine the state of voice recognition processing in a specific future period based on the predicted future communication state. Additionally, the voice recognition control unit 276 can decide whether to notify the user of the future state of voice recognition processing. On the other hand, if the communication state prediction unit 220 cannot predict future communication states, the voice recognition control unit 276 can decide to notify the user that the future communication state and / or the current communication state cannot be predicted.

[0096] [Summary of information processing in each unit of the information processing device 200]

[0097] When future communication throughput falls below a predetermined threshold, the communication control unit 230 limits the communication throughput for data communications with lower priority, as set by the priority setting unit 250, compared to data communications with higher priority set by the priority setting unit 250. This increases the likelihood of continuous communication of high-priority data. Furthermore, if future communication throughput temporarily falls below the predetermined threshold, the communication control unit 230 may not limit the communication throughput for low-priority data communications. For example, if the time during which future communication throughput falls below the predetermined threshold is less than a predetermined time, the communication control unit 230 may not limit the communication throughput for low-priority data communications.

[0098] When the future communication throughput falls below a predetermined threshold for an extended period, the communication control unit 230 can limit the communication throughput of low-priority data communications. The predetermined threshold can be a variable value. The communication control unit 230 can determine the threshold based on the data communication status with the external device 30. For example, the communication control unit 230 can determine the threshold based on the communication throughput required to provide a minimum quality of service in data communication with the external device 30. The communication control unit 230 can determine the threshold by multiplying the communication throughput required to provide this minimum quality of service by a predetermined coefficient. The predetermined coefficient can be any value greater than or equal to 1. Alternatively, the predetermined threshold can be a fixed value rather than a variable value.

[0099] Compared to high-priority data communications set by the priority setting unit 250, the communication control unit 230 delays low-priority data communications set by the priority setting unit 250, thereby limiting communication throughput. The communication control unit 230 can delay data communications by buffering data communicated in low-priority data communications. Since high-priority data is not delayed, the possibility of continuous high-priority data communication is increased. Furthermore, the communication control unit 230 can also limit communication throughput by reducing the amount of data in low-priority data communications. The communication control unit 230 can also limit communication throughput by reducing the bit rate of low-priority data communications. When the data transmitted via low-priority data communications is image data, the communication control unit 230 can limit communication throughput by reducing the image quality of the transmitted image.

[0100] The communication state prediction unit 220 can use the delay amount after the communication control unit 230 delays low-priority data communication to predict future communication states (e.g., future communication throughput). According to this embodiment, the communication state prediction unit 220 considers delay information to predict future communication states. This improves the accuracy of future communication state prediction.

[0101] As a type of data communication, the communication discrimination unit 240 determines whether it is data communication related to the control of the vehicle 50. The priority setting unit 250 sets the priority of predetermined data communication related to the control of the vehicle 50 to be higher than the priority of other data communication. Therefore, since data communication related to the control of the vehicle 50 can be continuously performed, the driving safety of the vehicle 50 can be improved. When the communication throughput of data communication related to the control system of the vehicle 50 cannot be guaranteed to continue providing services based on data communication related to the control system of the vehicle 50, the communication control unit 230 can stop other predetermined data communication. Furthermore, as a type of data communication, the communication discrimination unit 240 can determine whether it is data communication related to the control of the vehicle 50 or multimedia data communication. The priority setting unit 250 can set the priority of data communication related to the control of the vehicle 50 to be higher than the priority of multimedia data communication.

[0102] The communication control unit 230 can limit the communication throughput of low-priority data communications to a predetermined value required to continue providing services based on low-priority data communications. Therefore, even for low-priority data communications, the possibility of a complete interruption of data communication can be reduced.

[0103] The communication status prediction unit 220 can be an example of a throughput prediction unit. The voice recognition control unit can be an example of a status notification unit. The display unit 284 can be an example of a status notification unit. The display unit 284 can be an example of a first voice recognition unit and a second voice recognition unit. The display unit 284 can also be an example of a recognition device decision unit. The information processing apparatus 200 and a portion thereof can be an example of an information processing apparatus. At least a portion of the information processing apparatus 200 and the control device 24c can also function as examples of information processing apparatuses. In addition, at least a portion of the information processing apparatus 200, the control device 24c, and at least a portion of the device 25c can also function as examples of information processing apparatuses.

[0104] [Another example of an implementation]

[0105] Combination Figure 2 Details of the functional blocks constituting the vehicle 50 in relation to this embodiment have been described. However, the vehicle 50 is not limited to this embodiment. In another embodiment, the vehicle 50 may have, in addition to Figure 2 Functional blocks other than those shown in the diagram. Furthermore, vehicle 50 may not have these functional blocks. Figure 2 This is part of the functional block shown. Similarly, the information processing device 200 may have, in addition to Figure 2 Functional blocks other than those shown. The information processing device 200 may include at least a portion of the control device 24, and at least a portion of the device 25. Alternatively, the information processing device 200 may not include... Figure 2 Part of the function block shown.

[0106] Figure 3 This is a table representing the priority of data communication. Figure 3 In the table, "Category" indicates whether it is a control system or a non-control system for vehicle 50. "Nature" indicates whether the data communication is stable or unstable. "Service" indicates the content of the service provided through data communication. For example... Figure 3 As shown, data communication related to services of the vehicle 50's control system has a higher priority than data communication related to services of non-control systems. Furthermore, stable data communication has a higher priority than unstable data communication. In addition, besides... Figure 3 In addition to the "Category" and "Nature" shown, you can also set priorities for each service.

[0107] [Another example of an implementation]

[0108] In this embodiment, an example of data communication priority is illustrated, assuming that the priority of data communication is fixed. However, the priority of data communication is not limited to this embodiment.

[0109] In other embodiments, the priority of data communication can also change dynamically. For example, the priority of data communication can change dynamically based on the speed of vehicle 50. For example, information processing device 200 stores multiple data tables representing the priority of data communication and controls data communication based on the priorities represented by the data tables.

[0110] Specifically, the information processing device 200 stores (i) a data table used when the vehicle speed of 50 is less than a predetermined threshold and (ii) a data table used when the vehicle speed of 50 is greater than the predetermined threshold. The information processing device 200 may also store (i) a data table used when the vehicle 50 is stopped, (ii) a data table used when the vehicle 50 is not stopped and the vehicle speed of 50 is less than the predetermined threshold, and (iii) a data table used when the vehicle speed of 50 is greater than the predetermined threshold.

[0111] Furthermore, in this embodiment, an example of data communication priority is illustrated by taking the case where the priority of data communication related to all services classified as non-control systems is lower than the priority of data communication related to all services classified as control systems. However, the priority of data communication is not limited to this embodiment.

[0112] In other implementations, the priority of data communication for each service may be set in such a way that data communication related to services classified as part of the control system has a higher priority than data communication related to services classified as part of the control system. For example, when certain conditions are met, the priority of data communication for each service may be set in such a way that data communication related to services classified as part of the control system has a higher priority than data communication related to services classified as part of the control system.

[0113] Specifically, when the vehicle 50's speed exceeds a predetermined threshold, the priority of data communication is set such that data communication related to services classified as part of the non-control system has a lower priority than data communication related to services classified as part of the control system. On the other hand, when (i) the vehicle 50 is stopped, or (ii) the vehicle 50's speed is less than the predetermined threshold, the priority of data communication is set such that data communication used for downloading, for example, animated image content, voice content, game content, SNS content, Web content, etc., has a higher priority than data communication used for downloading, for example, map data of locations at a specific distance from the current location in navigation map data.

[0114] Figure 4An example of a data structure for data communication information stored in the information processing device 200 is shown. The data communication information establishes a correspondence between IP address, port number, priority, type ID, and minimum quality. The IP address is, for example, the IP address assigned to the control device 24. The port number is the port number used by the transport layer protocol in TCP / IP communication. "Priority" indicates the priority assigned to the data communication identified by the combination of IP address and port number. "Type ID" indicates the type determined based on the combination of IP address and port number. "Minimum Quality" indicates the minimum quality of service required to maintain the provision of services based on data communication. For example, MMq can be used as an indicator of minimum quality.

[0115] In this embodiment, it is assumed that the decision is made based on, for example, the combination of IP address and port number contained in the communication packet sent from the control device 24. Figure 3 Which service's data communication is being conducted among the services shown? The communication control unit 230 determines the priority and type of data communication based on the combination of IP address and port number contained in the communication packet and the data communication information. Furthermore, in the case of data communication involving data transmission, "IP address" refers to the sending source IP address, and "port number" refers to the sending source port number.

[0116] Figure 5 The communication control unit 230 conceptually illustrates communication throughput control based on priority. It is assumed that data communication occurs at time tx, including control system communication, interactive communication, and web browsing. Here, data communication from the control system has a higher priority than data communication from non-control system devices (interactive communication and web browsing). Furthermore, among the non-control system data communications, interactive communication has a higher priority than web browsing data communication.

[0117] The total communication throughput at time tx is Thr1. Assume that the communication state prediction unit 220 predicts that the communication throughput will decrease to Thr2 after Δt from time tx. When it is determined that the sum of the communication throughput required to provide the minimum quality of service in each of the currently executed data communications exceeds Thr2, the communication control unit 230 controls the throughput of the control system's data communication and interactive communication to a value that can guarantee the minimum quality of service for each. Furthermore, the communication control unit 230 temporarily suspends data communication for web browsing. Thus, while maintaining data communication related to the control system of the vehicle 50, the total communication throughput can be kept below the predicted throughput.

[0118] Figure 6The data flow between functional blocks of the information processing apparatus 200 is illustrated schematically. In this embodiment, for ease of explanation, the data flow will be described in detail using the case where the communication state prediction unit 220 predicts the future communication throughput as the future communication state.

[0119] The communication control unit 230 monitors the transmission data sent from the control device 24. The communication control unit 230 determines whether to initiate data communication for a new service based on the combination of IP address and port number. When new communication begins, the communication control unit 230 notifies the communication determination unit 240 of communication information including the IP address and port number. In response to a request from the throughput measurement unit 210, the communication control unit 230 sends information indicating the amount of communication data with the external device 30 to the throughput measurement unit 210.

[0120] The communication discrimination unit 240 determines the type of data communication based on communication information. For example, the communication discrimination unit 240 determines the type of data communication based on IP address and port number. Figure 4 The data communication information shown is used to determine the type of data communication. The communication determination unit 240 notifies the throughput measurement unit 210 and the priority setting unit 250 of control object information containing the type of data communication.

[0121] Throughput measurement unit 210 measures the communication throughput between communication unit 202 and external device 30. Throughput measurement unit 210 calculates the current communication throughput based on the amount of communication data notified from communication control unit 230. Throughput measurement unit 210 can measure the communication throughput for each controlled object. Throughput measurement unit 210 can measure the total communication throughput. Throughput measurement unit 210 notifies the priority setting unit 250 and communication status prediction unit 220 of the measured communication throughput of the controlled objects.

[0122] The communication state prediction unit 220 predicts future communication throughput based on the communication throughput measured by the throughput measurement unit 210. For example, the communication state prediction unit 220 identifies a prediction model for the time-series data based on the time-series data of the communication throughput. The identified prediction model can be any model capable of predicting future time-series data based on past time-series data. Examples of identified prediction models include time-series models such as the AutoRegressive Model (AR model) and probabilistic differential equation models such as the Vasicek model. For example, when using the Vasicek model, the model parameters of the general solution of the probabilistic differential equation of the Vasicek model can be identified by using the general solution of the probabilistic differential equation of the Vasicek model and the time-series data, and by using methods such as maximum likelihood estimation. The communication state prediction unit 220 calculates the probability distribution of the time-series data of future communication throughput based on the identified prediction model. The communication state prediction unit 220 can predict future communication throughput based on the probability distribution of the time-series data of future communication throughput. Furthermore, the communication state prediction unit 220 can use the methods described in Patent Document 1 and Patent Document 2 to calculate the probability distribution of time-series data for future communication throughput. As described in Patent Document 1 and Patent Document 2, a correction rate calculated based on a communication model that models the transient characteristics after the start of communication for communication protocols such as TCP communication can be used to correct the time-series data to eliminate the influence of transient characteristics, and a prediction model for the time-series data can be identified based on the corrected time-series data.

[0123] The communication status prediction unit 220 can predict communication throughput based on the communication status fed back from the external device 30. The communication status prediction unit 220 can predict the future communication throughput of each controlled object. Examples of the communication status fed back from the external device 30 include network transmission delay and packet loss rate. The communication status prediction unit 220 notifies the priority setting unit 250 of the predicted future communication throughput. The communication status prediction unit 220 can calculate the future communication throughput based on the delay information set in the priority setting unit 250 as described above.

[0124] The communication status prediction unit 220 can also predict future communication status by using map information indicating the location of each area in one or more areas where the communication status is likely to deteriorate. The map information can also be stored in a corresponding manner for each area in one or more areas, with information indicating the location of that area and information indicating the communication status of that location or area.

[0125] In one embodiment, the communication status prediction unit 220 obtains information indicating the vehicle 50's own position from a self-position estimation device (not shown) disposed in the vehicle 50 or the information processing device 200. Based on the vehicle 50's movement history as indicated by the information indicating its own position, the communication status prediction unit 220 predicts the vehicle 50's future position or area. The communication status prediction unit 220 can predict the future communication status by comparing the prediction results with the map information.

[0126] In another embodiment, the communication status prediction unit 220 obtains information indicating the movement route of the vehicle 50 from a path search device (not shown) configured in the vehicle 50 or the information processing device 200. Furthermore, the communication status prediction unit 220 obtains information indicating the vehicle 50's own position from a self-position estimation device (not shown) configured in the vehicle 50 or the information processing device 200. Based on the information indicating the vehicle 50's own position and the information indicating the vehicle 50's movement route, the communication status prediction unit 220 predicts the future position or area of ​​the vehicle 50. The communication status prediction unit 220 can predict the future communication status by comparing the prediction results with the map information.

[0127] The priority setting unit 250 sets a priority for each data communication and determines the communication throughput for each data communication. The priority setting unit 250 sets the priority for each type of data communication based on control object information for each type of data communication notified from the communication discrimination unit 240, the type of data communication of the control object notified from the communication discrimination unit 240, and the predicted communication throughput notified from the communication status prediction unit 220, and sets the communication throughput according to the priority. The priority setting unit 250 can set a data communication latency that achieves the set communication throughput. The priority setting unit 250 can notify the communication status prediction unit 220 of the set data communication latency.

[0128] Furthermore, the quality calculation unit 260 can calculate a service quality index value based on future communication throughput. The communication quality can be represented by MMq. The priority setting unit 250 can calculate the communication throughput based on the service quality calculated by the quality calculation unit 260. For example, the priority setting unit 250 can refer to information that establishes a correspondence between predetermined communication quality index values ​​and communication throughput, and select a communication throughput corresponding to an index value lower than the communication quality calculated by the quality calculation unit 260. The priority setting unit 250 can set a communication throughput corresponding to a service quality of at least the lowest quality for data communication with a priority higher than a predetermined value, and set a communication throughput corresponding to a service quality of the lowest quality for data communication with a priority lower than a predetermined value, thereby adapting to future communication throughput.

[0129] The priority setting unit 250 notifies the communication control unit 230 of the set communication throughput. The communication control unit 230, based on the communication throughput of each type of data communication notified by the priority setting unit 250, delays the transmission of data for each type of data communication. Furthermore, the priority setting unit 250 notifies the control device 24 of the amount of input communication data per unit time corresponding to the set communication throughput. The control device 24, based on the amount of input communication data notified by the priority setting unit 250, limits the transmission of data for each type of data communication. For example, for each service determined by the port number, the control device 24 limits the amount of data transmitted to the external device 30. Thus, the communication throughput can be appropriately limited according to the priority of the data communication.

[0130] Figure 7 A flowchart relating to the information processing method performed by the information processing device 200 is shown. The processing shown in this flowchart begins when the communication control unit 230 detects the occurrence of communication with the external device 30.

[0131] In S702, the communication discrimination unit 240 determines the type of data communication that has occurred. For example, the communication discrimination unit 240 determines the type of data communication based on the IP address and port number. In S704, the communication discrimination unit 240 determines whether the generated data communication is a control target for communication throughput. If the data communication is not a control target for communication throughput, in S720, the communication control unit 230 determines a maximum effort control method as the control method for the data communication, and the process proceeds to S714. When the data communication is a control target for communication throughput, in S706, the throughput measurement unit 210 measures the communication throughput of the control target. In S708, the throughput measurement unit 210 determines whether communication is in progress.

[0132] When it is determined in S708 that no communication has occurred, in S716, the throughput measurement unit 210 times the time during which no communication has occurred, and the process proceeds to S714. For example, the throughput measurement unit 210 times the elapsed time since the time at which no communication was determined to have occurred. When it is determined in S708 that communication is in progress, the communication status prediction unit 220 predicts the future throughput of the object to be controlled for communication throughput (S710). In S712, the communication control unit 230 controls the communication throughput, and the process proceeds to S714. Additionally, refer to... Figure 8 The processing in S712 will be explained later.

[0133] In S714, the communication control unit 230 determines whether to continue data communication. If the time without communication, as measured by the throughput measurement unit 210, exceeds a predetermined value, the communication control unit 230 determines that data communication should not continue. Furthermore, if it receives information from the control device 24 indicating a communication disconnection, the communication control unit 230 determines that data communication should not continue. If it is determined in S714 that data communication should continue, the process proceeds to S706. If it is determined in S714 that data communication should not continue, data communication is terminated.

[0134] Figure 8 A flowchart is shown relating to a method for controlling communication throughput executed by the information processing device 200. The processing described in this flowchart can be applied to the processing in S712.

[0135] In S802, the priority setting unit 250 determines the communication throughput (minimum throughput) required to ensure a minimum quality of service for data communication that is the object of communication throughput control. For example, the priority setting unit 250 determines the minimum throughput of data communication that is the object of communication throughput control based on the quality information of the data communication information.

[0136] In S804, the priority setting unit 250 determines whether data communication can proceed based on the future communication throughput predicted by the communication status prediction unit 220. The priority setting unit 250 also determines whether the predicted quality of service calculated by the quality calculation unit 260 is at least equal to the minimum quality determined by the data communication information. If data communication is possible, the process in this flowchart ends.

[0137] When it is determined that data communication can be performed, in S806, the priority setting unit 250 determines whether data communication can be performed by limiting the communication throughput of low-priority data communication. When it is determined that data communication can be performed by limiting the communication throughput of low-priority data communication, in S808, the data communication and communication throughput of the controlled object are determined based on priority. In S810, the communication control unit 230 limits the communication throughput of the controlled object's data communication. In S812, the priority setting unit 250 instructs the control device 24 performing the data communication of the controlled object to input a communication data amount. In S814, the communication control unit 230 gradually increases the communication throughput of the data communication to the minimum throughput and ends the processing in this flowchart.

[0138] When it is determined in S806 that data communication cannot be performed even by limiting the communication throughput, in S820, the priority setting unit 250 selects the data communication to be interrupted from the non-control system data communications currently in operation based on priority. For example, the priority setting unit 250 selects the data communications to be interrupted in ascending order of priority. In S822, the priority setting unit 250 instructs the control device 24, which is executing the selected data communication, to interrupt the selected data communication. In S284, the control device 24 notifies the occupants of vehicle 50 of a temporary interruption of data communication via the user interface, and the process is transferred to S806.

[0139] Reference Figure 9 and Figure 10 Provide a detailed example of the control system implementation. Figure 9 An implementation example of the control system 1000 of vehicle 50 is shown. Figure 10 An example of the internal structure of information system device 1041 is illustrated schematically.

[0140] like Figure 9 As shown, in this embodiment, the control system 1000 includes a core ECU 1010. In this embodiment, the control system 1000 includes a TCU 1020, an AD / ADAS ECU 1021, an information system ECU 1022, a region ECU 1023, and a region ECU 1024.

[0141] In this embodiment, the control system 1000 includes a drive system device 1030, a comfort system device 1031, an alarm system device 1032, a vision system device 1033, an advanced security system device 1034, an anti-theft system device 1035, a lighting system device 1036, a door system device 1037, a drive position system device 1038, an opening and closing system device 1039, a sensor device 1040, and an information system device 1041. In this embodiment, the control system 1000 includes communication networks 1080, 1081, 1082, 1084, and 1085.

[0142] Examples of drive system devices 1030 include electric parking brakes (EPB), electric power steering systems (EPS), vehicle stability control systems (VSA), shifters, powertrain units (PDU), intelligent power units (IPU), and fuel injection devices (FI). Examples of sensor devices 1040 include sensors such as cameras, radar, and LiDAR. Examples of information system devices 1041 include information communication equipment, multimedia-related equipment, and user interface equipment.

[0143] like Figure 10As shown, in this embodiment, the information system device 1041 includes an instrument device 1052, a tuner 1053, a player 1054, a narrowband communication system 1055, a wireless charger 1056, a USB port 1057, a microphone 1062, a speaker 1064, and a display device 1070. In this embodiment, the display device 1070 includes a display 1072 and a voice recognition system 1074.

[0144] The core ECU 1010 controls the entire vehicle 50. The core ECU 1010 controls the entire vehicle 50 by controlling the TCU 1020, AD / ADAS ECU 1021, information system ECU 1022, area ECU 1023, and area ECU 1024.

[0145] TCU1020 is a telematics control unit. AD / ADAS ECU1021 is an ECU that performs controls related to automated driving (AD) and advanced driver assistance systems (ADAS). AD / ADAS ECU1021 is connected via a bus to each sensor in sensor device 1040, controls each sensor in sensor device 1040, and acquires information detected by each sensor. Information system ECU1022 is connected via a bus to each device in information system device 1041 and controls each device in information system device 1041.

[0146] The area ECU 1023 is connected to and controls the various devices in the drive system device 1030 via a bus. The area ECU 1024 is connected to the comfort system device 1031, alarm system device 1032, visibility system device 1033, advanced security system device 1034, anti-theft system device 1035, lighting system device 1036, door system device 1037, drive position system device 1038, and opening / closing system device 1039 via a bus, and controls the devices in these devices.

[0147] Display device 1070 displays an image on display 1072. Display device 1070 can execute data 32 to generate a content image and display that content image on display 1072. In the event of anticipated future communication failure, display device 1070 can display a replacement image on display 1072 instead of the content image. Display device 1070 can also display information on display 1072 indicating the likelihood of future communication failure. Display device 1070 can be an example of display unit 284.

[0148] The voice recognition system 1074 performs voice recognition processing. For example, the voice recognition system 1074 analyzes the user's voice to infer the content of the instruction or command from the user. The voice recognition system 1074 executes the instruction or command based on the analysis result. Alternatively, the display device 1070 may replace the voice recognition system 1074 or, together with the voice recognition system 1074, include an input device such as a touch panel, indicator device, or switch.

[0149] Communication networks 1080, 1081, 1082, 1084, and 1085 are an example implementation of in-vehicle network 29. Communication networks 1080, 1081, 1082, 1084, and 1085 may be Ethernet (registered trademark) networks. TCU 1020, core ECU 1010, AD / ADAS ECU 1021, information system ECU 1022, area ECU 1023, and area ECU 1024 can communicate via IP through communication networks 1080, 1081, 1082, 1084, and 1085. Furthermore, communication networks 1084 and 1085 may be CAN-enabled.

[0150] TCU1020 can be an example of the aforementioned information processing device 200. TCU1020 and core ECU1010 can also cooperate to function as the aforementioned information processing device 200. AD / ADAS ECU1021, information system ECU1022, area ECU1023 and area ECU1024 can be examples of the aforementioned control device 24.

[0151] Drive system device 1030, sensor device 1040, comfort system device 1031, alarm system device 1032, vision system device 1033, advanced security system device 1034, anti-theft system device 1035, lighting system device 1036, door system device 1037, drive position system device 1038, and opening / closing system device 1039 may be examples of control system devices for vehicle 50. Information system device 1041 may be an example of a non-control system device.

[0152] Data communication relating to the devices of sensor device 1040, drive system device 1030, comfort system device 1031, alarm system device 1032, vision system device 1033, advanced security system device 1034, anti-theft system device 1035, lighting system device 1036, door system device 1037, drive position system device 1038, and opening / closing system device 1039 may have a lower priority than data communication relating to the devices of information system device 1041.

[0153] Additionally, comfort system device 1031, alarm system device 1032, visibility system device 1033, advanced security system device 1034, anti-theft system device 1035, lighting system device 1036, door system device 1037, drive position system device 1038, and opening / closing system device 1039 may include auxiliary devices for vehicle 50. These devices can also be examples of auxiliary devices for vehicle 50.

[0154] The priority of data communication related to the auxiliary devices of vehicle 50 can also be set lower than the priority of data communication related to other devices. Among the auxiliary devices of vehicle 50, the priority of data communication related to auxiliary devices other than those included in the advanced safety system device 1034 can also be set lower than the priority of data communication related to other devices. The priority of data communication related to the auxiliary devices included in the advanced safety system device 1034 can also be set higher than the priority of data communication related to auxiliary devices included in the comfort system device 1031, alarm system device 1032, visibility system device 1033, anti-theft system device 1035, lighting system device 1036, door system device 1037, driver's position system device 1038, and opening / closing system device 1039.

[0155] As described above, vehicle 50 is an example of a mobile body. Mobile bodies include automobiles such as passenger cars and buses, saddle-type vehicles, aircraft, and ships. A mobile body can be any movable device, not limited to transportation equipment.

[0156] As described above, according to the information processing apparatus 200 and an implementation example of it, by limiting low-priority data communications, the possibility of continuing high-priority data communications can be increased. Typically, when multiple data communications are performed in a device mounted on a mobile body, communication needs to be conducted within a communication speed (also known as a communication band). However, an appropriate communication speed is not always achievable for all multiple data communications. For example, when a deteriorating communication environment leads to a reduction in communication speed, there is a problem where high-priority data communications are sometimes restricted. On the other hand, according to the information processing apparatus 200 described above, this problem can be reduced.

[0157] The core ECU 1010 can be an example of an information processing device. The core ECU 1010 can be an example of a status notification unit. The TCU 1020 can be an example of an information processing device. The TCU 1020 can be an example of a throughput measurement unit. The TCU 1020 can be an example of a communication status prediction unit. The TCU 1020 can be an example of a status notification unit. The control device 24c can be an example of a status notification unit. The device 25c can be an example of a status notification unit. The speaker 1064 can be an example of a status notification unit. The display device 1070 can be an example of a notification unit.

[0158] Figure 11 An example of the internal configuration of the voice recognition control unit 276 is shown schematically. In this embodiment, the voice recognition control unit 276 includes a trigger detection unit 1122 and a status notification unit 1124.

[0159] In this embodiment, the trigger detection unit 1122 detects trigger information indicating that the user wishes to start speech recognition processing. As described above, examples of trigger information include the "WakeUp" keyword being spoken, the start button for speech recognition processing being pressed, and an icon containing the start command for speech recognition processing being selected.

[0160] In one embodiment, the voice recognition control unit 276 acquires the user's voice data via the microphone 1062 and analyzes the voice data to detect trigger information. In another embodiment, the voice recognition control unit 276 may also receive a signal indicating that trigger information has been detected from the voice recognition system 1074.

[0161] In this embodiment, when the trigger detection unit detects trigger information, the status notification unit 1124 determines whether to notify the user of the status of the speech recognition processing based on the future communication status predicted by the communication status prediction unit 220. If the status notification unit 1124 decides to notify the user of the status of the speech recognition processing, the status notification unit 1124 may output a signal to the control device 24c indicating that the status of the speech recognition processing has been notified to the user.

[0162] Therefore, based on the future communication state predicted by the communication state prediction unit 220, the status of the speech recognition processing is notified to the user. Furthermore, if the status notification unit 1124 decides not to notify the user of the status of the speech recognition processing, the status notification unit 1124 may either not output a signal indicating that the status of the speech recognition processing has been notified to the user, or it may output a signal indicating that the status of the speech recognition processing has not been notified to the user.

[0163] The status notification unit 1124 can determine the period during which speech recognition processing (sometimes referred to as online speech recognition) utilizing the speech recognition function provided by the external device 30c can be performed, based on the future communication status predicted by the communication status prediction unit 220. The status notification unit 1124 can also notify the user of the period during which speech recognition processing utilizing the speech recognition function provided by the external device 30c can be performed as the status of speech recognition processing. The status notification unit 1124 can output a signal indicating the result of the above decision to the control device 24c.

[0164] As described above, for example, depending on the communication environment of vehicle 50, there may be a situation where the communication status prediction unit 220 cannot predict the future communication status. Therefore, when the communication status prediction unit 220 can predict the future communication status, the status notification unit 1124 can perform the various decision processing and signal output processing described above. On the other hand, when the communication status prediction unit 220 cannot predict the future communication status, the status notification unit 1124 can decide to notify the user of (i) the situation that the communication status prediction unit 220 cannot predict the future communication status, and / or (ii) the current communication status. In addition, the status notification unit 1124 can output a signal indicating the above decision result to the control device 24c.

[0165] (i) When the communication status prediction unit 220 predicts that the future communication status will be worse than the predetermined first status, or (ii) when the current communication status is worse than the predetermined second status, the status notification unit 1124 may also decide to notify the user of the situation where speech recognition processing is performed without using the speech recognition function provided by the external device 30c. The first status and the second status may be substantially the same or different. The first status may be a state where the communication status is better than the second status, or a state where the communication status is worse than the second status.

[0166] For example, the second state is determined based on (i) the frequency or probability that the communication state prediction unit 220 cannot predict future communication states, or (ii) the prediction accuracy of the communication state prediction unit 220 for future communication states. In one embodiment, a communication state in which the frequency or probability that the communication state prediction unit 220 cannot predict future communication states is greater than a predetermined value can be defined as a communication state in the second state. In other embodiments, a communication state in which the prediction accuracy of the communication state prediction unit 220 for future communication states is less than a predetermined value can be defined as a communication state in the second state.

[0167] For example, if the speech recognition system 1074 has the function of performing speech recognition offline (sometimes referred to as offline speech recognition function), the status notification unit 1124 decides to notify the user that the speech recognition processing is being performed using the offline speech recognition function. The status notification unit 1124 may output a signal indicating the result of the above decision to the control device 24c.

[0168] (i) When the communication status prediction unit 220 predicts that the future communication status will be worse than the predetermined first status, or (ii) when the current communication status is worse than the predetermined second status, the status notification unit 1124 may also decide to notify the user of at least a portion of one or more instructions available using the offline speech recognition function as the status of speech recognition processing. The status notification unit 1124 may output a signal indicating the result of the above decision to the control device 24c.

[0169] (i) If the communication status prediction unit 220 predicts that the future communication status will be worse than the predetermined first status, or (ii) if the current communication status is worse than the predetermined second status, and (i) when the speech recognition system 1074 does not have offline speech recognition functionality, or (ii) when the user has indicated an input command that uses offline speech recognition functionality but is unavailable, the status notification unit 1124 may also decide to notify the user of the status of speech recognition processing as (i) the situation where the user's input command was not accepted, or (ii) the period during which speech recognition processing using the first speech recognition function can be performed. The status notification unit 1124 may output a signal indicating the result of the above decision to the control device 24c.

[0170] Figure 12 An example of the internal configuration of the speech recognition system 1074 is shown schematically. In this embodiment, the speech recognition system 1074 includes a first speech recognition unit 1222, a second speech recognition unit 1224, a recognition device determination unit 1226, and a response unit 1228.

[0171] In this embodiment, the first speech recognition unit 1222 utilizes the speech recognition function provided by the external device 30c to perform speech recognition processing. The first speech recognition unit 1222 provides the aforementioned online speech recognition function.

[0172] In this embodiment, the second speech recognition unit 1224 does not use the speech recognition function provided by the external device 30c to perform speech recognition processing. The second speech recognition unit 1224 provides the aforementioned offline speech recognition function.

[0173] In this embodiment, the recognition device decision unit 1226 determines which of the first speech recognition unit 1222 and the second speech recognition unit 1224 to use for speech recognition processing based on the future communication state predicted by the communication state prediction unit 220. The recognition device decision unit 1226 may also determine which of the first speech recognition unit 1222 and the second speech recognition unit 1224 to use for speech recognition processing based on the current communication state. As described above, the current communication state is measured, for example, by the throughput measurement unit 210.

[0174] For example, (i) when the communication state prediction unit 220 predicts that the future communication state will be worse than the predetermined first state, or (ii) when the current communication state is worse than the predetermined second state, the recognition device decision unit 1226 decides to use the second speech recognition unit 1224 to perform speech recognition processing. The first state and the second state can be substantially the same state or different states. The first state can be a state where the communication state is better than the second state, or a state where the communication state is worse than the second state.

[0175] When the communication state prediction unit 220 predicts that the future communication state will not worsen than the predetermined first state, the recognition device decision unit 1226 may decide to use the first speech recognition unit 1222 to perform speech recognition processing. When the communication state prediction unit 220 predicts that the future communication state will be better than the predetermined first state, the recognition device decision unit 1226 may also decide to use the first speech recognition unit 1222 to perform speech recognition processing.

[0176] When the recognition device decision unit 1226 decides to use the second speech recognition unit 1224 to perform speech recognition processing, and when communication is subsequently restored, and / or when communication is expected to be restored, the recognition device decision unit 1226 may also decide whether to perform the speech recognition processing of the first speech recognition unit 1222 in addition to the speech recognition processing of the second speech recognition unit 1224. When there is information that cannot be obtained through the speech recognition processing of the second speech recognition unit 1224, or instructions or commands that cannot be corresponding to the speech recognition processing of the second speech recognition unit 1224, the recognition device decision unit 1226 may also decide to perform the speech recognition processing of the first speech recognition unit 1222.

[0177] In this embodiment, the response unit 1228 responds to the user's speech or instructions. For example, the response unit 1228 obtains information indicating the result of speech recognition processing from at least one of the first speech recognition unit 1222 and the second speech recognition unit 1224. Thus, the response unit 1228 can obtain information indicating the user's instruction or command. The response unit 1228 executes the instruction or command. The response unit 1228 can notify the user of information indicating the execution result of the instruction or command.

[0178] In one embodiment, the response unit 1228 notifies the user of the execution result by outputting information representing the execution result to the display 1072. In another embodiment, the response unit 1228 notifies the user of the execution result by outputting information representing the execution result from 1064.

[0179] The response unit 1228 can be an example of a status notification unit.

[0180] Figure 13 An example of information processing performed by vehicle 50 is shown in general terms. According to this embodiment, firstly, in step 1322 (sometimes referred to simply as step S), the communication state prediction unit 220 predicts the future communication state. In this embodiment, for ease of explanation, an example of information processing of vehicle 50 is described using the case where the communication state prediction unit 220 predicts the future communication throughput (sometimes called future throughput) as the future communication state.

[0181] Next, in S1324, it is determined whether the information processing device 200 should start speech recognition processing. For example, when the trigger detection unit 1122 detects trigger information, it is determined that the information processing device 200 should start speech recognition processing. On the other hand, when the trigger detection unit 1122 does not detect trigger information, it is determined that the information processing device 200 should not start speech recognition processing.

[0182] When it is decided that the information processing device 200 will not start speech recognition processing (the case of "No" in S1324), the processing in S1322 is repeated. On the other hand, when it is decided that the information processing device 200 will start speech recognition processing (the case of "Yes" in S1324), in S1330, it is determined whether the quality of future throughput is good. For example, the priority setting unit 250 determines whether the quality of future throughput is good based on the prediction result of the communication status prediction unit 220.

[0183] When it is determined in S1330 that the quality of future throughput is poor (the case of "No" in S1330), in S1332, the priority setting unit 250 determines the priority of data communication related to speech recognition processing. This determines the bandwidth allocated to data communication related to speech recognition processing (i.e., data communication related to data 36), the communication throughput of data communication related to speech recognition processing, etc.

[0184] Next, in S1334, the status notification unit 1124 determines the status of voice processing in the voice recognition system 1074. For example, the status notification unit 1124 determines whether the online voice recognition function can be used based on the priority of data communication related to voice recognition processing determined in S1332. The status notification unit 1124 may also estimate the response time when using the online voice recognition function based on the priority of data communication related to the aforementioned voice recognition processing. Thus, the status of voice processing in the voice recognition system 1074 is determined. In addition, the status notification unit 1124 decides to notify the user of the status of voice processing in the voice recognition system 1074.

[0185] The status notification unit 1124 can also determine the method for notifying the user of the status of voice processing of the voice recognition system 1074. Examples of the notification methods described above include voice output from the speaker 1064, and output of messages or icons to the display 1072.

[0186] Next, in S1336, it is determined whether the user's speech has been detected. For example, the speech recognition system 1074 analyzes the user's speech collected by the microphone 1062 and performs speech detection processing. When no user speech is detected (in the case of "No" in S1336), the processing ends. On the other hand, when the user's speech is detected in S1336, online speech recognition processing is performed in S1340. Specifically, the recognition device decision unit 1226 decides to use the first speech recognition unit 1222 to perform speech recognition processing. Furthermore, the first speech recognition unit 1222 uses the speech recognition function provided by the external device 30c to perform speech recognition processing. If the speech recognition processing ends, the processing ends.

[0187] On the other hand, in S1330, if it is determined that the quality of future throughput is good (the case of "yes" in S1330), in S1340, online speech recognition processing is performed according to the steps described above. If the speech recognition processing ends, the processing ends.

[0188] Figure 14 An example of information processing performed by vehicle 50 is shown in a simplified manner. According to this embodiment, firstly, by referring to... Figure 13The same steps described are used to execute S1322, S1324, and S1330.

[0189] If it is determined in S1330 that the quality of future throughput is poor (the case of "No" in S1330), in S1434, the status notification unit 1124 determines the status of speech processing of the speech recognition system 1074. For example, the status notification unit 1124 determines whether the online speech recognition function can be used based on the priority of data communication related to speech recognition processing determined in S1332. The status notification unit 1124 may also estimate the response time when using the online speech recognition function based on the priority of data communication related to the aforementioned speech recognition processing. Thus, the status of speech processing of the speech recognition system 1074 is determined.

[0190] Furthermore, the status notification unit 1124 decides to notify the user of the status of the speech processing of the speech recognition system 1074. According to this embodiment, since the quality of future throughput is poor, the notified speech recognition system 1074 cannot use the online speech recognition function, and the speech recognition system 1074 uses the offline speech recognition function to respond to the user.

[0191] Then, in S1436, it is determined whether (i) the user's speech was detected and (ii) whether the user's end instruction was accepted. For example, the speech recognition system 1074 analyzes the user's speech collected by the microphone 1062 and performs speech detection processing. When no user speech is detected (in the case of "No" in S1436), the processing ends. For example, the user can input an instruction to end the speech recognition processing by operating a touch panel, switch, etc. The user can also input the above instruction by voice. If the instruction to end the speech recognition processing is accepted (in the case of "No" in S1436), the processing ends.

[0192] On the other hand, when a user's speech is detected in S1436, in S1440, for example, the recognition device decision unit 1226 determines whether the user's instruction can be handled using an offline voice function (sometimes referred to as offline mode). If it can be handled in offline mode (the case of "yes" in S1440), offline voice recognition processing is performed in S1452. Specifically, the recognition device decision unit 1226 decides to use the second voice recognition unit 1224 to perform voice recognition processing. Furthermore, the second voice recognition unit 1224 does not use the voice recognition function provided by the external device 30c to perform voice recognition processing.

[0193] In addition, in S1454, it is determined whether the communication status has been restored. If it is not determined that the communication status has been restored (the case of "No" in S1454), the process of S1454 is repeated. On the other hand, if it is determined that the communication status has been restored in S1454 (the case of "Yes" in S1454), in S1456, the recognition device decision unit 1226 decides whether to perform online voice recognition processing to obtain additional information (sometimes called supplementary information) or to execute additional instructions or commands. If the recognition device decision unit 1226 decides to perform online voice recognition processing, online voice processing is performed in S1470.

[0194] In S1470, specifically, the recognition device decision unit 1226 decides to use the first speech recognition unit 1222 to perform speech recognition processing. Furthermore, the first speech recognition unit 1222 uses the speech recognition function provided by the external device 30c to perform speech recognition processing. If the speech recognition processing is completed, the processing ends.

[0195] On the other hand, when a response cannot be handled in offline mode in S1440 (the case of "No" in S1440), in S1462, the status notification unit 1124 decides to notify the user of the status of voice processing in the voice recognition system 1074. For example, the status notification unit 1124 decides to notify the user that the instruction or command given by the user is an instruction or command that cannot be handled in offline mode. Furthermore, the status notification unit 1124 may also decide to notify the user that the online voice recognition function cannot be used. The status notification unit 1124 may also decide to notify the user of the predicted period when the online voice recognition function can be resumed.

[0196] In addition, in S1464, it is determined whether the communication status has been restored. If it is not determined that the communication status has been restored (the case of "No" in S1464), the processing of S1464 is repeated. On the other hand, if it is determined in S1464 that the communication status has been restored, and / or there is an expectation that the communication status will be restored (the case of "Yes" in S1464), in S1466, the status notification unit 1124 decides to notify the user that online voice recognition processing can be used. For example, if the communication status prediction unit 220 predicts that the future communication status will become better than the predetermined status, it is determined that there is an expectation that the communication status will be restored. Then, in S1470, online voice processing is performed.

[0197] On the other hand, in S1330, if it is determined that the quality of future throughput is good (the case of "yes" in S1330), in S1470, online speech recognition processing is performed according to the steps described above. When the speech recognition processing ends, the processing ends.

[0198] Figure 15Examples of computer 3000 that may embody all or part of the various embodiments of the present invention are shown. For example, at least a portion of information processing device 200 is implemented by computer 3000. For example, at least a portion of control device 24 is implemented by computer 3000. For example, at least a portion of device 25 is implemented by computer 3000. For example, at least a portion of content execution unit 282 is implemented by computer 3000. For example, at least a portion of display unit 284 is implemented by computer 3000. For example, at least a portion of display device 1070 is implemented by computer 3000.

[0199] A program installed on computer 3000 enables computer 3000 to function as an operation associated with the apparatus according to this embodiment or one or more "units" of the apparatus, or enables computer 3000 to perform the operation or the one or more "units", and / or enables computer 3000 to perform the process according to this embodiment or the steps of the process. Such a program can be executed by CPU 3012 to enable computer 3000 to perform specific operations associated with some or all of the functional blocks in the flowcharts and block diagrams described in this specification.

[0200] The computer 3000 of this embodiment includes a CPU 3012, RAM 3014, GPU 3016, and a display device 3018, which are interconnected via a main controller 3010. The computer 3000 also includes input / output units such as a communication interface 3022, a hard disk drive 3024, a DVD-ROM drive 3026, and an IC card drive, which are connected to the main controller 3010 via an input / output controller 3020. The computer 3000 also includes conventional input / output units such as a ROM 3030 and a keyboard 3042, which are connected to the input / output controller 3020 via an input / output chip 3040.

[0201] CPU 3012 operates according to the programs stored in ROM 3030 and RAM 3014, thereby controlling each unit. GPU 3016 obtains image data generated by CPU 3012 from the frame buffer provided in RAM 3014 or from itself, and displays the image data on display device 3018.

[0202] The communication interface 3022 communicates with other electronic devices via a network. The hard disk drive 3024 stores programs and data used by the CPU 3012 within the computer 3000. The DVD-ROM drive 3026 reads programs or data from the DVD-ROM 3001, etc., and provides programs or data to the hard disk drive 3024 via RAM 3014. The IC card drive reads programs and data from the IC card and / or writes programs and data to the IC card.

[0203] ROM3030 internally stores the boot program executed by computer 3000 when activated, and / or programs dependent on the hardware of computer 3000. Input / output chip 3040 can also connect various input / output units to input / output controller 3020 via parallel port, serial port, keyboard port, mouse port, etc.

[0204] The program is provided by a computer-readable storage medium such as a DVD-ROM 3001 or an IC card. The program is read from the computer-readable storage medium, installed into a hard disk drive 3024, RAM 3014, or ROM 3030, which is also an example of a computer-readable storage medium, and executed by the CPU 3012. The information processing contained within these programs is read by the computer 3000, enabling cooperation between the program and the aforementioned types of hardware resources. The apparatus or method can be configured to perform information manipulation or processing in accordance with the use of the computer 3000.

[0205] For example, when communication is performed between the computer 3000 and an external device, the CPU 3012 can execute a communication program loaded into the RAM 3014, and instruct the communication interface 3022 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 3012, the communication interface 3022 reads the transmission data stored in the transmission buffer processing area provided in the recording medium such as the RAM 3014, hard disk drive 3024, DVD-ROM 3001, or IC card, and sends the read transmission data to the network, or writes the received data received from the network to the receive buffer processing area provided on the recording medium, etc.

[0206] In addition, CPU 3012 can read all or a required portion of files or databases stored on external recording media such as hard disk drive 3024, DVD-ROM drive 3026 (DVD-ROM 3001), and IC cards into RAM 3014, and perform various types of processing on the data in RAM 3014. CPU 3012 can then write the processed data back to the external recording media.

[0207] Various types of information, such as programs, data, tables, and databases, can be saved to the recording medium and processed. The CPU 3012 can perform various operations, including operations specified by a sequence of program instructions, conditional judgments, conditional branches, unconditional branches, and information retrieval / replacement, on data read from the RAM 3014, as described throughout this disclosure, and write the results back to the RAM 3014. Furthermore, the CPU 3012 can retrieve information from files, databases, etc., within the recording medium. For example, when multiple items, each having an attribute value associated with a second attribute, are stored in the recording medium, the CPU 3012 can retrieve the item whose first attribute value matches the condition from these multiple items, read the second attribute value stored in that item, and thereby obtain the second attribute value associated with the first attribute that satisfies a pre-set condition.

[0208] The programs or software modules described above can be stored on or near the computer 3000 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 3000 via the network.

[0209] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. Such modifications or improvements can also be included within the technical scope of the present invention, as is evident from the claims.

[0210] Regarding the execution order of actions, processes, steps, and stages in the apparatus, system, program, and method shown in the claims, specification, and drawings, it should be noted that unless explicitly stated as "before" or "firstly," any order is permissible as long as the output of a previous process is not used in a subsequent process. Even if terms such as "firstly" or "next" are used for convenience in describing the flow of actions in the claims, specification, and drawings, this does not imply that the actions must be performed in that specific order.

[0211] [Explanation of reference numerals in the attached figures]

[0212] 24 Control Device

[0213] 25 Equipment

[0214] 29. In-vehicle network

[0215] 30 External devices

[0216] 32 Data

[0217] 34 Data

[0218] 36 Data

[0219] 50 vehicles

[0220] 90 Communication Network

[0221] 92 Wireless Communication System

[0222] 200 Information Processing Devices

[0223] 202 Ministry of Communications

[0224] 210 Throughput Measurement Department

[0225] 220 Communication Status Prediction Department

[0226] 230 Communications Control Department

[0227] 240 Communication Discrimination Unit

[0228] 250 Priority Setting Department

[0229] 260 Quality Calculation Department

[0230] 276 Voice Recognition Control Department

[0231] 282 Content Execution Department

[0232] 284 Display Section

[0233] 1000 Control System

[0234] 1010 Core ECU

[0235] 1020 TCU

[0236] 1021 AD / ADAS ECU

[0237] 1022 Information System ECU

[0238] 1023 Area ECU

[0239] 1024 Area ECU

[0240] 1030 Drive System Equipment

[0241] 1031 Comfort System Equipment

[0242] 1032 Alarm System Equipment

[0243] 1033 Vision System Equipment

[0244] 1034 Advanced Security System Equipment

[0245] 1035 Anti-theft System Equipment

[0246] 1036 Lighting System Equipment

[0247] 1037-door system equipment

[0248] 1038 Driver Position System Equipment

[0249] 1039 Switchgear System Equipment

[0250] 1040 Sensor Device

[0251] 1041 Information System Equipment

[0252] 1052 Instrumentation Equipment

[0253] 1053 Tuner

[0254] 1054 Player

[0255] 1055 Narrowband Communication System

[0256] 1056 Wireless Charger

[0257] 1057 USB port

[0258] 1062 microphone

[0259] 1064 speakers

[0260] 1070 display device

[0261] 1072 monitor

[0262] 1074 Speech Recognition System

[0263] 1080 communication network

[0264] 1081 Communication Network

[0265] 1082 Communication Network

[0266] 1084 Communication Network

[0267] 1085 communication network

[0268] 1122 Trigger Detection Unit

[0269] 1124 Status Notification Department

[0270] 1222 First Speech Recognition Department

[0271] 1224 Second Speech Recognition Department

[0272] 1226 Identification Device Decision Unit

[0273] 1228 Response Department

[0274] 3000 computers

[0275] 3001 DVD-ROM

[0276] 3010 Main Controller

[0277] 3012 CPU

[0278] 3014 RAM

[0279] 3016 GPU

[0280] 3018 Display Devices

[0281] 3020 Input / Output Controller

[0282] 3022 Communication Interface

[0283] 3024 Hard Drive

[0284] 3026 DVD-ROM drive

[0285] 3030 ROM

[0286] 3040 Input / Output Chip

[0287] 3042 Keyboard.

Claims

1. An information processing apparatus that performs speech recognition processing via a communication network and by utilizing a first speech recognition function provided by an external speech recognition device, wherein... have: The trigger detection unit detects trigger information indicating that the user wishes to begin the speech recognition process. A communication state prediction unit that predicts future communication states; and The status notification unit, upon the trigger detection unit detecting the trigger information, notifies the user of the status of the speech recognition processing based on the future communication status predicted by the communication status prediction unit. The status notification unit determines the period during which the speech recognition process utilizing the first speech recognition function can be performed based on the future communication state predicted by the communication state prediction unit, and notifies the user of the period during which the speech recognition process utilizing the first speech recognition function can be performed as the status notification of the speech recognition process.

2. The information processing apparatus according to claim 1, wherein, The communication status refers to bandwidth or throughput.

3. The information processing apparatus according to claim 1 or 2, wherein, The status notification unit performs the following processing: If the communication status prediction unit cannot predict the future communication status, the user will be notified of (i) the situation that the communication status prediction unit cannot predict the future communication status and / or (ii) the current communication status.

4. The information processing apparatus according to claim 1 or 2, wherein, It also has: A communication control unit that controls data communication between the information processing device and external devices via a communication network; A communication discrimination unit that determines the type of the data communication; as well as The priority setting unit sets the priority of communication for multiple data communications based on the type determined by the communication discrimination unit. The communication control unit performs the following processing: Based on the future communication state predicted by the communication state prediction unit and the communication priority set by the priority setting unit, the priority of data communication with the external speech recognition device is determined. The latency is lower than the data communication priority between the external speech recognition device and the data communication, or the bandwidth of the lower priority data communication is limited.

5. The information processing apparatus according to claim 1 or 2, wherein, have: The first speech recognition unit performs the speech recognition processing by utilizing the first speech recognition function provided by the external speech recognition device; The second speech recognition unit does not utilize the first speech recognition function provided by the external speech recognition device to perform the speech recognition processing; as well as The recognition device decision unit determines, based on the future communication state predicted by the communication state prediction unit, which of the first speech recognition unit and the second speech recognition unit should be used to perform the speech recognition processing.

6. The information processing apparatus according to claim 5, wherein, The identification device's decision unit performs the following processing: (i) if the communication state prediction unit predicts that the future communication state is worse than the predetermined first state, or (ii) if the current communication state is worse than the predetermined second state. It is decided to use the second speech recognition unit to perform the speech recognition processing.

7. The information processing apparatus according to claim 5, wherein, The status notification unit performs the following processing: (i) if the communication state prediction unit predicts that the future communication state is worse than the predetermined first state, or (ii) if the current communication state is worse than the predetermined second state. The user will be notified of the status of the speech recognition process as that the second speech recognition unit is used to perform the speech recognition process.

8. The information processing apparatus according to claim 5, wherein, The status notification unit performs the following processing: (i) if the communication state prediction unit predicts that the future communication state is worse than the predetermined first state, or (ii) if the current communication state is worse than the predetermined second state. The user is notified of the status of the speech recognition process by at least a portion of one or more instructions available using the second speech recognition unit.

9. The information processing apparatus according to claim 5, wherein, The status notification unit performs the following processing: (i) when the communication state prediction unit predicts that the future communication state is worse than a predetermined first state, or (ii) when the current communication state is worse than a predetermined second state, and the user instructs to input a command that is unavailable using the second speech recognition unit. The system notifies the user of the status of the speech recognition process, including the fact that the user's input of an instruction has not been accepted and the period during which the speech recognition process utilizing the first speech recognition function can be performed.

10. A mobile body, wherein, The information processing apparatus comprising any one of claims 1 to 9.

11. A computer-readable storage medium storing a program, When the program is executed by the processor, it uses the first speech recognition function provided by an external speech recognition device through a communication network to perform the information processing method for speech recognition processing. The information processing method has the following characteristics: The trigger detection step indicates that the user wants to start the speech recognition processing. Communication status prediction steps, predicting future communication status; as well as In the status notification step, when the trigger information is detected in the trigger detection step, the user is notified of the status of the speech recognition processing based on the future communication status predicted in the communication status prediction step. In the status notification step, the period during which the speech recognition processing utilizing the first speech recognition function can be performed is determined based on the future communication state predicted in the communication state prediction step, and the period during which the speech recognition processing utilizing the first speech recognition function can be performed is notified to the user as the status of the speech recognition processing.

12. An information processing method that performs speech recognition processing by utilizing a first speech recognition function provided by an external speech recognition device through a communication network, wherein... have: The trigger detection step indicates that the user wants to start the speech recognition processing. Communication status prediction steps, predicting future communication status; as well as In the status notification step, when the trigger information is detected in the trigger detection step, the user is notified of the status of the speech recognition processing based on the future communication status predicted in the communication status prediction step. In the status notification step, the period during which the speech recognition processing utilizing the first speech recognition function can be performed is determined based on the future communication state predicted in the communication state prediction step, and the period during which the speech recognition processing utilizing the first speech recognition function can be performed is notified to the user as the status of the speech recognition processing.

Images