Communication Systems

The communication system dynamically adjusts communication load based on predicted throughput discrepancies, enhancing stability and usability by anticipating network changes, thus addressing the inefficiencies in existing systems.

JP7590906B2Active Publication Date: 2024-11-27HONDA MOTOR CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2021062087
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-03-31
Publication Date
2024-11-27
Estimated Expiration
2041-03-31

AI Technical Summary

Technical Problem

Existing communication systems in vehicles fail to seamlessly adjust communication load based on predicted future throughput, leading to instability and reduced usability when network conditions deteriorate.

Method used

A communication system that includes a terminal and a server, with prediction units to forecast future communication throughput, allowing for dynamic adjustment of communication load based on discrepancies in predicted throughput, and includes imaging and superimposition of information to enhance usability.

Benefits of technology

Seamless control of communication load between vehicles and servers, improving usability by anticipating throughput changes and adjusting data transmission parameters to maintain stability and quality of information presentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007590906000001
    Figure 0007590906000001
  • Figure 0007590906000002
    Figure 0007590906000002
  • Figure 0007590906000003
    Figure 0007590906000003
Patent Text Reader

Abstract

To provide a communication system that includes a terminal and a server for communicating with the terminal and that can change a communication load.SOLUTION: In an information presentation system 100, an in-vehicle terminal 11 includes a terminal-side communication prediction unit 13 that outputs first communication throughput prediction information in which future communication throughput between the in-vehicle terminal 11 and a server 20 is predicted. The server 20 includes a server-side communication prediction unit 23 that outputs second communication throughput prediction information in which the future communication throughput between the terminal 11 and the server 20 is predicted. A control unit 16 and a control unit 25 perform control to change a communication load based on the first communication throughput prediction information and the second communication throughput prediction information.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to communication systems. [Background technology]

[0002] Patent Document 1 describes "an information presentation system capable of recognizing an imaging target and presenting high-quality information without losing real-time presentation characteristics." [Prior art document] [Patent documents] [Patent Document 1] JP 2016-81225 A Summary of the Invention

[0003] In a first aspect of the present invention, there is provided a communication system including a terminal and a server communicating with the terminal. The terminal has a terminal-side communication prediction unit that outputs first communication throughput prediction information that predicts a future communication throughput between the terminal and the server. The server has a server-side communication prediction unit that outputs second communication throughput prediction information that predicts a future communication throughput between the terminal and the server. The system has a control unit that executes control to change a communication load based on the first communication throughput prediction information and the second communication throughput prediction information.

[0004] The control unit may compare the first communication throughput prediction information with the second communication throughput prediction information, and determine whether the result of the comparison is a first state in which a deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, or a second state in which a deviation between the first communication throughput prediction information and the second communication throughput prediction information is smaller than a predetermined value.

[0005] The communication system may further include an imaging unit that captures an image of an external environment of the terminal to obtain a captured image, and the terminal may further include a terminal-side transmitting unit that transmits the captured image and the first communication throughput prediction information to the server.

[0006] The server may further have a drawing information generation unit that generates drawing information, which is information related to the external environment of the terminal, based on the captured image, and a server-side transmission unit that transmits the drawing information and second communication throughput prediction information to the terminal, and the terminal may further have a superimposition display unit that displays presentation information in which the drawing information is superimposed on the external environment of the terminal.

[0007] The control unit may control the amount of data of the drawing information such that the amount of data of the drawing information in the first state is smaller than the amount of data of the drawing information in the second state.

[0008] The control unit may control the amount of data of the captured image so that the amount of data of the captured image in the first state is smaller than the amount of data of the captured image in the second state.

[0009] The terminal may be mounted in a vehicle, and the superimposed display unit may display the presentation information by superimposing it on scenery that is the external environment of the vehicle.

[0010] The drawing information may be additional information for vehicle navigation, and the superimposition display unit may display the presentation information in which the additional information for navigation is superimposed on scenery that is the external environment of the vehicle.

[0011] The drawing information may be augmented reality information for vehicle navigation, and the superimposition display unit may display the presentation information in which the augmented reality information is superimposed on scenery that is the external environment of the vehicle.

[0012] The terminal may further have a second drawing information generation unit that generates drawing information that is related information of the terminal's external environment based on the captured image, and the control unit may stop communication between the terminal and the server when in the first state, and the overlay display unit may display presentation information in which the drawing information generated by the second drawing information generation unit is superimposed on the terminal's external environment.

[0013] The vehicle is an autonomous vehicle that performs autonomous driving, and the control unit may switch driving control of the vehicle from remote driving control to stand-alone autonomous driving control when the vehicle is in the first state.

[0014] The control unit may be included in the terminal.

[0015] The control unit may be included in the server.

[0016] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0017] [Figure 1] 1 is a functional block diagram showing a schematic configuration of a communication system 100 according to a first embodiment. [Diagram 2] 13(a) and 13(b) are diagrams showing an example of presentation information D5 in a first example of the first embodiment. [Diagram 3] FIG. 4 is a diagram illustrating a first example of communication control in the first embodiment. [Figure 4] 13(a) and (b) are diagrams showing an example of presented information D5 in a second example of the first embodiment. FIG. [Diagram 5] FIG. 11 is a diagram illustrating a second example of communication control in the first embodiment. [Figure 6] 5 is a flowchart showing a process on the vehicle-mounted terminal 11 side in the first embodiment. [Figure 7] 5 is a flowchart showing processing on the server 20 side in the first embodiment. [Figure 8] FIG. 11 is a functional block diagram showing a schematic configuration of a communication system 200 according to a second embodiment. [Figure 9] 13(a) and (b) are diagrams showing an example of presentation information D5 in the second embodiment. [Figure 10] FIG. 11 is a diagram illustrating an example of communication control in the second embodiment. [Figure 11] FIG. 11 is a functional block diagram showing a schematic configuration of a communication system 300 according to a third embodiment. [Figure 12] 13(a) and (b) are diagrams showing an example of presentation information D5 in the third embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of communication control in the third embodiment. [Figure 14] An example of a computer 2200 is shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0019] 1 is a functional block diagram showing a schematic configuration of a communication system 100 according to a first embodiment. The communication system 100 includes a vehicle 10 and a server 20. The vehicle 10 and the server 20 communicate with each other via a network such as the Internet 30. Note that there may be a plurality of vehicles 10, and the server 20 can perform processes for presenting information in parallel with each of the vehicles 10 independently of each other.

[0020] The vehicle 10 has a camera 2 as an imaging unit, a sensor 3, a GPS 4, a display 5, a HUD (head-up display) 6, and an in-vehicle terminal 11. The in-vehicle terminal 11 has a parameter extraction unit 12, a communication throughput prediction unit 13, a superimposed display unit 14, a terminal-side transmission unit 15, a control unit 16, and a terminal-side reception unit 17. The server 20 has a server-side reception unit 21, a parameter extraction unit 22, a communication throughput prediction unit 23, a drawing information generation unit 24, a control unit 25, and a server-side transmission unit 26.

[0021] The parameter extraction unit 12 of the in-vehicle terminal 11 receives the captured image D1, the sensor information D2, and the position information D3 of the vehicle 10 from the camera 2, the sensor 3, and the GPS 4, respectively. The parameter extraction unit 12 extracts a plurality of parameters based on the input captured image D1, the sensor information D2, and the position information D3. The parameters include the frame rate (FPS) of the video, the resolution of the video or image, the compression level, or the compression method. The parameter extraction unit 12 outputs the extracted parameters to the communication throughput prediction unit 13. The parameter extraction unit 12 also outputs the captured image D1, the sensor information D2, and the position information D3 to the control unit 16.

[0022] The communication throughput prediction unit 13 as a terminal-side communication prediction unit generates first communication throughput prediction information in which future communication throughput between the vehicle 10 and the server 20 is predicted using the parameters input from the parameter extraction unit 12. The communication throughput prediction unit 13 outputs the generated first throughput prediction information to the control unit 16. The communication throughput prediction unit 13 predicts future communication throughput based on the current communication throughput. For example, the communication throughput prediction unit 13 identifies a prediction model of time series data based on time series data of the communication throughput. The identified prediction model may be any model as long as it can predict future time series data from past time series data. For example, examples of the identified prediction model include a time series model such as an AR model (AutoRegressive Model) and a stochastic differential equation model such as a Vasicek model. As an example, when the Vasicek model is used, a model parameter of a general solution of the stochastic differential equation of the Vasicek model may be identified using a method such as a maximum likelihood estimation method using a general solution of the stochastic differential equation of the Vasicek model and time series data. The communication throughput prediction unit 13 calculates a probability distribution of time series data of future communication throughput based on the identified prediction model. The communication throughput prediction unit 13 may predict future communication throughput based on the probability distribution of the time series data of future communication throughput. Note that the communication throughput prediction unit 13 may correct the time series data to remove the influence of the transient characteristics using a correction factor calculated based on a communication model that models the transient characteristics after the start of communication of a communication protocol such as TCP communication, and identify the prediction model of the time series data based on the corrected time series data.

[0023] The communication throughput prediction unit 13 may receive future throughput information predicted from an external source. For example, the communication throughput prediction unit 13 may output throughput information received from another vehicle traveling ahead of the vehicle 10 or throughput information received through V2V communication as the first throughput prediction information.

[0024] The terminal-side transmitting unit 15 transmits the captured image D1, the sensor information D2, and the position information D3 to the server-side receiving unit 21 of the server 20. In addition, the terminal-side transmitting unit 15 transmits the first throughput prediction information generated by the communication throughput predicting unit 13 to the server-side receiving unit 21 of the server 20.

[0025] The terminal-side receiving unit 17 receives the captured image D1, sensor information D2, and position information D3 transmitted by the terminal-side transmitting unit 15, and drawing information D4 as a response to the first communication throughput prediction information, from the server-side transmitting unit 26 of the server 20, and outputs it to the control unit 16. In addition, the terminal-side receiving unit 17 receives second throughput prediction information generated by a communication throughput predicting unit 23 of the server 20, which will be described later, from the server-side transmitting unit 26, and outputs it to the control unit 16.

[0026] The control unit 16 executes control to change the communication load based on the first communication throughput prediction information and the second communication throughput prediction information. Control to change the communication load refers to control to change the resolution, compression rate, resolution, etc. of the data to be transmitted. For example, the control unit 16 changes the communication load from high to low. Also, the control unit 16 changes the communication load from low to high. For example, the control unit 16 controls the communication load to be high while the network is stable, and controls the communication load to be low while the network is unstable. Here, it is assumed that the network is stable under normal circumstances, and the control unit 16 controls the communication load to be high.

[0027] In this embodiment, control with a high communication load refers to control with a high resolution and a low compression rate (compression rate when encoding for communication) for the captured image D1, and control with a high resolution for the sensor information D2 and the position information D3, and control with a large amount of data to be transmitted. Control with a low communication load refers to control with a low resolution and a high compression rate (compression rate when encoding for communication) for the captured image D1, and control with a low resolution for the sensor information D2 and the position information D3, and control with a small amount of data to be transmitted. Note that the control with a high communication load and the control with a low communication load may be control related to any one of the resolution, compression rate, and resolution.

[0028] For example, when a moving vehicle 10 enters a tunnel, the communication throughput prediction unit 13 of the in-vehicle terminal 11 recognizes that the vehicle 10 has entered the tunnel and predicts that the future communication throughput will deteriorate, but the communication throughput prediction unit 23 on the server 20 side does not yet recognize that the vehicle 10 has entered the tunnel and does not predict that the future communication throughput will deteriorate. In such a case, a deviation occurs between the first communication throughput prediction information and the second communication throughput prediction information. Using such a characteristic, in the first embodiment, the control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information, and as a result of the comparison, determines whether the state is a first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, or a second state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is smaller than a predetermined value.

[0029] The control unit 16 executes control to change the communication load for the captured image D1, the sensor information D2, and the position information D3 based on the determination result. The control unit 16 adjusts parameters of the captured image D1, the sensor information D2, and the position information D3 based on the communication load of the control to be changed, and outputs them to the terminal side transmitting unit 15. Note that the control unit 16 may change the control to one of three types: control with a high communication load, control with a medium communication load, and control with a low communication load, based on the determination result.

[0030] In addition, the control unit 16 receives drawing information D4 from the terminal side receiving unit 17, and generates presentation information D5 by superimposing the input drawing information D4 on map information as the external environment, and outputs it to the superimposition display unit 14.

[0031] The superimposition display unit 14 receives presentation information D5 generated by superimposing the drawing information D4 and map information, which is the external environment of the vehicle 10, from the control unit 16, and the superimposition display unit 14 presents the presentation information D5 on the display 5 or the HUD 6. In a first example of the first embodiment, the drawing information D4 is navigation additional information, and the presentation information D5 is map information with the navigation additional information superimposed thereon. In a second example of the first embodiment, the drawing information D4 is augmented reality information in an augmented reality tour of the vehicle, and the presentation information D5 is map information with the augmented reality information superimposed thereon.

[0032] The server-side receiving unit 21 receives the captured image D1, the sensor information D2, the position information D3, and the first communication throughput prediction information from the terminal-side transmitting unit 15. The server-side receiving unit 21 outputs the received captured image D1, the sensor information D2, the position information D3, and the first communication throughput prediction information to the parameter extracting unit 22 of the server 20. The server-side receiving unit 21 also outputs the received captured image D1, the sensor information D2, and the position information D3 to the drawing information generating unit 24. The parameter extracting unit 22 extracts parameters based on the input information.

[0033] The communication throughput prediction unit 23 as a server-side communication prediction unit predicts future communication throughput between the vehicle 10 and the server 20 using the parameters input from the parameter extraction unit 22, and generates second communication throughput prediction information. The communication throughput prediction unit 23 outputs the generated second throughput prediction information to the control unit 25. The communication throughput prediction unit 23 predicts future communication throughput using a method similar to that of the communication throughput prediction unit 13. The communication throughput prediction unit 23 may receive future throughput information predicted from an external source. For example, the communication throughput prediction unit 23 may receive an actual measurement value of throughput obtained from map information or throughput information managed as a map information DB (database), and output it as second throughput prediction information.

[0034] The drawing information generating unit 24 generates drawing information D4 based on the captured image D1, the sensor information D2, and the position information D3 received from the server-side receiving unit 21. In a first example of the first embodiment, the drawing information D4 is additional navigation information, and in a second example of the first embodiment, the drawing information D4 is augmented reality information. The drawing information generating unit 24 outputs the generated drawing information D4 to the control unit 25.

[0035] The control unit 25 executes control to change the communication load based on the first communication throughput prediction information and the second communication throughput prediction information received from the vehicle 10. The control to change the communication load refers to control to change the resolution, compression rate, resolution, etc. of the data to be transmitted.

[0036] In the first embodiment, the control unit 25 compares the first communication throughput prediction information with the second communication throughput prediction information, and determines, as a result of the comparison, whether the state is a first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, or a second state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is smaller than a predetermined value.

[0037] The control unit 25 changes the drawing information D4 between a control with a high communication load and a control with a low communication load based on the determination result. The control unit 16 adjusts the parameters of the drawing information D4 based on the communication load of the control to be changed, and outputs the adjusted parameters to the server-side transmission unit 26.

[0038] The server-side transmitting unit 26 transmits the drawing information D4 to the terminal-side receiving unit 17 of the vehicle-mounted terminal 11. In addition, the server-side transmitting unit 26 transmits the second throughput prediction information generated by the communication throughput predicting unit 23 to the terminal-side receiving unit 17 of the vehicle-mounted terminal 11.

[0039] 2(a) and (b) are diagrams showing an example of the presented information D5 in a first example of the first embodiment. FIG. 2(a) shows map information before the drawing information D4 is superimposed, and FIG. 2(b) shows the presented information D5 after the drawing information D4 is superimposed on the map information. The first example is an example in which the in-vehicle terminal 11 executes object recognition and motion prediction, and the drawing information D4 is navigation additional information. As shown in FIG. 2(b), the navigation additional information as the drawing information D4 is, for example, information such as "Watch out for people!", "Watch out for people jumping out!", and "Next intersection on the right."

[0040] Fig. 3 is a diagram showing a first example of communication control in the first embodiment. In Fig. 3, only the terminal side transmitting unit 15 and the terminal side receiving unit 17 are shown for the vehicle 10, and only the server side transmitting unit 26 and the server side receiving unit 21 are shown for the server 20, and other components are omitted. When the radio wave environment of the vehicle 10 deteriorates, for example, when the vehicle 10 enters a tunnel, a discrepancy occurs between the first communication throughput prediction information predicted by the communication throughput prediction unit 13 of the in-vehicle terminal 11 and the second communication throughput prediction information predicted by the communication throughput prediction unit 23 of the server 20.

[0041] As described above, the first example is an example in which the in-vehicle terminal 11 executes object recognition and motion prediction, and it is more important that the object recognition and motion prediction are executed accurately and in real time. Therefore, in the first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, the control unit 16 determines that the network is unstable, but controls the captured image D1, the sensor information D2, and the position information D3, which are information necessary for accurate and real-time object recognition and motion prediction, to have a high communication load. Therefore, the control unit 16 does not change the parameters of the captured image D1, the sensor information D2, and the position information D3.

[0042] On the other hand, in the first example, the quality of the image of the drawing information D4 superimposed on the map information is not important. Therefore, in the first state in which the difference between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, the control unit 25 of the server 20 adjusts parameters of the drawing information D4 transmitted from the server-side transmitting unit 26 so as to reduce the communication load.

[0043] 4(a) and (b) are diagrams showing an example of the presentation information D5 in a second example of the first embodiment. FIG. 4(a) shows map information before the drawing information D4 is superimposed, and FIG. 4(b) shows the presentation information D5 after the drawing information D4 is superimposed on the map information. The second example is an example in which the in-vehicle terminal 11 executes an augmented reality tour, and the drawing information D4 is augmented reality information. As shown in FIG. 4(b), the augmented reality information as the drawing information D4 is, for example, an "illustration of a dinosaur" or an "illustration of a cat."

[0044] Fig. 5 is a diagram showing a second example of communication control in the first embodiment. In Fig. 5, only the terminal side transmitting unit 15 and the terminal side receiving unit 17 are shown for the vehicle 10, and only the server side transmitting unit 26 and the server side receiving unit 21 are shown for the server 2, and other components are omitted. When the radio wave environment of the vehicle 10 deteriorates, a discrepancy occurs between the first communication throughput prediction information predicted by the communication throughput prediction unit 13 of the in-vehicle terminal 11 and the second communication throughput prediction information predicted by the communication throughput prediction unit 23 of the server 20.

[0045] As described above, the second example is an example in which the in-vehicle terminal 11 executes an augmented reality tour, and it is relatively unimportant that object recognition and motion prediction are performed accurately and in real time. Therefore, the control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information, and when the result of the comparison indicates a first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, the control unit 16 changes the communication load of the captured image D1, the sensor information D2, and the position information D3 transmitted from the terminal side transmission unit 15 of the in-vehicle terminal 11 to a low communication load. Note that a difference in communication load may be set for each of the captured image D1, the sensor information D2, and the position information D3, and for example, the control unit 16 may control the communication load of the captured image D1 to be lower than the communication load of the sensor information D2.

[0046] On the other hand, in the second example, the quality of the image of the drawing information D4 superimposed on the map information is important. Therefore, the control unit 25 sets the drawing information D4 transmitted from the server-side transmission unit 26 of the server 20 to a control with a high communication load. Therefore, the control unit 16 does not particularly change the parameters of the drawing information D4, and keeps the resolution high and the compression rate low.

[0047] 6 is a flowchart showing the process on the in-vehicle terminal 11 side in the first embodiment. When the captured image D1, the sensor information D2, and the position information D3 are input to the parameter extraction unit 12 of the in-vehicle terminal 11 (S01), the parameter extraction unit 12 extracts a plurality of parameters based on the input captured image D1, the sensor information D2, and the position information D3 (S02). Next, the communication throughput prediction unit 13 generates first communication throughput prediction information based on the plurality of parameters (S03).

[0048] The control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information, and determines whether the difference between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value (threshold) (S04). If the difference between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than the predetermined value (S04: YES), the control unit 16 determines whether the type of the drawing information D4 is augmented reality information or navigation additional information (S05).

[0049] If the type of the drawing information D4 is augmented reality information (S05: YES), the control unit 16 adjusts parameters of the captured image D1, the sensor information D2, and the position information D3 transmitted from the terminal-side transmitting unit 15 of the in-vehicle terminal 11 so as to control the communication load to be low (S06), and generates transmission data (S07). The transmission data is transmitted to the server-side receiving unit 21 of the server 20 by the terminal-side transmitting unit 15 of the in-vehicle terminal 11 (S08). If the difference between the first communication throughput prediction information and the second communication throughput prediction information is not equal to or greater than a predetermined value (S04: NO) and if the type of the drawing information D4 is not augmented reality information (S05: NO), the control unit 16 does not adjust the parameters of the captured image D1, the sensor information D2, and the position information D3.

[0050] 7 is a flowchart showing the process on the server 20 side in the first embodiment. When the captured image D1, the sensor information D2, and the position information D3 are input to the parameter extraction unit 22 of the server 20 (S11), the parameter extraction unit 22 extracts a plurality of parameters based on the input captured image D1, the sensor information D2, and the position information D3 (S12). Next, the communication throughput prediction unit 23 generates first communication throughput prediction information based on the plurality of parameters (S13).

[0051] The control unit 25 compares the first communication throughput prediction information with the second communication throughput prediction information, and determines whether the difference between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value (S14). If the difference between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value (S14: YES), the control unit 25 determines whether the type of the drawing information D4 is augmented reality information or navigation additional information (S15).

[0052] If the type of the drawing information D4 is not augmented reality information (S15: NO), the control unit 25 adjusts parameters of the drawing information D4 transmitted from the server-side transmitting unit 26 of the server 20 so as to control the communication load to be low (S16), and generates transmission data (S17). The transmission data is transmitted to the terminal-side receiving unit 17 of the vehicle-mounted terminal 11 by the server-side transmitting unit 26 of the server 20 (S18). If the difference between the first communication throughput prediction information and the second communication throughput prediction information is not equal to or greater than a predetermined value (S14: NO) and if the type of the drawing information D4 is augmented reality information (S15: YES), the control unit 25 does not adjust the parameters of the drawing information D4.

[0053] According to the communication system 100 of the first embodiment, the future communication throughput between the vehicle 10 and the server 20 is predicted on the vehicle 10 side and the server 20 side, first communication throughput prediction information and second communication throughput prediction information are generated, and the communication load in the communication between the vehicle 10 and the server 20 is changed based on the first communication throughput prediction information and the second communication throughput prediction information. As a result, it is possible to seamlessly control the communication between the vehicle 10 and the server 20, and usability can be improved, compared to the conventional technology in which the communication between the vehicle 10 and the server 20 is controlled after the throughput has decreased.

[0054] FIG. 8 is a functional block diagram showing a schematic configuration of a communication system 200 according to a second embodiment. In the second embodiment, only configurations different from those of the first embodiment will be described, and the same configurations as those of the first embodiment will be denoted by the same reference numerals and will not be described. The in-vehicle terminal 11 of the communication system 200 according to the second embodiment has a drawing information generating unit 18 as a second drawing information generating unit. The drawing information generating unit 18 generates drawing information D4 to be superimposed on map information based on information received from the control unit 16, and the presentation information D5 generated by superimposing the drawing information D4 and the captured image by the superimposition display unit 14 is presented on the display 5 or the HUD 6 (route 2). In this case, the generation of the drawing information D4 is completed within the in-vehicle terminal 11 without going through the server 20, which is a stand-alone control. The stand-alone control is an example of a control with a low communication load.

[0055] As in the first embodiment, the drawing information D4 may be generated by the drawing information generating unit 24 on the server 20 side (route 1). That is, in the second embodiment, it is possible to switch between the case where the drawing information D4 is generated by the drawing information generating unit 24 on the server 20 side (route 1) and the case where the drawing information D4 is generated by the drawing information generating unit 18 on the vehicle-mounted terminal 11 side (route 2).

[0056] 9(a) and (b) are diagrams showing an example of the presentation information D5 in the second embodiment. FIG. 9(a) shows a case where the drawing information D4 is generated on the server 20 side, and FIG. 9(b) shows a case where the drawing information D4 is generated on the in-vehicle terminal 11 side. While the network is stable, the drawing information D4 is generated on the server 20 side as shown in FIG. 9(a), but when the network becomes unstable, a message "The network is unstable. Some functions will be controlled in a standalone manner" is displayed to the user of the vehicle 10 as shown in FIG. 9(b), and the process switches to the drawing information D4 being generated on the in-vehicle terminal 11 side.

[0057] FIG. 10 is a diagram showing an example of communication control in the second embodiment. In FIG. 10, only the terminal side transmitting unit 15 and the terminal side receiving unit 17 are shown for the vehicle 10, and only the server side transmitting unit 26 and the server side receiving unit 21 are shown for the server 2, and other components are omitted. The control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information, and when the result of the comparison indicates a first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, the control unit 16 stops the transmission and reception of information between the vehicle 10 and the server 20 via the Internet 30. In this case, the vehicle-mounted terminal 11 is in a stand-alone control state, and the generation of the drawing information D4 is performed within the vehicle-mounted terminal 11.

[0058] According to the communication system 200 of the second embodiment, the future communication throughput between the vehicle 10 and the server 20 is predicted on the vehicle 10 side and the server 20 side, and the communication load in the communication between the vehicle 10 and the server 20 is changed. As a result, compared to the conventional technology in which the communication between the vehicle 10 and the server 20 is controlled after the throughput has decreased, it is possible to seamlessly control the communication between the vehicle 10 and the server 20, and it is possible to improve usability.

[0059] According to the communication system 200 of the second embodiment, when the network is unstable, the vehicle-mounted terminal 11 is in stand-alone control, and the generation of the drawing information D4 is performed within the vehicle-mounted terminal 11. Therefore, when the network is significantly unstable, there is no need to communicate between the vehicle 10 and the server 20, and usability can be improved.

[0060] FIG. 11 is a functional block diagram showing a schematic configuration of a communication system 300 according to a third embodiment. In the third embodiment, only configurations different from the first embodiment will be described, and the same configurations as the first embodiment will be given the same reference numerals and description will be omitted. The in-vehicle terminal 11 of the communication system 300 according to the third embodiment has an automatic driving control unit 19. The vehicle 10 has a steering function 7. The vehicle 10 is an automatic driving vehicle. The steering function 7 of the vehicle 10 is a steering function such as a steering wheel, an accelerator, and a brake. In the third embodiment, the control unit 16 generates presentation information D5 for the user and presents it on a display or the like. In FIG. 11, configurations such as a display are omitted.

[0061] The autonomous driving control unit 19 generates autonomous driving instruction information D7 based on the captured image D1, sensor information D2, and position information D3 received from the control unit 16. The autonomous driving control unit 19 is connected to the vehicle steering function 7, and the autonomous driving instruction information D7 is transmitted to the steering function 7. In this case, the vehicle 10 performs standalone autonomous driving based on the autonomous driving instruction information D7 (route 2).

[0062] The server 20 has a remote driving control unit 27. The remote driving control unit 27 generates remote driving instruction information D6 based on the captured image D1, sensor information D2, and position information D3 transmitted from the terminal-side transmission unit 15 of the in-vehicle terminal 11. The remote driving instruction information D6 is transmitted to the steering function 7 of the vehicle 10 via the control unit 16 of the in-vehicle terminal 11, and the steering function 7 of the vehicle 10 performs remote automatic driving of the vehicle 10 based on the remote driving instruction information D6 (route 1).

[0063] As described above, in the third embodiment, it is possible to switch between a case in which the vehicle 10 performs stand-alone autonomous driving based on autonomous driving instruction information D7 (route 2) and a case in which the vehicle 10 performs remote autonomous driving based on remote driving instruction information D6 (route 1).

[0064] 12(a) and (b) are diagrams showing an example of the presentation information D5 in the third embodiment. FIG. 12(a) shows a case where the vehicle 10 performs remote automatic driving, and FIG. 10(b) shows a case where the vehicle 10 performs standalone automatic driving. While the network is stable, the vehicle 10 performs remote automatic driving as shown in FIG. 12(a), but when the network becomes unstable, a message "Network is unstable. Switching to standalone automatic driving" is displayed to the user of the vehicle 10 as shown in FIG. 12(b), and the vehicle 10 performs standalone automatic driving.

[0065] FIG. 13 is a diagram showing an example of communication control in the third embodiment. In FIG. 13, only the terminal side transmitting unit 15 and the terminal side receiving unit 17 are shown for the vehicle 10, and only the server side transmitting unit 26 and the server side receiving unit 21 are shown for the server 2, and other components are omitted. The control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information, and when the result of the comparison indicates a first state in which the deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, the control unit 16 stops transmission and reception of information between the vehicle 10 and the server 20 via the Internet 30. In this case, the vehicle 10 is controlled in a standalone manner, and the vehicle 10 is switched from remote automated driving to standalone automated driving.

[0066] In the third embodiment, if the network becomes unstable, a message saying "The network is unstable, switching to manual driving" may be displayed to the user of the vehicle 10, and the remote automatic driving may be switched to manual driving.

[0067] According to the communication system 300 of the third embodiment, the future communication throughput between the vehicle 10 and the server 20 is predicted on the vehicle 10 side and the server 20 side, and the communication load in the communication between the vehicle 10 and the server 20 is changed. As a result, compared to the conventional technology in which the communication between the vehicle 10 and the server 20 is controlled after the throughput has decreased, it is possible to seamlessly control the communication between the vehicle 10 and the server 20, and it is possible to improve usability.

[0068] According to the communication system 300 of the third embodiment, when the network is unstable, the in-vehicle terminal 11 is in stand-alone control and the vehicle 10 performs stand-alone automatic driving. Therefore, when the network is significantly unstable, communication between the vehicle 10 and the server 20 is not required, and usability can be improved.

[0069] In the above embodiment, the control unit 16 compares the first communication throughput prediction information with the second communication throughput prediction information. However, the control unit 16 may compare the first communication throughput prediction information with a predetermined setting value instead of the second communication throughput prediction information. Similarly, the control unit 25 may compare the second communication throughput prediction information with a predetermined setting value instead of the first communication throughput prediction information. In this case, the network may be determined to be unstable when the difference between the first communication throughput prediction information and the predetermined setting value or the difference between the second communication throughput prediction information and the predetermined setting value exceeds a threshold value.

[0070] In the above embodiment, the presentation information D5 presented on the display 5 or HUD 6 by the superimposed display unit 14 is not limited to navigation information or an augmented reality (AR) tour image, and may be, for example, a streaming video game. In this case, similar to the above embodiment, if the network becomes unstable and a dissociation (reduction) occurs in future throughput, the information may have a function of playing images in a stand-alone mode. Also, the user may be notified in advance that the future communication state will be unstable. This can improve usability.

[0071] 14 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to an embodiment of the present invention, and / or to perform a process or steps of the process according to an embodiment of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0072] The computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphic controller 2216, and a display device 2218, which are connected to each other by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0073] The CPU 2212 operates according to a program stored in the ROM 2230 and the RAM 2214, thereby controlling each unit. The graphic controller 2216 retrieves image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphic controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0074] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0075] The ROM 2230 stores therein a boot program or the like executed by the computer 2200 upon activation, and / or a program that depends on the hardware of the computer 2200. The input / output chip 2240 may also connect various input / output units to the input / output controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0076] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, the RAM 2214, or the ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by realizing information manipulation or processing according to the use of the computer 2200.

[0077] For example, when communication is performed between the computer 2200 and an external device, the CPU 2212 may execute a communication program loaded in the RAM 2214 and instruct the communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2212, the communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in the RAM 2214, the hard disk drive 2224, the DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0078] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0079] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 2212 may perform various types of processing on the data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 2212 may search for an entry that matches a condition, in which an attribute value of the first attribute is specified, from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0080] The above-described programs or software modules may be stored in a computer-readable medium on the computer 2200 or in the vicinity of the computer 2200. Also, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0081] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0082] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and may be realized in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is essential to perform the process in this order. [Explanation of symbols]

[0083] 2 camera, 3 sensor, 4 GPS, 5 display, 6 HUD, 10 vehicle, 11 in-vehicle terminal, 12 parameter extraction unit, 13 communication throughput prediction unit, 14 superimposition display unit, 15 terminal side transmission unit, 16 control unit, 17 terminal side reception unit, 20 server, 21 server side reception unit, 22 parameter extraction unit, 23 communication throughput prediction unit, 24 drawing information generation unit, 25 control unit, 26 server side transmission unit, 100 communication system

Claims

1. A communication system comprising a terminal and a server that communicates with the terminal, The terminal includes: a terminal-side communication prediction unit that outputs first communication throughput prediction information in which a future communication throughput between the terminal and the server is predicted; The server, a server-side communication prediction unit that outputs second communication throughput prediction information in which the future communication throughput between the terminal and the server is predicted; The communication system includes: a control unit that executes control to change a communication load based on the first communication throughput prediction information and the second communication throughput prediction information; Communication systems.

2. The control unit compares the first communication throughput prediction information with the second communication throughput prediction information, determining whether a result of the comparison indicates a first state in which a deviation between the first communication throughput prediction information and the second communication throughput prediction information is equal to or greater than a predetermined value, or a second state in which a deviation between the first communication throughput prediction information and the second communication throughput prediction information is smaller than the predetermined value; The communication system according to claim 1 .

3. The communication system includes: An imaging unit that captures an image of an external environment of the terminal and acquires the captured image, The terminal includes: a terminal side transmitting unit configured to transmit the captured image and the first communication throughput prediction information to the server; The communication system according to claim 2.

4. The server, a drawing information generating unit that generates drawing information related to the external environment of the terminal based on the captured image; a server-side transmitting unit that transmits the drawing information and the second communication throughput prediction information to the terminal; and The terminal includes: The display device further includes a superimposition display unit that displays presentation information in which the drawing information is superimposed on the external environment of the terminal. The communication system according to claim 3.

5. the control unit controls the amount of data of the drawing information such that the amount of data of the drawing information in the first state is smaller than the amount of data of the drawing information in the second state.

5. The communication system according to claim 4.

6. the control unit controls the amount of data of the captured image in the first state so that the amount of data of the captured image in the second state is smaller than the amount of data of the captured image in the first state. A communication system according to claim 4 or 5.

7. The terminal is mounted in a vehicle, the superimposed display unit displays the presentation information by superimposing it on a landscape that is the external environment of the vehicle. A communication system according to any one of claims 4 to 6.

8. the drawing information is additional information for navigation of the vehicle; the superimposition display unit displays the presentation information in which the navigation additional information is superimposed on a landscape that is the external environment of the vehicle.

8. The communication system according to claim 7.

9. the drawing information is augmented reality information in navigation of the vehicle; the superimposition display unit displays the presentation information in which the augmented reality information is superimposed on a landscape that is the external environment of the vehicle.

8. The communication system according to claim 7.

10. The terminal includes: a second drawing information generating unit configured to generate drawing information, which is information related to the external environment of the terminal, based on the captured image; the control unit stops communication between the terminal and the server when the terminal is in the first state; The superimposed display unit displays the presentation information in which the drawing information generated by the second drawing information generation unit is superimposed on the external environment of the terminal. A communication system according to any one of claims 7 to 9.

11. The vehicle is an autonomous vehicle that performs autonomous driving, The control unit switches a driving control of the vehicle from a remote driving control to a stand-alone automatic driving control when the vehicle is in the first state. A communication system according to any one of claims 7 to 10.

12. The control unit is provided in the terminal. A communication system according to any one of claims 1 to 11.

13. The control unit is provided in the server. A communication system according to any one of claims 1 to 11.

Citation Information

Patent Citations

  • Communication system, communication terminal device, information server device, repeating device, and communicating method

    JP2000278320A

  • Remote regenerating system, method of remote regenerating and computer program

    JP2006173973A

  • On-vehicle navigation device

    JP2008202987A

  • Reproduction device, control method and control program

    JP2014204270A

  • Communication apparatus, communication system and communication method

    JP2015029218A