Information processing device and program

Abstract

To provide a technology capable of presenting contents of image information in a more intuitive and easy-to-understand manner.SOLUTION: A camera information acquisition unit 80 acquires image information acquired by sensors constituting a vehicle sensor 10, particularly cameras (front, side, and rear). A language conversion unit 81 generates image verbalization information indicating changes in targets, events, and situations that may affect movements of the vehicle based on the image information acquired by the camera information acquisition unit 80 to output the image verbalization information. An output control unit 82 performs control to output various information including text information or audio information to a HMI 20, or the like.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device program. [Background technology]

[0002] Autonomous driving technology has been one of the most popular technologies in recent years. Image recognition technology is also important for realizing autonomous driving. For example, Patent Document 1 proposes a technique that outputs the recognition results of image information as text, enabling the user to intuitively grasp the results (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication 2015-184798 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned Patent Document 1 is merely intended to output text for general-purpose image information, and the content of the text to be output is basically limited to simple content that is based on the assumption that text will be generated on a rule-based basis.

[0005] The present invention has been made in view of the above circumstances, and has an object to provide a technique capable of presenting the contents of image information in a more intuitive and easy-to-understand manner.

[0006] In order to achieve the above object, the first aspect of the present invention is an acquisition unit that acquires information regarding an image of an external environment of a moving object; an output unit that converts at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputs the converted change; The information processing device includes:

[0007] Also, in the first aspect, the output unit can determine a priority for outputting the language according to a degree of influence that may be exerted on the moving object.

[0008] In addition, in a first aspect, the output unit can determine the priority to be high if the event or change in the situation to be converted is predicted to have a high impact on the moving body within a predetermined time from the current time, and can determine the priority to be low if the impact is predicted to be low.

[0009] Also, in the first aspect, when there are a plurality of the events or changes in the situation determined to have different priorities, the output unit can output in the language based on the priorities.

[0010] In addition, in the first aspect, when there are multiple events or changes in the situation having the same priority, the output unit may not impose any order restriction on the order of the events or changes in the situation having the same priority.

[0011] Also, in the first aspect, when a predetermined character limit is set for the output of the language and it is determined that the language to be output will exceed the set character limit, the output unit can not output the language in order of lowest priority.

[0012] Also, in the first aspect, when it is determined that the set character limit will be exceeded, the output unit can not output as the language those with lower priorities that have a lower impact on the moving body.

[0013] Also, in the first aspect, when privacy information is included in the event or the change in the situation that is the subject of the conversion, the output unit can determine that the priority is low.

[0014] Also, in the first aspect, the output unit can output traffic safety information to a user of the moving object.

[0015] Also, in the first aspect, when the output unit outputs a specified content in the language, it can restrict output of content similar to the content once output for a specified period of time.

[0016] The second aspect of the present invention is A computer-implemented information processing method An acquisition step of acquiring information regarding an image of an external environment of the moving body; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; The present invention relates to an information processing method.

[0017] The third aspect of the present invention is On the computer, An acquisition step of acquiring information regarding an image of an external environment of the moving body; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; It is a program that executes control processing including the above.

[0018] An information processing method and a program according to one aspect of the present invention are also provided as an information processing method or a program corresponding to the information processing device according to one aspect of the present invention. Effect of the Invention

[0019] According to the present invention, it is possible to provide a technique that can present the contents of image information in a more intuitive and easy-to-understand manner. [Brief description of the drawings]

[0020] [Figure 1]1 is a diagram illustrating an example of an information processing system including a vehicle system according to an embodiment of the present disclosure. [Diagram 2] 1 is a diagram illustrating an example of a hardware configuration of an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure. [Diagram 3] 1 is a diagram illustrating an example of a functional configuration of an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure. [Figure 4] FIG. 13 is a diagram showing an example of criteria for determining priority by an information legal processing system according to an embodiment of the present disclosure. [Diagram 5] A figure showing an example of a specific method in which an information law processing system according to an embodiment of the present disclosure performs verbalization. [Figure 6] FIG. 2 is a diagram illustrating an example of the flow of various processes executed by an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] [Embodiment] FIG. 1 is a diagram illustrating an example of an information processing system including a vehicle system according to an embodiment of the present disclosure (hereinafter, referred to as "this system").

[0022] As shown in FIG. 1, a vehicle system S according to an embodiment of the present disclosure includes an in-vehicle device 1, a vehicle sensor 10, and an HMI (Human Machine Interface) 20. and a control ECU 30 (Electronic Control Unit). These devices and equipment are connected to each other via a predetermined network such as a Controller Area Network (CAN) or Ethernet. Here, the vehicle on which the vehicle system S is mounted may include any vehicle, moving object, etc., including, for example, an automobile powered by electricity or gasoline.

[0023] The vehicle sensor 10 is composed of various sensors for detecting the external environment around the vehicle (an environment that may include other vehicles, pedestrians, structures, road shapes, etc.). Here, the external environment around the vehicle may include, for example, traffic participants (other vehicles, pedestrians, etc.), buildings such as commercial facilities, traffic signs installed on the side of the road, road markings formed on the road surface, dividing lines, traffic lights, utility poles, guardrails, animals, fallen objects, etc. In addition, the external environment around the vehicle may include, for example, weather, information about the road surface (roads, sidewalks, etc.) on which the mobile object can move and the condition of the road surface (wet road surface, uneven road surface, etc.). Specifically, as shown in FIG. 1, the vehicle sensor 10 includes a camera (front camera) installed to capture an image in front of the vehicle, a camera (side camera) installed to capture an image on the side of the vehicle, a camera (rear camera) installed to capture an image behind the vehicle, a millimeter wave radar, an ultrasonic radar, a LiDAR (Light Detection And Ranging), an acceleration sensor, a GNSS (Global Navigation Satellite System), an external microphone (sound collection device), on-board instruments, and the like.

[0024] Here, the camera is, for example, a camera using a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), etc. In the present disclosure, a total of multiple cameras may be installed in the front, sides, and rear of the vehicle.

[0025] In addition, the external microphone (sound collection device) is composed of a general-purpose microphone, etc., and is used to obtain information about sounds made by objects outside the vehicle, including, for example, ambulance and police car sirens, human voices, etc.

[0026] The HMI 20 presents various information to the driver and passengers of the vehicle and accepts various input operations. Specifically, as shown in Fig. 1, the HMI 20 includes, for example, a display, operation buttons, a microphone, various navigation systems, a speaker, etc.

[0027] The control ECU 30 is connected to the in-vehicle device 1, transmits and receives various information, and executes various controls related to the operation of the vehicle. Specifically, as shown in Fig. 1, the control ECU includes individual ECUs that execute various controls, and executes various controls such as brake control, accelerator control, steering control, lights such as turn signals and lights, power unit, transmission, suspension, etc.

[0028] As shown in FIG. 1, the system may include a vehicle system S (vehicle-mounted device 1) managed by a driver of the vehicle or the like, and a server 2 managed by an administrator of the system or the like. The vehicle system S and the server 2 may be connected to each other via a predetermined network N such as the Internet. However, the network N is not a required component, and for example, NFC (Near Field Communication), Bluetooth (registered trademark), LAN (Local Area Network), etc. may be used. The server 2 acquires various types of information related to vehicle operation that are periodically transmitted from the vehicle system S (particularly the in-vehicle device 1) and is used to manage the various types of information. In the following description, when the term "image" is used simply, it broadly includes "moving image," "still image," "time-series image," and the like.

[0029] FIG. 2 is a diagram illustrating an example of a hardware configuration of an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure.

[0030] As shown in FIG. 2, the in-car device 1 includes a control unit 41, a read only memory (ROM) 42, a random access memory (RAM) 43, a bus 44, an input / output interface 45, a storage unit 46, and a communication unit 47.

[0031] The control unit 41 is configured with a microcomputer including a CPU, a GPU, an FPGA (Field-Programmable Gate Array), a semiconductor memory, etc. The control unit 41 executes various processes according to a program recorded in a ROM 42 or a program loaded from a storage unit 46 to a RAM 43. The RAM 43 also stores information necessary for the control unit 41 to execute various processes as appropriate.

[0032] The control unit 41, the ROM 42, and the RAM 43 are connected to one another via a bus 44. An input / output interface 45 is also connected to the bus 44. The input / output interface 45 is connected to the vehicle sensor 10, the HMI 20, the control ECU 30, a storage unit 46, a communication unit 47, and the like.

[0033] The storage unit 46 is configured with a hard disk drive (HDD), a solid state drive (SSD), etc., and stores various information. For example, the storage unit 46 stores various programs and the like required for executing various processes related to the present system.

[0034] The communication unit 47 controls mutual communications with other hardware and the like via a network N including the Internet.

[0035] The hardware configuration of the server 2 is not described here because it can be basically the same as the hardware configuration of the in-vehicle device 1. The cooperation of such various hardware and software enables the in-vehicle device 1 to execute various processes, which will be described later.

[0036] FIG. 3 is a diagram illustrating an example of a functional configuration of an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure.

[0037] As shown in FIG. 3, in the control unit 41 of the in-car device 1, a camera information acquisition unit 80, a language conversion unit 81, and an output control unit 82 function by executing various programs. Moreover, a model information DB300 is provided in one area of ​​the storage unit 46 of the in-vehicle device 1. The model information DB300 stores a trained model (a program or the like that specifies the results of training that has been subjected to statistical processing using the in-vehicle device 1 or other large-scale language models, etc.) for generating image verbalization information, which will be described later, based on image information, which will be described later. Note that the training method used here may be, for example, a method classified into various deep learning methods such as DNN (Deep Neural Network), or a combination thereof.

[0038] The camera information acquisition unit 80 acquires information related to images of the external environment of the moving body. Specifically, the camera information acquisition unit 80 acquires various information (hereinafter referred to as "image information") related to images captured by the sensors constituting the vehicle sensor 10, particularly the cameras (front, side, rear).

[0039] The language conversion unit 81 converts at least one change in a target, an event, or a situation that may affect the movement of the moving body into a language that can be recognized by a person and outputs the converted information based on the image. Specifically, the language conversion unit 81 generates a text sentence (hereinafter referred to as "image verbalization information") indicating a change in a target, an event, or a situation that may affect the movement of the vehicle based on the image information acquired by the camera information acquisition unit 80, and outputs the image verbalization information. The language conversion unit 81 determines the order in which the generated image verbalization information is output according to a priority described later. Here, the target object typically includes, for example, vehicles other than the vehicle itself, pedestrians, road signs, billboards, buildings such as commercial facilities, road markings formed on the road surface, dividing lines, traffic lights, utility poles, guardrails, fallen objects, etc. Specific examples of language conversion by the language conversion unit 81 will be described later with reference to Figs. 4 and 5.

[0040] Here, the language conversion unit 81 is provided with a priority determination unit 100 . The priority determination unit 100 determines the priority of the output according to the degree of influence that the output may have on the moving body. Specifically, the priority determination unit 100 determines the degree of influence that each target, event, and situation change may have on the movement of the vehicle based on the contents of the image information acquired by the camera information acquisition unit 80, and determines the priority for verbalizing each target, event, and situation change according to the degree of influence. For example, the system determines a high priority if a target, event, or change in situation is predicted to have a high impact on the vehicle within a specified time from the current time, and determines a low priority if the impact is predicted to be low.

[0041] The output control unit 82 executes control for outputting various information including text information, audio information, and the like to the HMI 20 and the like.

[0042] FIG. 4 is a diagram illustrating an example of a criterion for determining a priority by the information legal processing system according to an embodiment of the present disclosure.

[0043] In the example of Fig. 4, a specific example of the method by which the present system determines the priority (calculates the degree of influence) is displayed as a table for each target, event, or situation. For example, in the example of Fig. 4, examples of targets, events, or situations that have a high priority for verbalization, that is, have a high degree of influence on the vehicle, are displayed as follows: (other vehicles) "vehicles approaching from the front, motorcycles and bicycles traveling on the side", (pedestrians) "pedestrians crossing or waiting to cross", (traffic signs, etc.) "changes in traffic light status, stop signs, no entry signs", (accidents, congestion, regulations, construction) "accidents or congestion ahead", (road surface conditions and conditions) "objects falling on the road", (road structure) "intersections, sharp curves", (others) "high risk to the vehicle ahead, sudden change in conditions". When such targets, events, or situations are included in the acquired image information, the present system verbalizes the targets, events, or situations with priority. On the other hand, in the example of Fig. 4, examples of targets, events, or situations with low priority for verbalization, i.e., with low impact on the vehicle, are displayed as follows: (other vehicles) "vehicles changing lanes", (traffic signs, etc.) "signs and speed limits on expressways and trunk roads", (accidents, congestion, restrictions, construction) "school zones, road construction, restrictions", (road conditions and status) "wet or icy roads, etc.", and (other) "information that can be normally obtained by navigation systems, information with low urgency, etc." When such targets, events, or situations are included in the acquired image information, the system does not prioritize verbalizing the targets, events, or situations (verbalizing when there are no other targets, events, or situations with high priority for verbalization). Further, in the example of FIG. 4, examples of targets, events, or situations that do not need to be verbalized are displayed, such as (other vehicles) "vehicles that do not affect the course, vehicle exteriors, designs, and license plates," (pedestrians) "pedestrians that do not affect the vehicle (e.g., not crossing the road)," (targets not related to traffic) "trees, buildings along the road, advertising billboards," and (others) "information that does not affect operation, privacy information, etc." In principle, the system may not verbalize such targets, events, or situations when the acquired image information includes such targets, events, or situations. For example, the system may determine the degree of influence that may be exerted on the movement of the vehicle according to such a judgment criterion, and determine the priority of verbalization. Note that information related to the judgment criterion as shown in FIG. 4 is stored in the storage unit 46 or the like as information in any format. Note that the privacy information is not limited to so-called personal information, and may broadly include, for example, information that can identify an individual, information that may be considered unpleasant if explicitly depicted, etc. Specifically, in the example of this embodiment, the face and physical features of people (pedestrians, passengers in a vehicle, etc.), the interior appearance of other vehicles, buildings, etc., are included in the privacy information. In summary, when there are a plurality of events or changes in circumstances that have been determined to have different priorities, the present system outputs them as image verbalization information based on the determined priorities.

[0044] FIG. 5 is a diagram showing an example of a specific method in which the information law processing system according to an embodiment of the present disclosure performs verbalization.

[0045] In the example of FIG. 5, a specific example of an image that is to be verbalized by this system is displayed. Looking at the example in FIG. 5, for example, there is an intersection ahead of the vehicle. Such an object, event, or situation corresponds to the high priority content (intersection) in the judgment criteria in the example in FIG. 4. Looking at the example in FIG. 5, for example, a traffic light installed ahead of the vehicle indicates that the vehicle is only permitted to move forward and turn left. Such an object, event, or situation corresponds to the high priority content (change in the traffic light status) in the judgment criteria in the example in FIG. 4. For example, this system verbalizes a sentence such as "There is an intersection ahead of the vehicle, and the traffic light ahead allows the vehicle to move forward and turn left" and outputs it with a high priority. On the other hand, in the example of FIG. 5, a large truck is traveling ahead of the vehicle on the right side, and depending on the situation, this large truck may change lanes into the lane in which the vehicle is traveling. Such an object, event, or situation corresponds to, for example, a low priority content (vehicle changing lanes) in the judgment criteria of the example of FIG. 4. For example, the system verbalizes a sentence such as "There is a possibility that a truck in the right lane will change lanes" and outputs it with a low priority. Note that, for example, information such as the design of the vehicle or the license plate is not output as a sentence in principle because it is not important or necessary to verbalize it to the vehicle occupants.

[0046] FIG. 6 is a diagram illustrating an example of the flow of various processes executed by an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure.

[0047] In step S1, the camera information acquisition unit 80 acquires image information acquired by the sensors constituting the vehicle sensor 10, particularly the cameras (front, side, rear).

[0048] In step S2, the priority determination unit 100 calculates, based on the contents of the image information acquired by the camera information acquisition unit 80, the degree of influence that each target, event, and change in situation may have on the movement of the vehicle.

[0049] In step S3, the priority determination unit 100 determines the priority for verbalizing each of the targets, events, and changes in the situation, depending on the calculated degree of influence.

[0050] In step S4, the language conversion unit 81 generates image language information indicating changes in targets, events, and situations that may affect the movement of the vehicle, based on the image information acquired by the camera information acquisition unit 80. In step S4, the language conversion unit 81 outputs the generated image language information in accordance with the priority determined by the priority determination unit 100. This ends the image language processing of the in-car device 1.

[0051] Although one embodiment of the present disclosure has been described above, the present disclosure is not limited to the above-described embodiment, and modifications and improvements within the scope that can achieve the object of the present disclosure are included in the present disclosure.

[0052] [Other embodiments] In the above embodiment, the image verbalization information is output as text information, but this is not limited to this. The image verbalization information may be output in other formats, such as audio.

[0053] In the above embodiment, the vehicle is described as a general-purpose autonomous vehicle, but is not limited thereto. Vehicles to which the present system is applicable may include any type of moving body, regardless of shape or power source, such as automobiles, trucks, motorcycles, railroad cars, and bicycles. Furthermore, the information processing device or information processing system according to the present system does not need to function independently as an information processing device, and may be provided as an integrated part with a vehicle (mobile body), for example.

[0054] Although the above embodiment has been described simply, it is assumed that the information stored in the model information DB 300 has undergone learning processing in advance by the in-vehicle device 1 or other hardware, etc. Specifically, the information stored in the model information DB 300 is model information that has been extensively learned from various videos, texts, etc., and has been sufficiently adjusted for application to autonomous driving. Furthermore, in particular, in advance learning of the model information DB 300, learning may be performed using a general-purpose large language model such as ChatGPT or BERT (Bidirectional Encoder Representations from Transformers).

[0055] Although the above embodiment has been described simply, the present system can perform reinforcement learning and update various model information. Specifically, for example, the present system can update the contents of the model in the model information DB300 using the generated image verbalization information, etc. For example, in this case, the present system may learn using other hardware, etc., based on various information acquired by the in-vehicle device 1, etc., and acquire the newly generated trained model via the Internet, etc.

[0056] Although the above embodiment has been described simply, the types and number of various sensors included in the vehicle sensor 10 can be determined at the discretion of the administrator of the system, etc. In the present system, for example, any sensor different from the above-mentioned sensors may be included as part of the configuration of the vehicle sensor 10, or unnecessary sensors may be omitted from the configuration of the vehicle sensor 10. Furthermore, since the number of various sensors in the vehicle sensor 10 is arbitrary, the present system may, for example, install a plurality of cameras, microphones, etc. at arbitrary positions on the vehicle.

[0057] In the above embodiment (particularly the embodiment in FIG. 4), the influence and priority of the system are determined based on the criteria summarized in the table, but this is not limited to the above. The system may calculate the distance from the vehicle, the presence or absence in the vehicle's traveling direction, etc., and calculate the influence and determine the priority according to the results.

[0058] Although the above-mentioned embodiment (particularly the embodiment in FIG. 5) has been described simply, the image verbalization information of this system may include advice for the driver of the vehicle to follow the road traffic law, advice for safe driving, etc. (collectively referred to as "traffic safety information"). For example, this system may output, as part of the verbalization information, directions of travel based on arrow lights (arrow signals), traffic volume of other vehicles, predicted behavior of other vehicles within the recognition range based on traffic volume, difficulty of entering a parking lot, etc. Specifically, for example, in the example of the above-described embodiment (particularly the embodiment of FIG. 5), the system may take into account the traffic safety information and additionally output a sentence such as, "The traffic light indicates that you are permitted to move forward and turn left, so please continue moving forward. There is relatively heavy traffic in the surrounding area, and a truck in the right lane may be changing lanes. In addition, there is a parking lot ahead and to the left, but there is no route to enter, making it difficult to enter."

[0059] In addition, although not explained in the above embodiment, the present system may have functions such as not only outputting image-verbalized information but also answering (responding) to questions from the driver regarding route conditions, etc., of the navigation system.

[0060] Although the above-described embodiment (particularly the embodiment of FIG. 4) has been merely a simplified explanation, the present system may employ a specification that places no restriction on the order in which events or situation changes of the same priority are output when there are multiple events or situation changes of the same priority.

[0061] Conversely, the system may employ a specification that, when there are multiple events or status changes with the same priority, order restrictions may be imposed on the order in which events or status changes with the same priority are output.

[0062] Also, although not explained in the above embodiment, a vehicle changing lanes may include, for example, a vehicle that is not present within a specified distance ahead of the vehicle, a vehicle that will not be present in the future, or a vehicle that is changing lanes. In addition, the system may, for example, treat a vehicle that is already within a specified distance ahead of the vehicle as having the same priority as a vehicle changing lanes if it is predicted that the vehicle's operation will be affected, or may treat the vehicle as having the same priority as a vehicle simply ahead if it is predicted that the vehicle's operation will not be affected.

[0063] Although not described in the above embodiment, the present system may have a function of not outputting image verbalization information having similar content to the content once output as image verbalization information for a predetermined period of time. Specifically, for example, when there are targets or environments that are reflected in the imaging range for a certain period of time (several seconds to several tens of seconds), such as a school zone or a construction area, there is a possibility that similar content will be output as image verbalization information multiple times. In such a case, the present system may limit the output of image verbalization information having similar content so that image verbalization information having similar content is output only once within a predetermined period of time.

[0064] Although not described in the above embodiment, the system may set a character limit for the image verbalization information to be output. The system may also have a specification for restricting output from information with a low priority when it is determined that the image verbalization information to be output exceeds the character limit.

[0065] In the above-mentioned embodiment (particularly the embodiment in FIG. 4), the object, event, or situation is classified into three categories: high priority, low priority, and no need for verbalization. However, this is not limited to this. For example, the present system may adopt three categories, for example, high priority, medium priority, and low priority, as classifications of the object, event, or situation, or may adopt any other classification, or may omit some of the classifications. The types of classifications and the contents of the criteria for each classification shown in FIG. 4 are merely examples and are not limited to this.

[0066] Furthermore, the above-described series of processes can be executed by hardware or software. In other words, the functional configurations in FIG. 3 and the like are merely examples and are not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not limited to the example in Fig. 3, etc. Furthermore, the location of the functional block is not limited to the example in Fig. 3, etc., and may be arbitrary. Furthermore, one functional block may be configured as a single piece of hardware, may be configured as a single piece of software, or may be configured as a combination of both.

[0067] Furthermore, the number of different hardware components constituting this system and the number of users are optional, and the system may also include other hardware components.

[0068] Furthermore, when the series of processes is executed by software, the programs constituting the software are installed into a computer or the like from a network or a recording medium.

[0069] The computer may be a computer that is built into dedicated hardware, or a computer that is capable of executing various functions by installing various programs.

[0070] Furthermore, the recording medium containing such a program may not only be constituted by a removable medium (not shown) provided separately from the device main body in order to provide the program to a user, etc., but may also be constituted by a recording medium that is provided to the user in a state in which it is pre-installed in the device main body.

[0071] In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices or a plurality of means.

[0072] Even when these other embodiments are adopted, the effects of the present embodiment are exhibited. In addition, the present embodiment can be appropriately combined with other embodiments, and other embodiments can be appropriately combined with each other.

[0073] In summary, the information processing system to which the present invention is applied can take various forms having the following configurations. An acquisition unit (e.g., a camera information acquisition unit 80) that acquires information related to an image captured of an external environment of the moving object; an output unit (e.g., a language conversion unit 81) that converts at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputs the converted change; Any information processing device having the above will suffice. [Explanation of symbols]

[0074] S Vehicle System 1 In-vehicle device 80 Camera information acquisition unit 81 Language Conversion Department 100 Priority determination section 82 Output control section 10 Vehicle Sensors 20 Human Machine Interfaces 30 Control ECU 41 Control section 2 Server

Claims

1. an acquisition unit that acquires information regarding an image of an external environment of a moving object; an output unit that converts at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputs the converted change; An information processing device comprising:

2. the output unit determines a priority for outputting the language according to a degree of influence that may be exerted on the moving object. The information processing device according to claim 1 .

3. The output unit determines the priority to be high when the effect of the event or the change in the situation to be converted on the moving body within a predetermined time from the current time is predicted to be high, and determines the priority to be low when the effect of the event or the change in the situation to be converted is predicted to be low. The information processing device according to claim 2 .

4. When there are a plurality of events or changes in the situation determined to have different priorities, the output unit outputs the event or changes in the situation in the language based on the priorities. The information processing device according to claim 3 .

5. when there are a plurality of events or changes in the situation having the same priority, the output unit does not impose any restriction on the order of the events or changes in the situation having the same priority. The information processing device according to claim 4.

6. When a predetermined character limit is set for the output of the language, and when it is determined that the language to be output will exceed the set character limit, the output unit does not output the language in order of the lowest priority. The information processing device according to claim 4.

7. when it is determined that the number of characters will exceed the set character limit, the output unit does not output, as the language, those having a low priority and having a low degree of influence on the moving object. The information processing device according to claim 6.

8. The output unit determines the priority to be low when privacy information is included in the event or the change in the situation that is the subject of the conversion. The information processing device according to claim 2 .

9. The output unit outputs traffic safety information to a user of the mobile object. The information processing device according to claim 1 .

10. when the output unit outputs a predetermined content in the language, the output unit restricts output of content similar to the content once output for a predetermined period of time. The information processing device according to claim 1 .

11. A computer-implemented information processing method An acquisition step of acquiring information regarding an image of an external environment of the moving body; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; An information processing method comprising:

12. On the computer, An acquisition step of acquiring information regarding an image of an external environment of the moving body; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; A program that executes control processing including:

Images