Information processing device and program

Abstract

To provide a technology capable of presenting contents of image information in a more intuitive and easy-to-understand manner.SOLUTION: A camera information acquisition unit 80 acquires image information acquired by sensors constituting a vehicle sensor 10, particularly cameras (front, side, and rear). A language conversion unit 81 generates image verbalization information indicating changes in targets, events, and situations that may affect movements of the vehicle based on the image information acquired by the camera information acquisition unit 80 to output the image verbalization information. An output control unit 82 performs control to output various information including text information or audio information to a HMI 20, or the like.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device program. [Background technology]

[0002] Autonomous driving technology has been attracting attention in recent years. Image recognition technology is also important for realizing autonomous driving. For example, Patent Document 1 proposes a technology that outputs the recognition results of image information as text, allowing the user to grasp the results intuitively (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2015-184798 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned Patent Document 1 merely outputs text for general-purpose image information, and the content of the text to be output is basically only simple content that is intended to be generated based on rules.

[0005] The present invention has been made in view of the above circumstances, and aims to provide a technique that can present the contents of image information in a more intuitive and easy-to-understand manner.

[0006] In order to achieve the above object, a first aspect of the present invention is an acquisition unit that acquires information about an image of an external environment of the moving object; an output unit that converts at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputs the converted change; The information processing device is provided with:

[0007] In addition, in the first aspect, the output unit can determine a priority for outputting the language according to the degree of influence that may be exerted on the moving object.

[0008] In addition, in the first aspect, the output unit can determine the priority to be high if it is predicted that the event or change in the situation that is the subject of the conversion will have a high impact on the moving body within a predetermined time from the current time, and can determine the priority to be low if it is predicted that the impact will be low.

[0009] Also, in the first aspect, when there are a plurality of events or changes in the situation determined to have different priorities, the output unit can output the information in the language based on the priorities.

[0010] Also, in the first aspect, when there are multiple events or changes in the situation with the same priority, the output unit may not impose any order restrictions on the order of the events or changes in the situation with the same priority.

[0011] Also, in the first aspect, when a predetermined character limit is set for the output of the language and it is determined that the language to be output will exceed the set character limit, the output unit can not output the language in order of lowest priority.

[0012] Also, in the first aspect, when it is determined that the set character limit will be exceeded, the output unit can not output as the language those with lower priority that have a lower impact on the moving body.

[0013] Furthermore, in the first aspect, when privacy information is included in the event or the change in the situation that is the subject of the conversion, the output unit can determine that the priority is low.

[0014] In addition, in the first aspect, the output unit can output traffic safety information to a user of the moving object.

[0015] Also, in the first aspect, when the output unit outputs a specified content in the language, it can restrict the output of content similar to the content that has been output once from being output for a specified period of time.

[0016] Moreover, a second aspect of the present invention is A computer-implemented information processing method an acquisition step of acquiring information about an image of an external environment of the moving object; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; The information processing method includes:

[0017] Moreover, a third aspect of the present invention is On the computer, an acquisition step of acquiring information about an image of an external environment of the moving object; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; It is a program that executes control processing including the above.

[0018] An information processing method and a program according to one aspect of the present invention are also provided as an information processing method or a program corresponding to the information processing device according to one aspect of the present invention. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide a technique that can present the contents of image information in a more intuitive and easy-to-understand manner. [Brief explanation of the drawings]

[0020] [Figure 1]1 is a diagram illustrating an example of an information processing system including a vehicle system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of a hardware configuration of an in-vehicle device that configures an information processing system according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a diagram illustrating an example of a functional configuration of an in-vehicle device that configures an information processing system according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of criteria for determining priority by an information forensic processing system according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a diagram showing an example of a specific method for verbalization performed by an information legal processing system according to an embodiment of the present disclosure. [Figure 6] FIG. 2 is a diagram illustrating an example of the flow of various processes executed by an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0021] [Embodiment] FIG. 1 is a diagram illustrating an example of an information processing system (hereinafter referred to as "this system") including a vehicle system according to an embodiment of the present disclosure.

[0022] As shown in FIG. 1, a vehicle system S according to an embodiment of the present disclosure includes an in-vehicle device 1, a vehicle sensor 10, and an HMI (Human Machine Interface) 20. and a control ECU 30 (Electronic Control Unit). These devices and equipment are connected to each other via a predetermined network such as a CAN (Controller Area Network) or Ethernet. Here, the vehicle on which the vehicle system S is mounted may include any vehicle, moving body, etc., including, for example, an automobile powered by electricity or gasoline.

[0023] The vehicle sensor 10 is configured by various sensors for detecting the external environment around the vehicle (an environment that may include other vehicles, pedestrians, structures, road shapes, etc.). Here, the external environment around the vehicle may include, for example, traffic participants (other vehicles, pedestrians, etc.), buildings such as commercial facilities, traffic signs installed on the side of the road, road markings formed on the road surface, dividing lines, traffic lights, utility poles, guardrails, animals, fallen objects, etc. Furthermore, the external environment around the vehicle may also include, for example, information about the weather and the road surface (roads, sidewalks, etc.) on which the mobile object can move and its condition (the road surface is wet, uneven, etc.). Specifically, as shown in FIG. 1, the vehicle sensor 10 includes a camera (front camera) that is installed to be able to capture images in front of the vehicle, a camera (side camera) that is installed to be able to capture images on the sides of the vehicle, a camera (rear camera) that is installed to be able to capture images behind the vehicle, millimeter-wave radar, ultrasonic radar, LiDAR (Light Detection And Ranging), an acceleration sensor, GNSS (Global Navigation Satellite System), an external microphone (sound collection device), on-board instruments, etc.

[0024] Here, the camera is configured, for example, with a camera using a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc. In the present disclosure, a total of multiple cameras may be installed in the front, sides, and rear of the vehicle.

[0025] In addition, the external microphone (sound collection device) is composed of a general-purpose microphone, etc., and is used to acquire information about sounds emitted by objects outside the vehicle, including, for example, sirens of ambulances and police cars, human voices, etc.

[0026] The HMI 20 presents various information to the driver and passengers of the vehicle and accepts various input operations. Specifically, as shown in Figure 1, the HMI 20 includes, for example, a display, operation buttons, a microphone, various navigation systems, a speaker, etc.

[0027] The control ECU 30 is connected to the in-vehicle device 1, transmits and receives various information, and executes various controls related to vehicle operation. Specifically, as shown in Fig. 1, the control ECU includes individual ECUs that execute various controls, such as brake control, accelerator control, steering control, lights such as turn signals and lights, a power unit, a transmission, and a suspension.

[0028] As shown in FIG. 1, the system may include a vehicle system S (on-board device 1) managed by a driver of the vehicle or the like, and a server 2 managed by an administrator of the system or the like. The vehicle system S and the server 2 may be connected to each other via a predetermined network N such as the Internet. However, the network N is not an essential component, and other networks such as NFC (Near Field Communication), Bluetooth (registered trademark), and LAN (Local Area Network) may also be used. The server 2 acquires various types of information relating to vehicle operation that are periodically transmitted from the vehicle system S (particularly the in-vehicle device 1), and is used to manage the various types of information. In the following description, when simply referring to an "image," it broadly includes "moving image," "still image," "time-series image," and the like.

[0029] FIG. 2 is a diagram illustrating an example of a hardware configuration of an in-vehicle device that constitutes an information processing system according to an embodiment of the present disclosure.

[0030] As shown in FIG. 2, the in-vehicle device 1 includes a control unit 41, a ROM (Read Only Memory) 42, a RAM (Random Access Memory) 43, a bus 44, an input / output interface 45, a storage unit 46, and a communication unit 47.

[0031] The control unit 41 is configured by a microcomputer or the like including a CPU, a GPU, an FPGA (Field-Programmable Gate Array), a semiconductor memory, etc. The control unit 41 executes various processes according to a program recorded in a ROM 42 or a program loaded from a storage unit 46 to a RAM 43. The RAM 43 also stores information necessary for the control unit 41 to execute various processes as needed.

[0032] The control unit 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output interface 45 is also connected to this bus 44. The input / output interface 45 is connected to the vehicle sensors 10, the HMI 20, the control ECU 30, a storage unit 46, a communication unit 47, and the like.

[0033] The storage unit 46 is configured with a hard disk drive (HDD), a solid state drive (SSD), etc., and stores various information. For example, the storage unit 46 stores various programs and the like required for executing various processes related to this system.

[0034] The communication unit 47 controls mutual communications with other hardware and the like via a network N including the Internet.

[0035] The hardware configuration of the server 2 is not described here because it can be basically the same as the hardware configuration of the in-vehicle device 1. Cooperation between such various hardware and software enables the in-vehicle device 1 and the like to execute various processes described below.

[0036] FIG. 3 is a diagram illustrating an example of a functional configuration of an in-vehicle device that constitutes an information processing system according to an embodiment of the present disclosure.

[0037] As shown in FIG. 3, in the control unit 41 of the in-car device 1, a camera information acquisition unit 80, a language conversion unit 81, and an output control unit 82 function by executing various programs. Furthermore, a model information DB 300 is provided in one area of ​​the storage unit 46 of the in-vehicle device 1. The model information DB 300 stores a trained model (a program or the like that defines the results of training that has undergone statistical processing using the in-vehicle device 1 or other large-scale language models, etc.) for generating image verbalization information, which will be described later, based on image information, which will be described later. Note that the training method used here may be, for example, a method classified into various deep learning methods such as DNN (Deep Neural Network), or a combination thereof.

[0038] The camera information acquisition unit 80 acquires information related to images of the external environment of the moving body. Specifically, the camera information acquisition unit 80 acquires various information (hereinafter referred to as "image information") related to images captured by the sensors constituting the vehicle sensor 10, particularly the cameras (front, side, rear).

[0039] The language conversion unit 81 converts at least one change in a target, event, or situation that may affect the movement of the moving body into a language that can be recognized by humans and outputs the converted information. Specifically, the language conversion unit 81 generates a text sentence (hereinafter referred to as "image verbalization information") indicating a change in a target, event, or situation that may affect the movement of the vehicle based on the image information acquired by the camera information acquisition unit 80, and outputs the image verbalization information. The language conversion unit 81 determines the order in which the generated image verbalization information is output according to a priority, which will be described later. Here, the targets typically include, for example, vehicles other than the own vehicle, pedestrians, road signs, billboards, buildings such as commercial facilities, road markings formed on the road surface, dividing lines, traffic lights, utility poles, guardrails, fallen objects, etc. Specific examples of language conversion by the language conversion unit 81 will be described later with reference to Figs. 4 and 5.

[0040] Here, the language conversion unit 81 is provided with a priority determination unit 100 . The priority determination unit 100 determines the priority of the output according to the degree of influence that the output may have on the moving body. Specifically, the priority determination unit 100 determines the degree of influence that each target, event, and situation change may have on the movement of the vehicle based on the content of the image information acquired by the camera information acquisition unit 80, and determines the priority for verbalizing each target, event, and situation change according to the degree of influence. For example, the system determines a high priority if a target, event, or change in situation is predicted to have a high impact on the vehicle within a specified time from the current time, and determines a low priority if the impact is predicted to be low.

[0041] The output control unit 82 executes control for outputting various information including text information, audio information, etc. to the HMI 20, etc.

[0042] FIG. 4 is a diagram illustrating an example of criteria for determining priority by an information legal processing system according to an embodiment of the present disclosure.

[0043] In the example of Figure 4, a specific example of the method by which the present system determines priorities (calculates impact levels) is displayed as a table for each target, event, or situation. For example, in the example of Figure 4, examples of targets, events, or situations with a high priority for verbalization, i.e., with a high impact on the vehicle, are displayed, including: (Other vehicles) "Vehicles approaching from the front, motorcycles and bicycles traveling to the side," (Pedestrians) "Pedestrians crossing or waiting to cross," (Traffic signs, etc.) "Changes in traffic light status, stop signs, no-entry signs," (Accidents, congestion, restrictions, and construction) "Accidents or congestion ahead," (Road surface conditions and conditions) "Fallen objects in the path," (Road structure) "Intersections, sharp curves," and (Other) "Sudden changes in conditions ahead that pose a high risk to the vehicle." When such targets, events, or situations are included in the acquired image information, the present system prioritizes verbalization of those targets, events, or situations. On the other hand, in the example of Figure 4, examples of targets, events, or situations that have a low priority for verbalization, that is, that have a low impact on the vehicle, are displayed, such as (other vehicles) "vehicles changing lanes," (traffic signs, etc.) "signs and speed limits on expressways and trunk roads," (accidents, congestion, restrictions, and construction) "school zones, road construction, restrictions," (road conditions and status) "wet or icy roads, etc.", and (other) "information that can normally be obtained by a navigation system, information with low urgency, etc." If such targets, events, or situations are included in the acquired image information, the system does not prioritize verbalizing the targets, events, or situations (it will verbalize them if there are no other targets, events, or situations with a high priority for verbalization). Furthermore, in the example of FIG. 4, examples of objects, events, or situations that do not need to be verbalized are displayed, such as (other vehicles) "vehicles that do not affect the path, vehicle exteriors, designs, and license plates," (pedestrians) "pedestrians that do not affect the vehicle (e.g., not crossing the road)," (non-traffic-related objects) "trees, roadside buildings, advertising billboards," and (other) "information that does not affect operation, private information, etc." If such objects, events, or situations are included in the acquired image information, the system may not, in principle, verbalize the objects, events, or situations. For example, the system may determine the degree of impact that the object may have on vehicle movement according to such criteria and determine the priority of verbalization. Note that information related to the criteria shown in FIG. 4 is stored in the memory unit 46 or the like in any format. Note that privacy information is not limited to so-called personal information, and may broadly include, for example, information that can identify an individual, information that would be considered unpleasant if explicitly depicted, etc. Specifically, in the example of this embodiment, the privacy information includes the faces and physical features of people (pedestrians, vehicle passengers, etc.), the interiors of other vehicles, buildings, etc. To summarize the above, when there are multiple events or changes in circumstances that have been determined to have different priorities, the system outputs the images as verbalized information based on the determined priorities.

[0044] FIG. 5 is a diagram showing an example of a specific method for performing verbalization by the information law processing system according to an embodiment of the present disclosure.

[0045] In the example of FIG. 5, an example of an image that is to be verbalized by this system is specifically displayed. Looking at the example in Figure 5, for example, there is an intersection ahead of the vehicle. Such an object, event, or situation corresponds to the high priority content (intersection) in the judgment criteria in the example in Figure 4. Looking at the example in Figure 5, for example, a traffic light installed ahead of the vehicle indicates that the vehicle is only permitted to move forward and turn left. Such an object, event, or situation corresponds to the high priority content (change in traffic light status) in the judgment criteria in the example in Figure 4. For example, this system verbalizes a sentence such as "There is an intersection ahead of the vehicle, and the traffic light ahead indicates that moving forward and turning left is permitted" and outputs it with a high priority. On the other hand, in the example of Figure 5, a large truck is traveling ahead of the vehicle on the right side, and depending on the situation, this large truck may change lanes into the lane in which the vehicle is traveling. Such an object, event, or situation corresponds to, for example, a low priority content (vehicle changing lanes) in the judgment criteria of the example of Figure 4. For example, this system verbalizes a sentence such as "There is a possibility that a truck in the right lane will change lanes" and outputs it with a low priority. Note that, for example, information such as the design of the vehicle or the license plate is not output as a sentence because it is not important or necessary to verbalize it to the vehicle occupants.

[0046] FIG. 6 is a diagram illustrating an example of the flow of various processes executed by an in-vehicle device constituting an information processing system according to an embodiment of the present disclosure.

[0047] In step S1, the camera information acquisition unit 80 acquires image information acquired by the sensors constituting the vehicle sensor 10, particularly the cameras (front, side, rear).

[0048] In step S2, the priority determination unit 100 calculates the degree of influence that each target, event, and change in situation may have on the movement of the vehicle, based on the content of the image information acquired by the camera information acquisition unit 80.

[0049] In step S3, the priority determination unit 100 determines the priority for verbalizing each target, event, and change in situation according to the calculated degree of influence.

[0050] In step S4, the language conversion unit 81 generates image language information indicating targets, events, and changes in the situation that may affect the movement of the vehicle, based on the image information acquired by the camera information acquisition unit 80. In step S4, the language conversion unit 81 outputs the generated image verbalization information in accordance with the priority determined by the priority determination unit 100. This completes the image verbalization process of the in-car device 1.

[0051] The above describes one embodiment of the present disclosure, but the present disclosure is not limited to the above-described embodiment, and modifications and improvements within the scope of achieving the object of the present disclosure are included in the present disclosure.

[0052] [Other embodiments] In the above embodiment, the image verbalization information is described as being output as text information, but this is not limited thereto, and the image verbalization information may be output in other formats such as audio.

[0053] In the above-described embodiment, the vehicle is described as a general-purpose autonomous vehicle, but this is not limited thereto. Vehicles to which the present system can be applied may include any type of moving body, regardless of shape or power source, such as automobiles, trucks, motorcycles, railroad cars, and bicycles. Furthermore, the information processing device or information processing system according to the present system does not need to function independently as an information processing device, and may be provided as an integrated part with a vehicle (mobile body), for example.

[0054] Although the above-described embodiment has been merely a simplified explanation, it is assumed that the information stored in the model information DB 300 has undergone learning processing in advance by the in-vehicle device 1 or other hardware, etc. Specifically, the information stored in the model information DB 300 is model information that has been obtained by extensively learning various videos, texts, etc., and has been sufficiently adjusted for application to autonomous driving. Furthermore, in particular, in the advance learning of the model information DB 300, learning may be performed using a general-purpose large language model such as ChatGPT or BERT (Bidirectional Encoder Representations from Transformers).

[0055] Although the above-described embodiment has been merely a simplified explanation, the present system can perform reinforcement learning and update various types of model information. Specifically, for example, the present system can also update the contents of the models in the model information DB 300 using the generated image verbalization information, etc. In this case, for example, the present system may learn using other hardware, etc., based on various types of information acquired by the in-vehicle device 1, etc., and acquire the newly generated trained model via the Internet, etc.

[0056] Although the above-described embodiment has been merely a simplified explanation, the types and number of various sensors included in the vehicle sensor 10 are at the discretion of the administrator of the system, etc. In the present system, for example, any sensor different from the above-described sensors may be included as part of the configuration of the vehicle sensor 10, or unnecessary sensors may be omitted from the configuration of the vehicle sensor 10. Furthermore, since the number of various sensors in the vehicle sensor 10 is also arbitrary, the present system may, for example, install a plurality of cameras, microphones, etc. at arbitrary positions on the vehicle.

[0057] In the above-described embodiment (particularly the embodiment in FIG. 4), the influence and priority of the system are determined based on criteria summarized in a table, but this is not limited to this. For example, the system may calculate the distance from the vehicle, the presence or absence of an obstacle in the direction of travel of the vehicle, etc., and calculate the influence and determine the priority based on the results.

[0058] Furthermore, although the above-described embodiments (particularly the embodiment in FIG. 5) have been described simply, the image verbalization information of the present system may include advice for vehicle drivers to comply with road traffic laws, advice for safe driving, etc. (collectively referred to as "traffic safety information"). The present system may output, as part of the verbalization information, for example, indications of the direction of travel based on arrow lights (arrow signals), the traffic volume of other vehicles, predicted behavior of other vehicles within the recognition range based on the traffic volume, the difficulty of entering a parking lot, etc. Specifically, for example, in the example of the above-described embodiment (particularly the embodiment of Figure 5), the present system may take into account traffic safety information and additionally output a sentence such as, "The traffic light indicates that you are permitted to move forward or turn left, so please continue moving forward. There is relatively heavy traffic in the surrounding area, and a truck in the right lane may be changing lanes. Also, there is a parking lot ahead on the left, but there is no route to enter, making it difficult to enter."

[0059] Furthermore, although not explained in the above embodiment, the present system may not only simply output image-verbalized information, but may also have functions such as answering (responding) to questions from the driver regarding route conditions, etc., of the navigation system.

[0060] Furthermore, although the above-described embodiments (particularly the embodiment of FIG. 4) have been described simply, the present system may employ a specification that does not impose any order restrictions on the order in which events or situation changes of the same priority are output when there are multiple events or situation changes of the same priority.

[0061] Conversely, the system may adopt a specification that, when there are multiple events or situation changes with the same priority, restrictions may be placed on the order in which events or situation changes with the same priority are output.

[0062] Also, although not explained in the above embodiment, a vehicle changing lanes may include, for example, a vehicle that is not present within a specified distance ahead of the vehicle, a vehicle that will not be present in the future, or a vehicle that is changing lanes. Furthermore, the system may treat a vehicle that is already within a predetermined distance ahead of the vehicle as having the same priority as a vehicle changing lanes if it is predicted that the vehicle's operation will be affected, or may treat the vehicle as having the same priority as a vehicle simply ahead if it is predicted that the vehicle will not be affected.

[0063] Furthermore, although not described in the above embodiment, the present system may have a function of not outputting image verbalization information with content similar to content that has been output once as image verbalization information for a predetermined period of time. Specifically, for example, in the case of a school zone, a construction zone, or the like, where there are objects or environments that are captured in the imaging range for a certain period of time (several seconds to several tens of seconds), there is a possibility that similar content will be output as image verbalization information multiple times. In such a case, the present system may limit the output of image verbalization information with similar content so that image verbalization information with similar content is output only once within a predetermined period of time.

[0064] Although not described in the above embodiment, the system may set a character limit for the image verbalization information to be output. The system may also have a specification that restricts output from information with a low priority when it is determined that the image verbalization information to be output exceeds the character limit.

[0065] Furthermore, in the above-described embodiment (particularly the embodiment of FIG. 4), targets, events, or situations are described as being classified into three categories: high priority, low priority, and no need for verbalization, but this is not limited thereto. For example, the present system may adopt three categories, such as high priority, medium priority, and low priority, as classifications of targets, events, or situations, or may adopt any other classification, or may omit some of the classifications. The types of classifications and the contents of the criteria for each classification shown in FIG. 4 are merely examples and are not limited thereto.

[0066] The above-described series of processes can be executed by hardware or software. In other words, the functional configurations in FIG. 3 and the like are merely examples and are not particularly limited. That is, it is sufficient that the information processing system is provided with a function that can execute the above-described series of processes as a whole, and the type of functional block used to realize this function is not limited to the example shown in Fig. 3. Furthermore, the location of the functional block is not limited to the example shown in Fig. 3, and may be arbitrary. Furthermore, one functional block may be configured by hardware alone, by software alone, or by a combination of these.

[0067] Furthermore, the number of various hardware components constituting this system and the number of users are optional, and the system may also include other hardware components.

[0068] Furthermore, when a series of processes is executed by software, the programs that make up the software are installed into a computer or the like from a network or a recording medium.

[0069] The computer may also be a computer that is built into dedicated hardware, or a computer that can execute various functions by installing various programs.

[0070] Furthermore, the recording medium containing such a program may not only be constituted by a removable medium (not shown) provided separately from the device main body in order to provide the program to the user, etc., but may also be constituted by a recording medium that is provided to the user in a state where it is pre-installed in the device main body.

[0071] In addition, in this specification, the term "system" refers to an overall device that is made up of a plurality of devices or a plurality of means.

[0072] The effects of this embodiment can be achieved even when these other embodiments are adopted. Furthermore, this embodiment can be combined with other embodiments, and other embodiments can be combined with each other as appropriate.

[0073] To sum up, the information processing system to which the present invention is applied can take various forms having the following configurations. an acquisition unit (for example, a camera information acquisition unit 80) that acquires information about an image of an external environment of the moving object; an output unit (e.g., a language conversion unit 81) that converts, based on the image, at least one change in a target, an event, or a situation that may affect the movement of the moving object into a language that can be recognized by a person and outputs the converted change; Any information processing device having the above is sufficient. [Explanation of symbols]

[0074] S Vehicle System 1 On-vehicle device 80 Camera information acquisition unit 81 Language Conversion Department 100 Priority determination section 82 Output control section 10 Vehicle Sensors 20 HMI 30 Control ECU 41 Control Unit 2 Server

Claims

1. an acquisition unit that acquires information about an image of an external environment of the moving object; an output unit that converts at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputs the converted change; Equipped with the output unit determines a priority for outputting the language in accordance with a degree of influence that may be exerted on the moving object; When a predetermined character limit is set for the output of the language and it is determined that the language to be output will exceed the set character limit, the output unit limits the output in order of the lowest priority, The output unit determines the priority to be low when privacy information is included in the event or the change in the situation that is the target of the conversion. Information processing device.

2. the output unit determines the priority to be high when it is predicted that the impact that the event or the change in the situation that is the target of the conversion will have on the moving body within a predetermined time from the current time will be high, and determines the priority to be low when it is predicted that the impact will be low. The information processing device according to claim 1 .

3. When there are a plurality of events or changes in the situation determined to have different priorities, the output unit outputs the event or change in the situation in the language based on the priorities. The information processing device according to claim 1 .

4. when the output unit outputs predetermined content in the language, the output unit restricts output of content similar to the content that has been output once for a predetermined period of time. The information processing device according to claim 1 .

5. A computer-implemented information processing method an acquisition step of acquiring information about an image of an external environment of the moving object; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; Including, the output step determines a priority for outputting the language in accordance with a degree of influence that may be exerted on the moving object; In the output step, when a predetermined character limit is set for the output of the language and it is determined that the language to be output will exceed the set character limit, the output is limited in order from the language with the lowest priority; In the output step, if privacy information is included in the event or the change in the situation that is the subject of the conversion, the priority is determined to be low. Information processing methods.

6. On the computer, an acquisition step of acquiring information about an image of an external environment of the moving object; an output step of converting at least one change in a target, an event, or a situation that may affect the movement of the moving object based on the image into a language that can be recognized by a person and outputting the converted language; Execute a process including the output step determines a priority for outputting the language in accordance with a degree of influence that may be exerted on the moving object; In the output step, when a predetermined character limit is set for the output of the language and it is determined that the language to be output will exceed the set character limit, the output is limited in order from the language with the lowest priority; In the output step, if privacy information is included in the event or the change in the situation that is the subject of the conversion, the priority is determined to be low. program.

Images