Straddled-vehicle-related output system

Abstract

The purpose of the present invention is to construct a system that can improve the well-being of a rider by effectively and efficiently strengthening a feeling of collaboration between a straddled vehicle and the rider, and to reduce the load on hardware resources constituting the system. In the system, at least one type of rider biological data acquired over time from at least one input device through a data acquisition process is used to generate estimated rider emotion prompt data that can be interpreted linguistically through a large language model (LLM) and has a reduced data volume. The estimated rider emotion prompt data is supplied to the LLM, and the linguistically-interpreted rider emotion data during travel is acquired from the LLM. Output for controlling the operation of at least one output device is implemented by using the obtained linguistically-interpreted rider emotion data during travel.

Description

Straddle vehicle related output system

[0001] The present invention relates to an output system for a straddle-type vehicle.

[0002] Patent Literature 1 discloses a processing device for a saddle-ride type vehicle. The processing device includes an input unit that receives information related to a rider's reaction, a calculation unit that learns the rider's reaction tendencies based on the information related to the rider's reaction input from the input unit and generates information to be output based on the learning results, and an output unit that transmits the information generated by the calculation unit. The processing device can learn output rules to match the rider's preferences by obtaining the rider's evaluation of the output information.

[0003] Japanese Patent No. 6737875

[0004] An object of the present invention is to build a system that can improve the rider's well-being by effectively and efficiently enhancing the sense of cooperation between the saddle-type vehicle and the rider, while also reducing the load on the hardware resources that make up the system.

[0005] In view of the above problems, the present inventors have conducted extensive research and have obtained the following findings.

[0006] The saddle-type vehicle referred to in the present invention is configured to be able to turn in a lean position. That is, the saddle-type vehicle is equipped with handlebars, and is configured so that the rider leans the vehicle body and turns by shifting his or her weight while gripping the handlebars with both hands. As a result, when the saddle-type vehicle turns, the rider leans together with the vehicle body toward the inside of the curve. In this way, when turning, the rider uses his or her entire body to move the saddle-type vehicle, and the saddle-type vehicle is configured to lean together with the rider in response to the rider's movement. Turning such a saddle-type vehicle can feel to the rider like a collaborative effort between the rider and the saddle-type vehicle. Therefore, compared to vehicles other than saddle-type vehicles that can turn in a lean position, such as ordinary four-wheeled automobiles (hereinafter also referred to as non-lean vehicles), the saddle-type vehicle is a vehicle that is more likely to create a sense of unity with the rider. The stronger the sense of unity the rider feels with the saddle-type vehicle, the more enjoyable the rider's experience of driving the saddle-type vehicle.

[0007] As such, compared to non-lean vehicles, saddle-ride vehicles are vehicles that tend to make the act of driving itself more enjoyable, and by enhancing the sense of collaboration between the saddle-ride vehicle and the rider, the rider's well-being can be improved. The sense of collaboration between the saddle-ride vehicle and the rider can be enhanced not only by the response of the saddle-ride vehicle (vehicle body) to the rider's operations, but also by the provision of information from the saddle-ride vehicle to the rider. However, while providing information from the saddle-ride vehicle to the rider can dramatically enhance the sense of collaboration if done well, there is a risk that the sense of collaboration may be significantly impaired if done poorly. Even if the saddle-ride vehicle responds well to the rider's operations, if the saddle-ride vehicle provides information to the rider poorly, the rider's impression of the saddle-ride vehicle will be negatively affected. Therefore, it is important to consider how information is provided from the saddle-ride vehicle to the rider.

[0008] In order to successfully provide information from a saddle-type vehicle to a rider, the rider's preferences, emotions, personality, and situation must be acquired in greater detail, increasing the number of factors that must be considered as input information. In addition, the output information, such as the content, transmission method, and transmission timing of the information, must also be set in greater detail, increasing the number of factors that must be considered. In other words, in order to achieve a higher level of interaction between the saddle-type vehicle and the rider, the number of factors that must be considered increases.

[0009] Furthermore, as described above, the saddle-type vehicle referred to in the present invention is configured to be able to turn in a lean position. Therefore, in addition to operating the accelerator, brakes, and steering, the rider must also shift his or her own weight to perform a turning operation in conjunction with steering. This weight shift and the resulting vehicle lean are not present in a non-lean vehicle, so compared to a non-lean vehicle, more information can be obtained from a saddle-type vehicle that can turn in a lean position. This can also be said to mean that there is more information available to achieve a higher level of interaction between the saddle-type vehicle and the rider. A notable example is a manual saddle-type vehicle in which different operations are performed with both hands and feet, as follows: Left hand: operates the clutch lever Right hand: operates the throttle (accelerator) and front brake lever Left foot: operates the gear shift pedal (however, gear shifting may also be performed using a manual lever) Right foot: operates the rear brake As the rider shifts his or her own weight, the operating parts corresponding to each of the hands and feet can be operated at different times for different vehicle operations, further increasing the amount of information that can be obtained. Saddle-type vehicles that can turn in a lean position are not limited to this example, but regardless of the type of vehicle, saddle-type vehicles that can turn in a lean position tend to have a greater variety of information that can be obtained compared to non-lean vehicles.

[0010] Furthermore, in a saddle-type vehicle capable of turning in a lean position, it is difficult to obtain much information actively input from the rider other than through driving operations while driving. Information actively input other than through driving operations tends to reflect the rider's emotions, which is important from the perspective of interaction with the saddle-type vehicle. However, in a saddle-type vehicle capable of turning in a lean position, many parts of the body (both hands, both legs, and weight shift) are required for driving operations, and many working areas of the brain are used for driving, leaving little working area available for operations other than driving, making it difficult for sudden emotional changes to occur. In addition, the installation space for operating components other than driving operations (such as plywood panels) is smaller than in a non-lean vehicle. Therefore, it is difficult for the rider to actively input a large amount and variety of information other than through driving operations while driving. As a result, in a saddle-type vehicle capable of turning in a lean position, although the types and amount of information that can be obtained are large, the types and amount of information actively input from the rider other than through driving operations tend to be limited.

[0011] As described above, in a saddle-type vehicle that can turn in a lean position, there are many factors that must be considered and a large amount of information that can be used to achieve a high level of interaction between the saddle-type vehicle and the rider. To achieve this high level of interaction between the saddle-type vehicle and the rider, it is necessary to take these factors and information into account and provide outputs that are more easily accepted by the rider in a more timely manner. This presents a particular challenge for high-level interaction between the saddle-type vehicle that can turn in a lean position and the rider.

[0012] As the above-mentioned trends of "increasing factors to be considered," "increasing amount of available information," and "higher demands for timeliness and appropriateness of output" become stronger, the load on the hardware resources required for the information output system increases, and more hardware resources are required. In addition, since a large amount of learning data needs to be collected, preparation takes a long time. The device disclosed in Patent Document 1 also has a similar problem. That is, the device disclosed in Patent Document 1 has the problem of requiring enormous hardware resources and learning data in order to achieve a high level of interaction between the saddle-riding vehicle and the rider that can enhance the sense of cooperation between the saddle-riding vehicle and the rider.

[0013] The present invention has been completed based on the above findings. The present invention can provide the following saddle-ride type vehicle-related output system.

[0014] (1) A saddle-riding type vehicle-related output system, comprising: at least one memory; at least one processor capable of communicating with an LLM (Large Scale Language Model), connected to the memory, and configured to execute at least one program stored in the memory; at least one input device provided on the saddle-riding type vehicle or on equipment or carried by a rider of the saddle-riding type vehicle, configured to acquire or generate at least one type of lidar biometric data while the saddle-riding type vehicle is traveling and supply the at least one type of lidar biometric data to the at least one processor; and at least one output device, wherein the at least one program comprises: a data acquisition process that acquires the at least one type of lidar biometric data from the at least one input device over time while the saddle-riding type vehicle is traveling; and a prompt generation process that uses the at least one type of lidar biometric data acquired over time by the data acquisition process to generate lidar emotion estimation prompt data that can be linguistically interpreted by the LLM, is related to lidar emotion estimation, and has a reduced amount of data. a linguistic interpretation process that supplies the rider emotion estimation prompt data to the LLM and acquires from the LLM, rider emotion linguistic interpretation data during travel generated by linguistic interpretation by the LLM; and an output process that uses the rider emotion linguistic interpretation data during travel acquired from the LLM to generate an output for controlling the operation of the at least one output device.

[0015] In the system (1), an LLM is used to provide information from the saddle-type vehicle to the rider in order to effectively and efficiently enhance the sense of cooperation between the saddle-type vehicle and the rider. Additionally, the system (1) has a configuration suitable for using an LLM to provide information from the saddle-type vehicle to the rider. That is, in the system (1), linguistic interpretation by the LLM is possible using at least one type of lidar biometric data acquired over time from at least one input device through a data acquisition process, and lidar emotion estimation prompt data related to ridder emotion estimation and having a reduced amount of data is generated. The lidar emotion estimation prompt data is supplied to the LLM, and riding-time lidar emotion linguistic interpretation data is acquired from the LLM. Furthermore, in the system (1), the operation of a device is controlled using the acquired riding-time lidar emotion linguistic interpretation data. This effectively and efficiently realizes high-level interaction between the saddle-type vehicle and the rider. As a result, the system (1) effectively and efficiently enhances the sense of collaboration between the saddle-type vehicle and the rider, thereby improving the rider's well-being and reducing the load on hardware resources. In other words, assuming the same hardware resources are used, more advanced processing is possible, thereby realizing a higher level of interaction. The system (1) can address at a high level the "increase in factors to be considered," "increase in available information," and "increased demands for timeliness and appropriateness of output" while reducing the load on hardware resources, thereby resolving the aforementioned challenges specific to high-level interaction between a saddle-type vehicle capable of turning in a lean position and the rider. From the perspective of achieving the above-mentioned effect (achieving both "reduced load on hardware resources" and "high-level interaction"), generating linguistically interpretable rider emotion estimation prompt data and supplying it to the LLM while reducing the amount of rider biometric data acquired over time has significant technical significance.

[0016] The saddle-riding-type vehicle-related output system includes at least one memory, at least one processor, at least one input device, and at least one output device. These devices are communicatively connected to each other. These devices may be provided in a single device to form a centralized processing system, or may be provided in multiple devices communicatively connected to each other to form a distributed processing system. The at least one input device is provided in the saddle-riding-type vehicle or in equipment or equipment carried by the rider. The at least one memory may be provided in the saddle-riding-type vehicle, in equipment or equipment carried by the rider, a user terminal, or a server. The at least one processor may be provided in the saddle-riding-type vehicle, in equipment or equipment carried by the rider, a user terminal, or a server. The at least one output device may be provided in the saddle-riding-type vehicle, in equipment or equipment carried by the rider, or a user terminal. The saddle-riding-type vehicle-related output system according to the present invention can employ, for example, any of the system configurations (a) to (x) shown in FIGS. 2 and 3 . FIG. 2 shows a device in which an input device (I), a processor (P), a memory (M), and an output device (O) are provided in the system configuration. FIG. 3 shows a device in which an input device, a processor, and an output device are provided. For ease of understanding, FIGS. 2 and 3 use the same reference numerals as those used in the embodiments described below. However, these reference numerals are not intended to limit the scope of the present invention. In FIG. 2, the memory is provided in the device in which the processor is provided. In FIG. 3, the memory is also provided in the device in which the processor is provided, although not shown. However, the present invention is not limited to this example. The memory may be provided in a device different from the device in which the processor is provided (e.g., a device in which the input device and / or output device is provided). Furthermore, there may be multiple input devices, output devices, memories, and processors. Therefore, if a single system has multiple devices, the single system may satisfy the requirements of multiple system configurations (a) to (x) shown in FIGS. 2 and 3. An example of a system configuration will now be described.As a specific example, a smartwatch corresponding to a rider's personal item may include at least one memory, at least one processor, at least one input device, and at least one output device, and therefore the smartwatch may constitute the system (see FIGS. 2(n) and 3(n)). As another specific example, the smartwatch may correspond to at least one input device, a meter of the saddle-riding type vehicle may correspond to at least one output device, and a server capable of communicating with the saddle-riding type vehicle may include at least one memory and at least one processor (see FIGS. 2(l) and 3(l)). As yet another specific example, a sensor provided on the saddle-riding type vehicle may correspond to at least one input device, a meter of the saddle-riding type vehicle may correspond to at least one output device, and a server capable of communicating with the saddle-riding type vehicle may include at least one memory and at least one processor (see FIGS. 2(d) and 3(d)). In a first example, the smartwatch corresponds to a saddle-type vehicle-related IPG (Interactive Prompt Generator) that generates rider emotion estimation prompt data to be supplied to the LLM and acquires rider emotion linguistic interpretation data during travel from the LLM. In the second and third examples, the server corresponds to the saddle-type vehicle-related IPG. In a saddle-type vehicle-related output system, all processes may be executed by a single processor, or each process may be executed by a different processor. The system may include: at least one input device and at least one output device, respectively, provided on the saddle-type vehicle or on equipment or carried by the rider; and at least one processor and at least one memory, respectively, provided on the saddle-type vehicle, on equipment or carried by the rider, on a server, or on a user terminal (see (a) to (p) of FIGS. 2 and 3 ). In this case, since the output device is provided on the saddle-type vehicle or on equipment or carried by the rider, output according to the rider's emotions can be performed at appropriate times while the saddle-type vehicle is traveling.The system may include at least one input device provided in the saddle-type vehicle or in equipment or a portable item of the rider, and at least one processor and at least one memory provided in the saddle-type vehicle, equipment or a portable item of the rider, a server, or a user terminal, respectively ((q) to (x) in FIGS. 2 and 3). Note that the hardware configuration of the saddle-type vehicle-related output system is not particularly limited.

[0017] As shown in FIGS. 2 and 3 , the output device is provided in the saddle-riding type vehicle, the rider's equipment or portable equipment, or a user terminal. For example, if the output device is the rider's equipment or portable equipment, the rider's equipment or portable equipment may be detachably attached in a position that is easily visible to the rider riding the saddle-riding type vehicle (for example, near the meter of the saddle-riding type vehicle). The output device itself may have a function for communicating with the outside via a network. The output device may be configured to be connected to a control device included in the device in which the output device itself is provided (the saddle-riding type vehicle, the rider's equipment or portable equipment, or the user terminal) and to be controlled by the control device. In this case, the control device may have a function for communicating with the outside via a network. The output device is not particularly limited, and it is desirable that the output device be configured to operate in response to commands input via communication from an outside source while the saddle-riding type vehicle is traveling, without being related to the traveling of the saddle-riding type vehicle. Specific examples of the output device include the following. - Audio system (ON / OFF, volume adjustment, song selection) - FM / AM radio (ON / OFF, volume adjustment, channel selection) - Satellite radio (ON / OFF, volume adjustment, channel selection) - Navigation system (ON / OFF, menu selection, etc.) - Action camera (ON / OFF, start / end of recording, effect selection, etc.) - Heated grips (ON / OFF, temperature adjustment) - Heated seats (ON / OFF, temperature adjustment) - Advanced driver assistance system (ADAS) (turning various functions ON / OFF) ADAS includes, for example, the following:・Quick shift system ・Electronically controlled suspension ・Power delivery control ・Wheelie control ・Front and rear linked brake system ・Brake control based on roll angle ・Engine brake control ・Automatic shift change system ・Anti-lock brake system ・Traction control system ・Slide control system ・Cruise control system ・Sudden acceleration control system ・Setting and changing driving modes

[0018] Furthermore, examples of output devices include CHL (Communicative Human Language) information output devices, which output CHL information visually or audibly. CHL information output devices are not particularly limited, and examples thereof include the following devices:・Meter (instrument panel) (presents visual output information) ・HMD (Head-Mounted Display) (presents visual output information) ・Speaker (presents auditory output information) ・Intercom (headset) (presents auditory output information) ・Mirror (presents visual output information using lights, etc.) ・Road surface illumination lights (presents visual output information) ・Roadside displays (presents visual output information) ・HUD (Head-Up Display) combiner (presents visual output information) ・HUD screen (presents visual output information) ・Smartphone (presents visual or auditory output information) ・Smartwatch (presents visual or auditory output information) ・Smartglasses (presents visual or auditory output information) ・Earphones (presents auditory output information) ・Bone conduction earphones (presents auditory output information)

[0019] As shown in FIGS. 2 and 3 , the input device may be provided in the saddle-type vehicle, or in equipment or a portable item of the rider. The input device acquires or generates at least one type of lidar biometric data while the saddle-type vehicle is traveling. For example, the input device detects lidar biometric parameters over time while the saddle-type vehicle is traveling. As a result, the input device generates lidar biometric data over time while the saddle-type vehicle is traveling. In other words, the lidar biometric data is data indicating the lidar biometric parameters. The input device may function as an independent device and have a communication function so that it can independently output the lidar biometric data. Examples of independently functioning input devices include, but are not limited to, a chest strap or a smart watch equipped with a heart rate sensor. The chest strap or the smart watch corresponds to rider equipment. The input device may also be connected to another control device and configured to supply the detection results to the control device. The control device is not particularly limited to, but may be, for example, an ECU (Electronic Control Unit). The input device that functions in combination with the control device is not particularly limited, and examples include a heart rate sensor module provided on the handlebar grip of a saddle-type vehicle. Since riders typically wear gloves as a protector, acquiring heart rates using a grip sensor may require ingenuity depending on the heart rate acquisition method. For example, an optical heart rate sensor that can ensure light transmittance even through gloves may be employed. By using gloves with a light-transmitting portion, heart rates may be measured using the optical heart rate sensor through the portion. For example, a pressure-type heart rate sensor configured to detect finger pulses even when the rider is wearing gloves may be employed. A pressure-type heart rate sensor may be employed on the assumption that the rider uses gloves suitable for detecting finger pulses using the pressure-type heart rate sensor. In the combination of an ECU and a heart rate sensor module, the ECU itself is not an input device, but when combined with the heart rate sensor module, the ECU functions as an input device and outputs rider biometric data.In this way, the input device, either as an independent device or in combination with a control device, realizes the function of outputting LIDAR biometric data. Note that, hereinafter, unless otherwise specified, the term "input device" is used to encompass both a method of communicating with the outside world independently and a method of communicating with the outside world via a control device. Examples of input devices and LIDAR biometric data (i.e., LIDAR biometric parameters) generated by the input devices include, but are not limited to, the following: LIDAR biometric data may be acquired over time while the saddle-type vehicle is traveling. The LIDAR biometric data is used to estimate the rider's emotions while the saddle-type vehicle is traveling, and LIDAR emotion estimation prompt data is generated based on the results. The following are examples of input devices and LIDAR biometric data, but are not limited to these. Heart rate sensor: heart rate Brain wave sensor: brain waves Sweat rate sensor: sweat rate Respiration rate sensor: breathing pattern (rhythm / frequency) Body temperature sensor: body temperature Blood pressure sensor: blood pressure Facial expression camera: rider's facial expression recognition Eye camera: eye movement tracking Oxygen saturation sensor: blood oxygen level Grip pressure sensor: grip pressure Microphone: audio (tone, pitch, speed, volume).

[0020] The memory includes, but is not limited to, RAM, ROM, non-volatile RAM, PROM, EPROM, EEPROM, flash memory, magnetic data storage, optical data storage, registers, or a combination thereof. There may be one or more memories.

[0021] Processors include central processing units (CPUs), microprocessors, general-purpose processors, digital signal processors (DSPs), graphics processing units (GPUs), controllers, microcontrollers, programmable logic devices (PLDs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. Multiple processors can be configured using any combination of these. Execution of a program by at least one processor is not limited to execution of the entire program by a single processor, but may also include execution of the program by parallel processing in a multiprocessor constructed by multiple processors. In a multiprocessor, either symmetric or asymmetric multiprocessing may be performed. A multiprocessor may be tightly coupled or loosely coupled. A processor may be a multi-core processor.

[0022] A large-scale language model (LLM) refers to a language model with a large number of parameters. The large-scale language model is capable of executing natural language processing tasks. The large-scale language model may be a natural language generation model. The natural language generation model is based on the large-scale language model and is capable of creating a sentence consisting of new CHL information from input CHL information. Note that creating a sentence consisting of new CHL information from input CHL information is an example of a natural language processing task. The number of parameters in the large-scale language model may be, for example, 1 billion or more, 10 billion or more, or 100 billion or more. A language model refers to a model in which CHL is modeled using word occurrence probability. In one embodiment, the large-scale language model is a language model capable of performing inference using techniques such as zero-shot learning (ZSL), one-shot learning (OSL), or few-shot learning (FSL) without fine-tuning. In one embodiment, the large-scale language model is configured to execute a task and produce output in response to an input prompt. The prompt includes CHL information. The large-scale language model is not particularly limited, and examples thereof include GPT-3, GPT-4, GShad, Switch Transformer, Gopher, HyperCLOVA, etc. The large-scale language model is not included in the saddle-riding type vehicle-related output system, but can communicate with the saddle-riding type vehicle-related output system.

[0023] CHL information is linguistic information that can be understood, recognized, and memorized by humans. CHL information is based on the linguistic system used by humans in everyday conversation. CHL information is transmitted visually or audibly as text information or audio information.

[0024] A saddle-type vehicle is a vehicle equipped with a saddle-type seat. A saddle-type vehicle is a vehicle configured so that a rider rides the vehicle while straddling a saddle. In the present invention, a saddle-type vehicle is equipped with a handlebar and configured to be able to turn in a lean position. Saddle-type vehicles include motorcycles, three-wheeled motor vehicles having a pair of front or rear wheels (left and right), and four-wheeled motor vehicles having a pair of front and rear wheels (left and right). Examples of motorcycles include scooters, mopeds, off-road vehicles, and on-road vehicles. Saddle-type vehicles that can turn in a lean position have the property of making the rider feel a sense of unity and cooperation with the saddle-type vehicle, and therefore the saddle-type vehicle-related output system according to the present invention is suitable for saddle-type vehicles that can turn in a lean position.

[0025] Rider equipment or personal items refer to devices that are worn or carried by the rider while riding a saddle-type vehicle and are capable of communication. The equipment or personal items are moved along with the rider by the saddle-type vehicle while the saddle-type vehicle is traveling. The rider here refers to a person riding in the saddle-type vehicle. Therefore, the rider includes not only the person driving the saddle-type vehicle but also a passenger. An example of a passenger is a tandem rider. Equipment is not particularly limited, and examples include HMDs, smart glasses, intercoms (headsets), shoes, boots, gloves, helmets, clothing, earphones, and bone conduction earphones. Personal items are not particularly limited, and examples include smartphones, electronic keys, personal digital assistants (PDAs), and smart watches. It is not necessary to strictly distinguish between equipment and personal items; items that fall under either or both are considered "equipment or personal items."

[0026] A user terminal refers to a device that is not a rider's equipment or personal belongings and is used by a person who is not a rider (hereinafter also referred to as a user). However, a person may be a rider when riding a saddle-type vehicle and a user when using the user terminal without riding the saddle-type vehicle. A user terminal refers to, for example, a device used by a rider after riding the saddle-type vehicle. The user terminal can constitute the system. The user is not particularly limited, and examples include the rider after riding the saddle-type vehicle. The user terminal is not particularly limited, and examples include a terminal owned by the rider (e.g., a tablet, laptop PC, or desktop PC). When the system includes a user terminal, the rider can obtain rider emotion linguistic interpretation data during riding via the user terminal, thereby understanding the rider's emotions during riding after riding.

[0027] The server is not a rider's equipment or a personal item, nor is it a user terminal. The server is not intended to be operated by a rider. However, the server may be operated by a person for purposes other than processing related to the system (e.g., maintenance).

[0028] The "at least one program" does not necessarily have to be stored in a single memory, but may be stored separately in different memories in devices that can communicate with each other. Execution of the "at least one program" performs data acquisition processing, prompt generation processing, linguistic interpretation processing, and output processing.

[0029] The data acquisition process is a process in which at least one processor acquires at least one type of lidar biometric data over time from an input device provided on the saddle-type vehicle or on equipment or portable items of the rider while the saddle-type vehicle is traveling. The input device and the lidar biometric data are as described above. Acquisition over time refers to repeated acquisition of data over time. The specific manner of acquisition over time is not particularly limited, and may be, for example, periodic or aperiodic acquisition. Multiple types of data may be acquired over time. In this case, the acquisition timing of each data may be the same or different.

[0030] The prompt generation process is a process in which at least one processor generates lidar emotion estimation prompt data using lidar biometric data. In addition to lidar biometric data, other data may also be used. For example, data obtainable via a network or data stored in memory may be used as the other data. The network is not particularly limited and may be constructed using various wireless networks. The communication method and communication protocol are not particularly limited. Examples of networks include the Internet, an in-vehicle network (e.g., CAN), an OBD-II network, and a mobile network such as LTE / 5G. The other data may include, for example, data used as a template when generating a prompt, history data indicating past interactions, and the like. The other data may be customizable. For example, saddle-type vehicle-related data obtained via a sensor may be used as the other data. The saddle-type vehicle-related data is data indicating parameters related to the traveling state of the saddle-type vehicle and changing as the saddle-type vehicle travels. The lidar emotion estimation prompt data can be linguistically interpreted by the LLM. The LIDAR emotion estimation prompt data has a smaller data volume than the LIDAR biometric data. This means that the data volume of the generated LIDAR emotion estimation prompt data is smaller than the data volume of the LIDAR biometric data used to generate it. Examples of the LIDAR emotion estimation prompt data include the following (i) to (iii).

[0031] (i) CHL information relating to the amount of change over time in the lidar biometric parameters indicated by the lidar biometric data and the method of evaluating those parameters. Because this prompt data is CHL information, it can be directly interpreted linguistically by the LLM. Furthermore, since it also includes data on the evaluation method, evaluation can be performed. Examples of the amount of change over time in the parameters are not particularly limited and include, for example: - Changes over time in heart rate - Changes over time in brain waves - Changes over time in sweat rate - Changes over time in respiratory rate - Changes over time in body temperature - Changes over time in blood pressure - Changes over time in the rider's facial expression - Changes over time in eye movement - Changes over time in blood oxygen level - Changes over time in grip pressure - Changes over time in voice

[0032] (ii) Visual data visually showing the amount of change over time in the LIDAR biometric parameter indicated by the LIDAR biometric data, and CHL information related to the evaluation method for that parameter. The visual data is, for example, data that graphically shows the amount of change over time in the LIDAR biometric parameter. If the LLM is multimodal, the multimodal LLM can also interpret the visual data linguistically. Furthermore, since the evaluation method data is also included, evaluation can be performed. Specific examples are not particularly limited, and include, for example, the same examples as in (i) above.

[0033] (iii) CHL information relating to a reading method for reading LIDAR biometric data as a time-dependent change in LIDAR biometric parameters and a method for evaluating the parameters. LIDAR biometric data is numerical data. Therefore, the LLM cannot directly interpret the LIDAR biometric data linguistically. However, since the LLM also includes data on the reading method, the LLM can read the LIDAR biometric data based on the reading method. Furthermore, since the LLM also includes data on the evaluation method, evaluation can be performed. Specific examples are not particularly limited, and include, for example, the same examples as in (i) above.

[0034] The LIDAR emotion estimation prompt data is data for linguistically interpreting the LIDAR biometric data, and as shown in (i) to (iii) above, includes two elements: (I) how to read the LIDAR biometric data and (II) how to evaluate the read data. Element (I) is, for example, how to obtain characteristics or trends of the LIDAR biometric data. Element (II) is how to communicate the characteristics or trends to the LIDAR. Element (II) may include, for example, advice to the LIDAR. When the LIDAR emotion estimation prompt data is generated using the LIDAR biometric data, elements (I) and (II) are included in the LIDAR emotion estimation prompt data. Furthermore, another element (III) may be included. Examples of element (III) include, but are not limited to, a designation of the tone of the output (utterance), a designation of the creativity or accuracy used when generating the output (utterance), a designation of continuity or consistency with previous outputs (utterances), and a combination of at least two of these.

[0035] The linguistic interpretation process is a process in which at least one processor supplies lidar emotion estimation prompt data to the LLM and acquires lidar emotion linguistic interpretation data during driving from the LLM. The lidar emotion linguistic interpretation data during driving is data generated by linguistic interpretation by the LLM.

[0036] The output processing is a process for controlling the operation of at least one output device using the rider emotion linguistic interpretation data acquired from the LLM while the saddle-riding vehicle is traveling. The output processing is not particularly limited. Examples of the output processing include turning on and selecting a music track for the audio system, turning on and selecting a station for the FM / AM radio, turning on and selecting a station for the satellite radio, turning on and off the navigation system, starting and ending filming with the action camera, turning on and off and adjusting the temperature of the heated grips, and turning on and off and adjusting the temperature of the heated seat. As a result, music track selection for the audio system, FM / AM radio, and satellite radio can be performed according to the rider's emotions. The navigation system can also be used according to the rider's emotions. Filming with the action camera can be performed when the rider's emotions are high. The heated grips can also be turned on and off and adjusted in temperature, and the heated seat can also be turned on and off and adjusted in temperature according to the rider's emotions. Furthermore, a function corresponding to the traveling state of the saddle-riding vehicle is selected and executed from among various functions possessed by the ADAS. In short, the output processing may be processing for performing output to control the operation of at least one output device so as to execute a function corresponding to the traveling state of the saddle-riding vehicle or the rider's emotion from among a plurality of functions possessed by the saddle-riding vehicle, using linguistic interpretation data related to the saddle-riding vehicle acquired from the LLM, while the saddle-riding vehicle is traveling. Furthermore, the output processing may be processing for outputting, from at least one output device, CHL information indicating a linguistic interpretation result of the rider's emotion estimated while traveling, using the traveling rider's emotion linguistic interpretation data acquired from the LLM, while the saddle-riding vehicle is traveling. The output processing may be performed such that the traveling rider's emotion linguistic interpretation data acquired from the LLM is output directly from the output device. The traveling rider's emotion linguistic interpretation data acquired from the LLM may be modified, and the modified traveling rider's emotion linguistic interpretation data may be used to perform the output processing. This output is performed visually and / or audibly.

[0037] The present invention can further employ the following configuration.

[0038] (2) The saddle-riding type vehicle-related output system of (1), wherein the at least one program is further programmed to cause the at least one processor to execute the following in the output processing: visually or audibly output CHL information corresponding to the on-road rider emotion linguistic interpretation data by the at least one output device, and output visual information consisting of an image and / or video so as to overlap at least partially in time with the CHL information.

[0039] According to the system of (2), CHL information is output visually or audibly. CHL information and visual information are output so as to overlap in time. By using visual information in combination, the efficiency of information transmission to the rider can be improved. It is possible to increase the information transmitted to the rider and improve the impression given to the rider while suppressing the increase in output information. It is possible to achieve a higher level of interaction while suppressing the increase in load on hardware resources. The system of (2) can more effectively solve the above-mentioned unique problems.

[0040] The CHL information according to the linguistic interpretation data of the rider's emotion during driving is, for example, CHL information that indicates the result of a linguistic interpretation of the rider's emotion estimated during driving.

[0041] The visual information may be stored in at least one memory in advance. The visual information may be received from outside the saddle-riding type vehicle-related output system via a communication line such as the Internet. The visual information may be generated within the saddle-riding type vehicle-related output system. The visual information may be generated by at least one processor. The visual information may be generated by the LLM. Images are static visual information. Videos are dynamic visual information. Images may consist of or include visualizations, or may include non-visualizations such as photo images. Videos may be animations or simulations, or may be live-action captures.

[0042] (3) The saddle-riding type vehicle-related output system of (2), wherein the at least one program is further programmed to cause the at least one processor to execute a visual information generation process that generates the visual information using the at least one type of rider biological data acquired over time by the data acquisition process.

[0043] According to the system of (3), both CHL information and visual information are generated based on at least one type of LIDAR biometric data and are output so as to overlap each other in time. By increasing the correlation between the visual information and CHL information, the efficiency of information transmission to the LIDAR can be improved. While suppressing the increase in output information, the information transmitted to the LIDAR can be increased and the impression given to the LIDAR can be improved. A higher level of interaction can be achieved while suppressing the increase in load on hardware resources. The system of (3) can more effectively solve the above-mentioned unique problems.

[0044] (4) The saddle-riding type vehicle-related output system of (3), wherein the at least one program is further programmed to cause the at least one processor to execute the following in the visual information generation process: generating the visual information by the LLM using the at least one type of lidar biometric data acquired over time by the data acquisition process.

[0045] According to the system of (4), both the CHL information and the visual information are generated by the LLM based on at least one type of LIDAR biometric data and output so as to overlap each other in time. This makes it possible to improve the correlation between the visual information and the CHL information while reducing the load on hardware resources. The system of (4) can more effectively solve the above-mentioned unique problems.

[0046] (5) The saddle-riding type vehicle-related output system according to any one of (2) to (4), wherein in the output processing, the CHL information output by the at least one output device includes an explanation about or related to the visual information that is output so as to overlap in time with the CHL information.

[0047] According to the system of (5), the correlation between CHL information and visual information can be further enhanced. It is possible to have the rider recognize multiple types of highly relevant information in a timely manner. The efficiency of information transmission to the rider can be improved. It is possible to increase the information transmitted to the rider and improve the impression given to the rider while suppressing the increase in output information. It is possible to achieve a higher level of interaction while suppressing the increase in load on hardware resources. The system of (5) can more effectively solve the above-mentioned unique problems.

[0048] A description of or relating to visual information may be a description or explanation of the visual content contained in the visual information itself, or may be a description of how to view the visual content rather than the visual content contained in the visual information itself.

[0049] The present invention can further employ the following configuration: (6) The saddle-riding type vehicle-related output system according to any one of (1) to (5), including the at least one input device, the at least one output device, and a saddle-riding type vehicle-related IPG (Interactive Prompt Generator) capable of communicating with each of the at least one input device, the at least one output device, and the LLM via a network, and having the at least one memory and the at least one processor.

[0050] The system (6) includes a saddle-type vehicle-related IPG, so customization by entities such as saddle-type vehicle manufacturers can be easily realized.

[0051] The present invention can further employ the following configurations: (7) A straddle-type vehicle-related IPG included in the straddle-type vehicle-related output system of (6).

[0052] The saddle-type vehicle-related IPG (7) can be easily customized by entities such as saddle-type vehicle manufacturers.

[0053] According to the present invention, by effectively and efficiently enhancing the sense of cooperation between the saddle-type vehicle and the rider, it is possible to build a system that can improve the rider's well-being and reduce the load on the hardware resources that make up the system.

[0054] FIG. 1(a) is a system overview diagram illustrating a saddle-riding-type vehicle-related output system according to an embodiment of the present invention, and FIG. 1(b) is a flowchart illustrating processing performed by the saddle-riding-type vehicle-related output system. FIGS. 2(a) to 2(x) are diagrams illustrating devices in which an input device (I), a processor (P), a memory (M), and an output device (O) are provided in the saddle-riding-type vehicle-related output system according to the present invention. FIGS. 3(a) to 3(x) are tables illustrating devices in which an input device, a processor, and an output device are provided in the saddle-riding-type vehicle-related output system according to the present invention. FIG. 4 is a flowchart illustrating processing performed by a saddle-riding-type vehicle-related output system according to a first modified embodiment of the present invention.

[0055] FIG. 1A is a system overview diagram for explaining a straddle-type vehicle-related output system 11 according to an embodiment of the present invention.

[0056] The saddle-riding-type vehicle-related output system 11 (hereinafter simply referred to as the system 11) includes at least one input device 13, at least one output device 14, and an IPG 1 for the saddle-riding-type electric vehicle. The at least one input device 13 is provided in at least one of a saddle-riding-type vehicle 16 and a rider's equipment 17 and / or a rider's portable item 17. At least one processor 2 acquires rider biometric data from the at least one input device 13. Examples of the input device 13 illustrated in the figure include a smartwatch, a grip sensor, and an ECU. The smartwatch is configured to acquire, for example, heart rate and sweat rate. The grip sensor is configured to acquire, for example, heart rate and grip pressure. Note that, since riders typically wear gloves as protective gear, acquiring heart rate using the grip sensor may require ingenuity depending on the heart rate acquisition method, as described above. As described above, the ECU does not function as a biometric sensor on its own, but functions as an input device in combination with the grip sensor. The at least one processor 2 acquires rider biometric data from the grip sensor via the ECU. The smart watch functions as an input device by itself, but the input device 13 is not limited to this example.

[0057] The saddle-type electric vehicle IPG 1 includes at least one processor 2, at least one memory 3, and at least one communication module 4. The memory 3 stores programs for executing (A) data acquisition processing, (B) prompt generation processing, (C) linguistic interpretation processing, and (D) output processing. The at least one processor 2 executes the programs. The number of processors 2, memories 3, and communication modules 4 is not particularly limited, and each may be one or more.

[0058] The hardware configuration of the system 11 is not particularly limited. The output device 14 may also have the functions of the saddle-riding-type vehicle-related IPG 1. If the saddle-riding-type vehicle-related IPG 1 is separate from the output device 14, the saddle-riding-type vehicle-related IPG 1 may be configured by a single server 19 or multiple servers 19 connected to each other so that they can communicate with each other. In this case, the multiple servers 19 may be configured to provide cloud computing services. The communication module 4 enables communication between the processor 2 and each of the LLM (Large Scale Language Model) 6, the input device 13, and the output device 14.

[0059] Each of the multiple output devices 14 can communicate with the saddle-riding vehicle-related IPG 1 via the network 12. The number of output devices 14 is not particularly limited. At least one output device is required. The output device 14 may be provided in the saddle-riding vehicle 16, in the rider's equipment 17 or personal belongings 17, or in the user terminal 18. Examples of the output device 14 shown in the figure include a meter on the saddle-riding vehicle 16, a helmet HUD, and a smartphone. Of course, the output device 14 is not limited to these examples.

[0060] The network 12 enables communication between the multiple output devices 14, the saddle-ride type vehicle-related IPG 1, and the LLM 6. The network 12 is not particularly limited and can be constructed using various wireless networks. There are no particular limitations on the communication method or communication protocol.

[0061] The LLM 6 is stored in one server or in multiple servers that are communicatively connected to each other. The LLM 6 can communicate with the system 11. This is the same as the LLM 6 being able to communicate with the saddle-ride type vehicle-related IPG 1 in this embodiment. The LLM 6 outputs to the system 11 in response to input from the system 11. Communication between the LLM 6 and the system 11 is performed using at least CHL information.

[0062] 1B is a flowchart illustrating the processing performed by the saddle-riding-type vehicle-related output system 11. In this embodiment, the rider biological data is data related to the rider's heart rate generated by a smart watch and a grip sensor. The processing shown in FIG. 1B can be implemented in any of the system configurations shown in FIGS. 2 and 3A to 3X.

[0063] [Data Acquisition Process (A)] At least one processor 2 acquires at least one type of rider biometric data over time from an input device 13 provided on the saddle-ride type vehicle 16 or on the rider's equipment 17 or portable equipment 17 while the saddle-ride type vehicle 16 is traveling (step S11). Specifically, the at least one processor 2 acquires data related to the rider's heart rate over time from a smartwatch and a grip sensor.

[0064] [Prompt Generation Process (B)] At least one processor 2 generates rider emotion estimation prompt data using at least one type of rider biometric data acquired in step S11 (step S12). In this embodiment, the rider emotion estimation prompt data is data indicating the following CHL information: - Transition of the rider's heart rate over time, and - Method for evaluating the heart rate transition over time.

[0065] [Linguistic Interpretation Process (C)] At least one processor 2 supplies the rider emotion estimation prompt data generated in step S12 to the LLM 6 (step S13).

[0066] In the LLM 6, the rider emotion estimation prompt data supplied in step S13 is linguistically interpreted by the LLM 6 (step S21). In this embodiment, the LLM 6 uses the supplied rider emotion estimation prompt data to evaluate the rider's heart rate based on the evaluation method. For example, if an increase in the rider's fatigue level is detected based on the rider's heart rate, the LLM 6 generates riding-time rider emotion linguistic interpretation data such as, "You seem tired. I recommend you take a break." As another example, if an increase in the rider's anxiety is detected based on the rider's heart rate, the LLM 6 generates riding-time rider emotion linguistic interpretation data such as, "You seem mentally unstable. Is something wrong?"

[0067] Next, the at least one processor 2 acquires the moving rider emotion linguistic interpretation data generated in step S21 from the LLM 6 (step S14).

[0068] [Output Process (D)] At least one processor 2 supplies the moving rider emotion linguistic interpretation data acquired from the LLM 6 in step S14 to the output device 14 (step S15).

[0069] The output device 14 outputs CHL information indicating the linguistic interpretation result of the rider's emotion estimated while traveling, based on the linguistic interpretation data of the rider's emotion while traveling supplied from at least one processor 2 (step S31). In this embodiment, the meter of the saddle-type vehicle displays CHL information such as, for example, "You seem tired. We recommend you take a rest," or "You seem mentally unstable. Is something wrong?" In other words, in this embodiment, the obtained linguistic interpretation data of the rider's emotion while traveling is used to perform output for controlling the operation of at least one output device 14.

[0070] (Variation 1) A saddle-riding-type vehicle-related output system according to Variation 1 of the embodiment of the present invention will be described with reference to FIG. 4 . In Variation 1, the memory 3 stores programs for executing (A) data acquisition processing, (B) prompt generation processing, (C) linguistic interpretation processing, (D1) output processing, and (E) visual information generation processing. At least one processor 2 executes the programs. In Variation 1, the (A) data acquisition processing, (B) prompt generation processing, and (C) linguistic interpretation processing are the same as those in the above embodiment. Therefore, a description of these will be omitted. The (D1) output processing and (E) visual information generation processing will be described below.

[0071] [Visual Information Generation Process (E)] At least one processor 2 generates visual information using at least one type of LIDAR biometric data acquired in step S11. In Modification 1, at least one processor 2 generates visual information using the at least one type of LIDAR biometric data acquired in step S11 via the LLM 6 (step S22). More specifically, the process is as follows: At least one processor 2 supplies the LIDAR biometric data acquired in step S11 to the LLM 6 (step S16). The LLM 6 generates visual information using the LIDAR biometric data provided in step S16 (step S22). At least one processor 2 acquires the visual information generated in step S22 from the LLM 6 (step S17). Note that in the example shown in FIG. 4 , the visual information generation process (E) is performed after the linguistic interpretation process (C), but the visual information generation process (E) may be performed before the linguistic interpretation process (C) or before the prompt generation process (B).

[0072] The visual information may be an image or a video. In the example shown in Fig. 4, the visual information is generated by the LLM 6, but the visual information may be stored in advance in at least one memory 3, or may be received from outside the system 11 via a communication line such as the Internet.

[0073] [Output Process (D1)] At least one processor 2 supplies the traveling rider emotion linguistic interpretation data acquired from the LLM 6 in step S14 and the visual information acquired from the LLM 6 in step S17 to the output device 14 (step S15A). At least one processor 2 outputs the visual information so as to at least partially overlap with the CHL information in step S31A. In the example shown in FIG. 4 , the output of the CHL information and the visual information starts at the same time. However, for example, the output of the CHL information may start before the visual information, or the output of the CHL information may start after the visual information. The same applies to the end of the output of the CHL information and the visual information. In Variation 1, the CHL information may include an explanation about or related to the visual information that is output so as to overlap with the CHL information in time.

[0074] In this embodiment, a case has been described in which data related to the rider's heart rate is used to output a message from an output device in accordance with the rider's emotions. However, the present invention is not limited to the above embodiment. The present invention can be implemented in other embodiments, and various modifications can be added. For example, data related to the rider's body temperature may be used to adjust the temperature of heated grips or a heated seat as an output device. Alternatively, data related to the rider's heart rate may be used to output music in accordance with the rider's emotions from an output device. For example, data related to the rider's heart rate may be used to recommend a change to another riding mode. For example, data related to the rider's heart rate may be used to recommend a break to the rider and output information about the nearest rest area. The rider's biological data used to recommend a break to the rider is not limited to data related to the rider's heart rate.

[0075] 1: Saddle-ride type vehicle-related IPG (saddle-ride type vehicle-related interactive prompt generator) 2: Processor 3: Memory 4: Communication module 6: LLM 11: Saddle-ride type vehicle-related output system (system) 12: Network 13: Input device 14: Output device

Claims

1. A saddle-ride type vehicle-related output system, comprising: at least one memory; at least one processor capable of communicating with an LLM (Large Scale Language Model), connected to the memory, and configured to execute at least one program stored in the memory; at least one input device provided on the saddle-ride type vehicle or on equipment or carried by a rider of the saddle-ride type vehicle, configured to acquire or generate at least one type of lidar biometric data while the saddle-ride type vehicle is traveling and supply the at least one type of lidar biometric data to the at least one processor; and at least one output device, wherein the at least one program comprises: a data acquisition process that acquires the at least one type of lidar biometric data from the at least one input device over time while the saddle-ride type vehicle is traveling; and a prompt generation process that uses the at least one type of lidar biometric data acquired over time by the data acquisition process to generate lidar emotion estimation prompt data that can be linguistically interpreted by the LLM, is related to lidar emotion estimation, and has a reduced amount of data. a linguistic interpretation process that supplies the rider emotion estimation prompt data to the LLM and acquires from the LLM, rider emotion linguistic interpretation data during travel generated by linguistic interpretation by the LLM; and an output process that uses the rider emotion linguistic interpretation data during travel acquired from the LLM to generate an output for controlling the operation of the at least one output device.

2. A saddle-riding type vehicle-related output system according to claim 1, wherein said at least one program is further programmed to cause said at least one processor to execute the following in said output processing: visually or audibly output CHL (Communicative Human Language) information corresponding to said in-motion rider emotional linguistic interpretation data by said at least one output device, and output visual information consisting of images and / or video so as to at least partially overlap in time with said CHL information.

3. A saddle-riding type vehicle related output system according to claim 2, wherein said at least one program is further programmed to cause said at least one processor to execute a visual information generation process for generating said visual information using said at least one type of rider biological data acquired over time by said data acquisition process.

4. A saddle-riding type vehicle related output system according to claim 3, wherein said at least one program is further programmed to cause said at least one processor to execute the following in said visual information generation process: generate said visual information by said LLM using said at least one type of lidar biological data acquired over time by said data acquisition process.

5. A saddle-riding type vehicle-related output system according to any one of claims 2 to 4, wherein, in the output processing, the CHL information output by the at least one output device includes an explanation about or related to the visual information that is output so as to overlap in time with the CHL information.

6. A saddle-type vehicle-related output system according to any one of claims 1 to 5, comprising: the at least one input device; the at least one output device; and a saddle-type vehicle-related IPG (Interactive Prompt Generator) capable of communicating with each of the at least one input device, the at least one output device and the LLM via a network, and having the at least one memory and the at least one processor.

7. A straddle-type vehicle-related IPG included in the straddle-type vehicle-related output system according to claim 6.

Images