Vehicle-mounted device

Abstract

The present disclosure provides a technique for discriminating the type of a charging device in a field where the type of the charging device is unknown. An in-vehicle apparatus according to a first aspect of the present disclosure acquires a captured image from an in-vehicle camera configured to capture an image outside a vehicle, determines a type of charging device presented in the captured image by performing image analysis on the acquired captured image, and outputs information related to a determination result of the charging device.

Description

Technical Field

[0001] This disclosure relates to vehicle-mounted devices. Background Technology

[0002] Patent document 1 proposes the following technology: searching for charging facilities within the search range, selecting the charging facility closest to the destination, and providing information on the selected charging facility.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2021-085685 Summary of the Invention

[0006] One of the purposes of this disclosure is to provide a technology for identifying the type of charging equipment in a situation where the type of charging equipment is unclear due to reasons such as outdated information on charging facilities registered in map information.

[0007] The first embodiment of this disclosure includes a vehicle-mounted device comprising a control unit. The control unit is configured to perform the following actions: acquiring captured images from a vehicle-mounted camera configured to capture images of the exterior of the vehicle; determining the type of charging device presented in the captured images by performing image analysis on the acquired images; and outputting information related to the determination result of the charging device.

[0008] Invention Effects

[0009] According to this disclosure, it is possible to identify the type of charging equipment on-site when the type of charging equipment is unknown. Attached Figure Description

[0010] Figure 1 An example of a scenario in which this disclosure is applied is illustrated.

[0011] Figure 2 This schematically illustrates an example of the hardware configuration of the vehicle-mounted device of this disclosure.

[0012] Figure 3 This is an example illustrating the processing procedure of the vehicle-mounted device of this disclosure.

[0013] Explanation of reference numerals in the attached figures

[0014] 1: Vehicle-mounted device; 11: Control unit; 12: Storage unit; 13: Communication interface; 14: External interface; 15: Output device; 31: Program; 41: Charging equipment image; 51: Charging equipment information; 61: Trained model; 71: Remaining charge; 81: Charging settings; M: Vehicle; CE: Charging equipment; DC: Vehicle-mounted camera; S: Server; MDB: Map information database. Detailed Implementation

[0015] [1 Application Example]

[0016] Figure 1 An example of a scenario in which the present disclosure is applied is illustrated schematically. The vehicle-mounted device 1 of this embodiment includes a control unit 11. The control unit 11 is configured to: acquire captured images from a vehicle-mounted camera DC configured to capture images of the exterior of a vehicle M; determine the type of charging device CE presented in the captured images by performing image analysis on the acquired images; and output information related to the determination result.

[0017] It should be noted that, as another embodiment of the vehicle-mounted device 1 described above, one aspect of this disclosure may also be an information processing method that implements all or part of the above-described components, or it may be a program, or it may be a machine-readable storage medium such as a computer storing such a program. A machine-readable storage medium such as a computer is a medium that accumulates information such as programs through electrical, magnetic, optical, mechanical, or chemical action.

[0018] In this embodiment, vehicle M is an electric vehicle equipped with a battery and external charging capabilities, configured to convert electricity into power for propulsion. As long as it has a battery and charging capability, the type of vehicle M (number of wheels, size, etc.) can be arbitrarily selected. For example, vehicle M can be selected from two-wheeled vehicles, three-wheeled vehicles, four-wheeled vehicles, etc. When vehicle M is a car, its size can be selected from large, medium, semi-medium, ordinary, large-special, small-special, etc. When vehicle M is a two-wheeled vehicle, its size can be selected from large, ordinary, etc. It should be noted that the user can be anyone associated with vehicle M. Typically, the user can be a driver.

[0019] (Types of charging equipment)

[0020] There are various types of charging equipment (CE). For example, charging equipment (CE) used to charge vehicles (M) can be broadly categorized into standard chargers and fast chargers. Standard chargers include those with different output powers, such as 3kW and 6kW. Similarly, fast chargers include those with different output powers, such as 30kW and 90kW. Each charger is installed in a different facility. Chargers can be distinguished by their appearance, such as size and display. Furthermore, a recommended charging capacity can be set based on the type of charger.

[0021] The types of facilities for the installation of charging equipment CE are not particularly limited and can be determined arbitrarily. In one example, facilities may include detached houses, apartments, buildings, outdoor parking lots, car dealerships, convenience stores, supermarkets, hospitals, commercial facilities, leisure facilities, accommodation facilities, highway service areas / parking areas, gas stations, road stations, etc.

[0022] Information about facilities equipped with charging devices (CEs) (such as the type of charger) can be pre-registered in the map information database (MDB). The MDB can be managed and provided by the server S. Furthermore, the vehicle-mounted device 1 can be configured to perform navigation (route guidance). In this case, the vehicle-mounted device 1 can be configured to confirm the location of the facility and the type, quantity, and operational status of the charging devices (CEs) on the navigation system. However, since the charging devices (CEs) may be frequently updated, the following situations may occur: the information on the charging devices (CEs) on the navigation system becomes outdated, and the type of charging device (CE) indicated by the navigation information differs from the actual type of charging device (CE). In this case, adverse conditions may occur due to a deviation between the actual charging time and the assumed charging time. Therefore, to keep the information on the charging devices (CEs) up-to-date, the vehicle M (vehicle-mounted device 1) can be configured to: when the type of the actual charging device (CE) determined by the image analysis differs from the type indicated by the navigation information, notify the server S of feedback regarding the type of the actual charging device (CE), thereby providing the latest information to the map information database (MDB).

[0023] Furthermore, ideally, during the charging of vehicle M, a charging setting 81 corresponding to the type of charging device CE is performed. By knowing the type of charging device CE and performing the corresponding charging setting 81, charging time can be optimized, and battery degradation can be reduced. In this embodiment, the charging device CE is photographed by an onboard camera DC, and the type of charging device CE is determined by image analysis of the photographed image. The output corresponding to the determination result is then performed (sending to server S, changing charging setting 81), thereby automatically achieving the above objectives.

[0024] (Filming of the charging equipment by a vehicle-mounted camera)

[0025] As long as an image capable of identifying the type of charging device CE can be acquired, the type of vehicle-mounted camera DC is not particularly limited and can be appropriately selected according to the implementation method. The vehicle-mounted camera DC is configured to capture images of the exterior of the vehicle M in order to capture images of the charging device CE.

[0026] (Image Analysis)

[0027] The image analysis method for images captured by the vehicle-mounted camera (DC) of the charging device (CE) is not particularly limited and can be appropriately selected according to the implementation method. In one example, the image analysis method may be a method of determining the type of charging device (CE) through pattern matching. In another example, the image analysis method may also be a method of determining the type of charging device (CE) using a trained machine learning model.

[0028] Image analysis via pattern matching may include: the control unit 11 referring to a charging device image 41 pre-stored in the storage unit 12 and applying a matching method. In one example, the matching method may include template matching, feature point matching, etc. Template matching is performed by the control unit 11 calculating the similarity between a template image of the charging device CE included in the charging device image 41 and a captured image of the charging device CE, thereby determining whether the objects in the two images are the same. The similarity can be calculated by calculating the consistency of pixel values ​​in the captured image of the charging device CE corresponding to the template image, such as correlation coefficients and mean square errors. Feature point matching is performed by the control unit 11 observing the correspondence between feature points extracted from the template image and the captured image of the charging device CE, thereby determining whether the objects in the two images are the same. Feature points can be points in an image containing objects that distinguish the quantity of a pixel from surrounding pixels. The quantity of a pixel can be color, brightness, etc.

[0029] In image analysis using a trained machine learning model, the control unit 11 uses the trained model 61 stored in the storage unit 12 to determine the type of charging device CE presented in the captured image. The trained model 61 has one or more operational parameters that can be adjusted through machine learning. The one or more operational parameters are used for calculations that estimate the target conclusion. The trained model 61 can be composed of, for example, a neural network, a support vector machine, a regression model, or other functional expressions (operational models). The machine learning method can be appropriately selected depending on the machine learning model used (e.g., backpropagation).

[0030] When the trained model 61 includes a neural network, the structure of the neural network is not particularly limited and can be appropriately determined according to the implementation method. The structure of the neural network can be determined, for example, by the number of layers from the input layer to the output layer, the types of each layer, the number of nodes (neurons) contained in each layer, and the connection relationships between the nodes in each layer. The neural network can include any mechanism such as a recursive structure, self-attention mechanism, or autoregressive model. For example, the neural network can include any layer such as fully connected layers, convolutional layers, pooling layers, deconvolutional layers, unpooling layers, normalization layers, dropout layers, or LSTM (Long Short-Term Memory) networks. Furthermore, the neural network can include any type of model such as a Diffusion model, a Transformer model, or a generative model. The weights of the connections between the nodes in the neural network and the thresholds of each node are examples of computational parameters.

[0031] (Output of information related to the judgment result)

[0032] After determining the type of charging device CE, the control unit 11 can be configured to output information related to the determination result. The content of the information related to the determination result can be appropriately determined according to the implementation method. In one example, the control unit 11 can output the type of charging device CE as determined by the result. In another example, the control unit 11 can also perform arbitrary information processing based on the type of charging device CE. The control unit 11 can also output the result obtained by performing this information processing as information related to the determination result. The output of the result obtained by performing this information processing may include, for example, outputting messages such as recommended charging amount and estimated charging time; or sending the type of charging device CE determined by the result to the server S.

[0033] The recommended charging amount represents the charging amount recommended for each charging device CE (full charge, 80% charge, etc.). The estimated charging time represents the charging time from the current charging amount of vehicle M to the recommended charging amount. The control unit 11 can calculate this information by referring to the storage unit 12 and obtaining the charging device information 51 and the remaining charging amount 71.

[0034] The destination of the above-mentioned determination result can be appropriately determined according to the implementation method. In one example, the output destination can be RAM (Random Access Memory), storage unit 12, output device 15, server S, user terminal, or other computers. For example, when the determination result is output to storage unit 12, the current charging setting 81 stored in storage unit 12 is updated and can be set to a setting corresponding to the type of charging device CE in the determination result. Furthermore, when the determination result is output to server S, if server S, upon receiving the determination result, determines that the information of the charging device CE held in map information database MDB is different from the information of the charging device CE in the determination result, it updates map information database MDB, thereby keeping the data of the charging device CE up-to-date.

[0035] (Route-guided reconstruction)

[0036] When route guidance includes charging via a charging device (CE), malfunctions may occur if the type of CE indicated in the navigation information differs from the actual type. For example, a shorter charging time might cause the planned travel period to overlap with peak hours, increasing the likelihood of getting caught in traffic jams. Therefore, the on-board unit 1 can also reconstruct the route guidance based on the determination of the type of charging device (CE) (e.g., extending the charging time, skipping charging and charging at other facilities, etc.).

[0037] [2 Examples of Composition]

[0038] Figure 2 This schematically illustrates an example of the hardware configuration of the vehicle-mounted device 1 of this disclosure. For example... Figure 2 As shown, the vehicle-mounted device 1 in this embodiment is a computer electrically connected to a control unit 11, a storage unit 12, a communication interface 13, an external interface 14, and an output device 15. The control unit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and is configured to perform arbitrary information processing.

[0039] The storage unit 12 may be composed of, for example, a hard disk drive, a solid-state drive, etc. In this embodiment, the storage unit 12 stores a program 31, a charging device image 41, a charging device information 51, a trained model 61, remaining charge amount 71, and charging settings 81. The program 31 is a program for causing the vehicle-mounted device 1 to perform the information processing of this embodiment. The program 31 contains a series of instructions for this information processing. The charging device image 41 includes a reference image of each charging device CE for pattern matching. The charging device information 51 contains various information such as the recommended charge amount for each charging device CE. The trained model 61 is a model that has been trained (adjusted with computational parameters) in advance for determining the type of charging device CE presented in the captured image. The remaining charge amount 71 represents the remaining charge amount (kW) of the vehicle M. The charging settings 81 records various settings when the vehicle M is charging, and are set according to the type of charging device CE.

[0040] Communication interface 13 is configured for wired or wireless data communication over a network. Communication interface 13 may be, for example, a wired LAN (Local Area Network) module, a wireless LAN module, etc. In this embodiment, the vehicle-mounted device 1 can use communication interface 13 to perform network-based data communication with other computers (e.g., server S, etc.). External interface 14 may be, for example, a USB (Universal Serial Bus) port, a dedicated port, a wireless communication port, etc., and is configured to connect to external devices via wired or wireless means. In this embodiment, the vehicle-mounted device 1 can be connected to a vehicle-mounted camera DC via external interface 14. Output device 15 may be, for example, a display, a speaker, or other device for output.

[0041] It should be noted that the specific hardware configuration of the vehicle-mounted device 1 can be appropriately omitted, replaced, or added depending on the implementation method. For example, the control unit 11 may also include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs (field-programmable gate arrays), DSPs (digital signal processors), GPUs (graphics processing units), ASICs (application-specific integrated circuits), etc.

[0042] [3 Action Examples]

[0043] Figure 3This is an example of the processing procedure of the vehicle-mounted device 1 of this disclosure. The following processing procedure is an example of an information processing method performed by a computer.

[0044] In step S101, the control unit 11 acquires an image of the charging device CE captured by the vehicle-mounted camera DC. In step S102, the control unit 11 performs image analysis on the acquired image of the charging device CE to determine the type of the charging device CE. Regarding the image analysis method, either a pattern matching method using the template image included in the charging device image 41 can be applied, or a method for determining the type of charging device CE using the trained model 61 can be applied. When the determination of the type of charging device CE by image analysis is completed, the control unit 11 proceeds to step S103.

[0045] In step S103, the control unit 11 outputs the determination result of the type of charging device CE obtained in step S102. The output content and output destination can be appropriately determined according to the implementation method. In one example, the control unit 11 may be set to output the determination result of the type of charging device CE to the storage unit 12 to change the charging setting 81. In another example, the control unit 11 may also perform at least one of the following actions: outputting messages such as recommended charging amount and estimated charging time to the output device 15; and notifying the server S of the type of charging device CE. When this step is completed, the information processing of this embodiment ends.

[0046] [feature]

[0047] In this embodiment, in step S102, the control unit 11 determines the type of the charging device CE by performing image analysis on the captured image of the charging device CE. Therefore, in step S102, the type of the charging device CE can be determined even in situations where the information on the charging device CE registered in the map information is outdated, or where multiple charging devices of different types exist, making the type of the charging device CE unclear. By providing this determination result to the user, an improvement in the user experience value in the vehicle M can be expected. Furthermore, by applying the charging settings 81 corresponding to the determination result before actual charging, appropriate charging of the vehicle M can be performed, resulting in optimized charging time and reduced battery degradation. Moreover, if the type of the charging device CE registered in the map information database MDB differs from the determination result, the information on the charging device CE in the map information database MDB can be updated to the latest state by notifying the server S of the determination result. This update is not proactively performed by the administrator of the map information database MDB, but is caused by the actions of each user, thus improving the efficiency of map information database MDB updates.

[0048] [4 Variations]

[0049] The embodiments of this disclosure have been described in detail above, but the descriptions above are merely examples of this disclosure at all points. Various modifications or variations can be made without departing from the scope of this disclosure. The processes and units described in this disclosure can be freely combined and implemented as long as no technical contradiction is created.

Claims

1. A vehicle-mounted device comprising a control section configured to perform the following actions: acquiring a captured image from a vehicle-mounted camera configured to capture an image outside a vehicle; determining a type of a charging device present in the captured image by performing image analysis on the captured image; and outputting information related to the determination result of the charging device.

Images