Learning System and Learning Device Using Convolutional Neural Network

Abstract

To accurately predict or estimate a change state of an apparatus, a member and so on continuously changing during a course of time by using a convolution neural network.SOLUTION: The present invention is directed to a convolution neural network matrix data generating method of executing convolution calculation based on matrix data arranged by setting vehicle information as a component of a matrix. The matrix data is formed of predetermined time-series data in which each row of the matrix is continuously changed during a course of time in an arrangement direction of each column. The time-series data is formed of first data having high influence on the convolution calculation and second data having lower influence as compared with the first data. The convolution calculation is executed by using a kernel which sections the matrix data with the same number of the rows and the columns as predetermined coefficients, and at least one row of the first data is arranged for the same number of rows as the coefficients (step S5).SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a convolutional neural network that performs calculation (learning) based on various collected data to extract features, and particularly to a learning system and a learning device using the convolutional neural network. Study

Background Art

[0002] In recent years, the practical application of technologies that efficiently perform processes such as learning, inference, recognition, and judgment by utilizing advanced technologies such as artificial intelligence (AI) and information and communication technology (ICT) has been promoted. Among them, there is machine learning as a learning method executed by AI. Machine learning is a process in which a machine (computer) learns on its own using a large number of given data, and based on the learning result (trained model), optimizes the output data for the input data, and performs estimation and prediction based on the output data. As an example of such machine learning, there is a processing technology using a convolutional neural network (CNN; Convolutional Neural Network). Convolutional neural networks are utilized in various fields such as, for example, image recognition, speech recognition, natural language processing, and machine translation.

[0003] A technique for estimating the degradation of a power storage element using the convolutional neural network as described above is disclosed in Patent Document 1. The degradation estimation device described in this Patent Document 1 acquires the SOH [State Of Health] of the power storage element at a first time point and the SOH at a second time point after the first time point. At the same time, a time-series data related to the state of the power storage element from the first time point to the second time point and the SOH at the first time point are used as input data, and a learning model is learned based on learning data having the SOH at the second time point as output data. Then, based on the learned learning model, the degradation state of the power storage element is estimated.

[0004] ​Note that Patent Document 2 describes a battery life learning device for accurately predicting the remaining life of a vehicle battery. The battery life learning device described in this Patent Document 2 predicts the remaining life of a vehicle battery using a neural network. Specifically, the battery life learning device described in Patent Document 2 learns a prediction model for predicting the remaining life of a vehicle battery from time-series data of deterioration indicators of the vehicle battery based on learning data including time-series data of deterioration indicators and the remaining life at a predetermined past time of a "learning vehicle battery" that has reached the end of its life, and obtains a learned prediction model. Then, based on the time-series data of the deterioration indicators of the "vehicle battery to be predicted" and the learned prediction model, the remaining life of the "vehicle battery to be predicted" is predicted.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] The degradation estimation device described in the above Patent Document 1 and the battery life learning device described in Patent Document 2 generate a learned model (prediction model) using a neural network, and estimate the degradation state and remaining life of a power storage element or a vehicle battery. For example, in the degradation estimation device described in Patent Document 1, time-series data related to the SOC of a power storage element based on voltage data of the power storage element is input to a convolutional neural network, and the degradation state of the power storage element is estimated. Further, in the battery life learning device described in Patent Document 2, battery information such as the internal resistance, voltage, current, temperature, charge amount, and full charge capacity of a vehicle battery is input to a neural network, and the remaining life of the vehicle battery is predicted. In addition, for example, there is also a conventional technique in which a value obtained by correcting the starting voltage of a vehicle battery with an outside air temperature is input to a neural network to estimate the degradation state of the vehicle battery. By incorporating data that directly affects or has a high degree of influence on the performance and degradation state of the battery and utilizing a neural network, the degradation state and remaining life of the battery can be accurately estimated. However, for example, like the vehicle battery described in Patent Document 2, in addition to direct battery information such as the internal resistance and voltage of the vehicle battery, there is data related to the driving state and usage state of the vehicle, which indirectly affects the performance and degradation state of the vehicle battery. However, in the conventional techniques as described above, peripheral data including factors related to the performance and degradation state of the battery other than data with a high degree of influence has not been utilized. Also, a technique for incorporating and appropriately utilizing such peripheral data has not been established.

[0007] Thus, in order to more accurately predict or estimate the change state of a predetermined device, member, or substance that changes continuously over time (degrades) by applying a neural network, particularly a convolutional neural network that exhibits advanced recognition performance or learning performance, there was still room for improvement.

[0008] This invention was conceived by paying attention to the above technical problems, and uses a convolutional neural network that can accurately predict or estimate the change state of a device, member, or substance that changes continuously over time by applying a convolutional neural network. Study An object of the present invention is to provide a learning system and a learning device.

Means for Solving the Problems

[0012] In order to achieve the above objectives, This invention is a learning system using a convolutional neural network that includes a control unit mounted on a vehicle and a server installed outside the vehicle, executes a convolutional operation based on matrix data in which predetermined information is arranged as components of a matrix, and estimates the state of change over time of a power storage device mounted on the vehicle. The control unit acquires vehicle information detecting the behavior and state of each part of the vehicle, and transmits the vehicle information to the server. The server constitutes the matrix data with time-series data of the vehicle information in which each row of the matrix data changes continuously in time in the arrangement direction of each column of the matrix data, and first data related to the starting voltage of the power storage device output to start the engine, and second data including at least any one of the travel distance of the vehicle, the outside air temperature, the temperature of the power storage device, the travel time of the vehicle, the parking time of the vehicle, and the number of starts of the engine. The server executes the convolutional operation using a kernel (or filter, or mask) that partitions the matrix data by a predetermined number of rows and columns, arranges at least one row of the first data for each row by the coefficient, arranges the second data in the remaining rows excluding the rows arranging the first data, executes the convolutional operation using the arranged first data and second data, and estimates the state of change over time.

[0014] Further, this invention executes a convolutional operation based on matrix data in which vehicle information detecting the behavior and state of each part of the vehicle is arranged as components of a matrix, and the vehicle The power storage devices installed in both vehiclesA learning device using a convolutional neural network for estimating a state over time, comprising: a detection unit that detects the behavior and state of each part; a data acquisition unit that acquires the data detected by the detection unit as the vehicle information; and time-series data of the vehicle information in which each row of the matrix data changes continuously in time in the arrangement direction of each column of the matrix data, and the matrix data is constituted a second voltage related to a starting voltage of the power storage device outputted for starting the engine; 1 data, and the a second temperature sensor including at least one of a mileage of the vehicle, an outside air temperature, a temperature of the power storage device, a driving time of the vehicle, a parking time of the vehicle, and a number of starts of the engine; 2 data to constitute the time-series data, and for each row of the coefficient of a kernel (or filter, or mask) that partitions the matrix data by the coefficient of the rows and columns by a predetermined coefficient, at least one row of the first data is arranged, and the second data is arranged in the remaining rows excluding the rows in which the first data is arranged, a data generation unit , the first data and the second data arranged the aforementioned kernel Ru ( or filter, or mask ) and An arithmetic unit that executes the convolution operation using the above, and based on the result of executing the convolution operation, the Storage device A learning unit that estimates the state of change over time, characterized in that it is provided with be.

[0016] Incidentally, the "kernel" in this invention may be referred to as "filter", or "mask", or "window" as described above. Therefore, the "kernel" in this invention can be read as any of the names of "filter", "mask", or "window". In this invention and the following description of the embodiments of this invention, it is mainly referred to as "kernel".

Effects of the Invention

[0017] This invention performs learning using a convolutional neural network and applies the learning result to the vehicle The power storage devices installed in both vehiclesEstimate or predict regarding the state of change over time, etc. For example, estimate the deterioration state of the power storage device mounted on the vehicle, that is, the state of change over time of the battery performance. And in the convolutional neural network applied in this invention, for example, convolution operation is executed using matrix data in which predetermined information such as vehicle information regarding the behavior and state of each part of the vehicle is arranged as components of the matrix. By performing learning using the data obtained by subjecting the data to the convolution operation processing , the storage device Learning for estimating the state of change over time, the deterioration state, etc. can be accurately executed.

[0018] The matrix data serving as the input data for the convolution operation is generated, for example, by storing each data of predetermined information (for example, vehicle information) in components or cells obtained by dividing the entire input data in a grid pattern. The matrix data in the convolutional neural network of this invention stores, arranged, time-series data of predetermined items that change continuously over time in the arrangement direction of each column of the matrix data (or the order direction of the arrangement, or the column number direction). And the time-series data in this invention , which is related to the starting voltage of the power storage device outputted for starting the engine first data, and a vehicle mileage, an outside air temperature, a temperature of the power storage device, a vehicle driving time, a vehicle parking time, and an engine start count.It is composed of two types of data. That is, the time-series data is composed of first data with a high degree of influence on the convolution operation and second data with a lower degree of influence on the convolution operation than the first data. Therefore, in the matrix data of this invention, the first data and the second data are alternately arranged in the row direction (vertical direction). At the same time, the first data and the second data are arranged so that at least one row of the kernel (or filter, or mask) in the convolution operation becomes the first data. That is, the first data and the second data in the matrix data are arranged so that at least one row of the first data is necessarily included in the kernel (or filter, or mask) in the convolution operation. Specifically, if the coefficient determining the size of the kernel (or filter, or mask) is "n", the kernel (or filter, or mask) has a size of n rows × n columns, and matrix data is generated so that at least one row of the n rows becomes the first data. For example, when using a kernel (or filter, or mask) with a coefficient of "3", matrix data is generated so that at least one row of the first data is arranged every three rows. The second data is arranged in the remaining rows other than the rows where the first data is arranged.

[0019] In the convolutional neural network that uses the matrix data generated as described above, the main features to be learned by the convolutional operation are accurately extracted by the first data. At the same time, the secondary features of the learning target are extracted by the second data. Then, the first data and the second data are combined (convolved) in a complex manner, and learning by the convolutional operation is performed. For example, taking the deterioration state of a power storage device mounted on a vehicle as the learning target, the starting voltage of the power storage device is taken into the convolutional neural network as the first data. Also, for example, factors other than the first data related to the deterioration state of the power storage device, such as the usage state and usage environment of the vehicle, are taken into the convolutional neural network as the second data. Therefore, in the convolutional neural network of this invention, compared with the conventional learning that uses only the main factors such as the first data in this invention, or the conventional learning that simply corrects the main factors, more accurate learning can be performed.

[0020] Therefore, according to this invention, by applying a convolutional neural network, it is possible to appropriately and accurately estimate the change state of a device, member, or substance that changes continuously over time, such as the deterioration state of a power storage device mounted on a vehicle.

Brief Description of the Drawings

[0021] [Figure 1] It is a diagram showing an example of the configuration and control system of a vehicle that is the learning target in the convolutional neural network of this invention. [Diagram 2] It is a block diagram for explaining an in-vehicle control unit and an external server in a learning system and a learning device using the convolutional neural network of this invention. [Diagram 3] It is a diagram showing matrix data and a kernel (or filter, or mask) used in the convolutional operation of the convolutional neural network of this invention. (a) shows an image of the stored data in the matrix data, and (b) is a diagram showing an image of the kernel (or filter, or mask). [Figure 4] A diagram showing matrix data and a kernel (or filter, or mask) used in the convolution operation of the convolutional neural network of the present invention. In particular, it is a diagram showing time-series data constituting the matrix data of the present invention, and images of first data and second data constituting the time-series data. [Diagram 5] A diagram for explaining an example of control executed by a learning system and a learning device using the convolutional neural network of the present invention, which is a flowchart showing control contents including a method for generating matrix data for a convolutional neural network. [Figure 6] A diagram for explaining an example of a convolution operation executed using the matrix data shown in FIG. 3, which is a diagram showing an image of the movement (stride) of a kernel (or filter, or mask).

Embodiments for Carrying Out the Invention

[0022] Embodiments of the present invention will be described with reference to the drawings. Note that the embodiments shown below are merely examples of the case where the present invention is embodied, and do not limit the present invention.

[0023] The learning system and learning device using the convolutional neural network in the embodiments of the present invention perform a convolution operation based on a large number of collected information and extract features of the information source. For example, an existing general vehicle is used as a control target, and the state of the time-varying change of a predetermined element of the vehicle is estimated. In that case, the learning system and learning device using the convolutional neural network in the embodiments of the present invention include a control unit mounted on the vehicle and a server installed outside the vehicle.

[0024] FIG. 1 shows an example of a vehicle equipped with a control unit as components of a learning system and a learning apparatus using a convolutional neural network in an embodiment of the present invention. The vehicle Ve shown in FIG. 1 mainly includes a driving power source (POWER) 1, driving wheels 2, a starter motor 3, a battery 4, a detection unit 5, a control unit (ECU) 6, and a communication module (DCM) 7.

[0025] The driving power source 1 is a power source that outputs a driving torque for running the vehicle Ve. The driving power source 1 is, for example, an internal combustion engine such as a gasoline engine or a diesel engine, and is configured such that output adjustment and operating states such as start and stop are electrically controlled. In the case of a gasoline engine, the opening degree of the throttle valve, the fuel supply amount or injection amount, the execution and stop of ignition, and the ignition timing are electrically controlled. Alternatively, in the case of a diesel engine, the fuel injection amount, the fuel injection timing, or the opening degree of the throttle valve (in the EGR system) is electrically controlled. In the example shown in FIG. 1, an engine 8 equipped with a starter motor 3 described later is mounted as the driving power source 1.

[0026] Note that the driving power source 1 may be, for example, an electric motor such as a permanent magnet synchronous motor or an induction motor. In that case, the electric motor may be a so-called motor-generator having both a function as a prime mover that is driven by supplying electric power to output torque and a function as a generator that generates electricity by being driven by receiving an external torque. If it is a motor-generator, the rotational speed, torque, or the switching between the function as a prime mover and the function as a generator is electrically controlled. Further, the driving power source 1 may be a so-called hybrid drive unit equipped with both the engine 8 and an electric motor (motor-generator).

[0027] The drive wheel 2 generates the driving force of the vehicle Ve by transmitting the driving torque output by the driving force source 1. In the embodiment shown in FIG. 1, the drive wheel 2 is connected to the driving force source 1 via a transmission 9, a differential gear 10, and a drive shaft 11. Note that the vehicle Ve in the embodiment of the present invention may be a front-wheel drive vehicle that transmits the driving torque to the front wheels and generates the driving force at the front wheels as in the embodiment shown in FIG. 1. Alternatively, the vehicle Ve may be a rear-wheel drive vehicle that transmits the driving torque to the rear wheels via, for example, a propeller shaft (not shown) and generates the driving force at the rear wheels. Alternatively, the vehicle Ve may be a four-wheel drive vehicle provided with a transfer mechanism (not shown) that transmits the driving torque to both the front wheels and the rear wheels and generates the driving force at both the front wheels and the rear wheels.

[0028] When the engine 8 is mounted as the driving force source 1 of the vehicle Ve as described above, the starter motor 3 is mounted on the engine 8 and drives a crankshaft (not shown) when the engine 8 is started. The starter motor 3 operates by being supplied with electric power from a battery 4 described later. Note that, instead of the starter motor 3, an alternator (not shown) may function as the starter of the engine 8. Alternatively, a motor (not shown) having both the function of a starter and the function of an alternator may be used.

[0029] The battery 4 corresponds to the "power storage device" in the embodiment of the present invention and supplies electric power to the starter motor 3 described above. In the example shown in FIG. 1, the battery 4 is a so-called auxiliary battery and supplies electric power to in-vehicle devices (not shown) such as the lighting lamps and air conditioner of the vehicle Ve. Note that the "power storage device" in the embodiment of the present invention may be, for example, a main battery or a driving battery (not shown) in a hybrid vehicle or an electric vehicle.

[0030] The detection unit 5 is a device or apparatus for acquiring various types of data and information necessary for controlling the vehicle Ve. For example, it includes a power supply unit, a microcomputer, sensors, and input / output interfaces, etc. In particular, the detection unit 5 in the embodiment of this invention has a battery voltage sensor 5a that detects the voltage of the battery 4 as information with a high degree of influence or relevance on the deterioration of the battery 4 when estimating the deterioration state of the "power storage device", that is, the battery 4. Further, although the degree of influence or relevance of the detection unit 5 on the deterioration of the battery 4 is lower than that of the voltage of the battery 4, as information that indirectly or comprehensively affects the deterioration of the battery 4, it has an odometer sensor 5b that detects the mileage of the vehicle Ve, an outside air temperature sensor 5c that detects the outside air temperature, a battery temperature sensor 5d that detects the temperature of the battery 4, a timer 5e that measures the driving time, parking time, etc. of the vehicle Ve, and a counter 5f that measures the number of starts of the engine 8. In addition, the detection unit 5 has, for example, a vehicle speed sensor (or wheel speed sensor) 5g that detects the vehicle speed, a rotation speed sensor 5h that detects the rotation speed of the engine 8, etc. Further, the detection unit 5 may include, for example, a GPS [Global Positioning System] receiver (not shown) for acquiring the position information of the vehicle Ve, an in-vehicle camera (not shown) for acquiring imaging information regarding the external situation of the vehicle Ve, etc. And the detection unit 5 is electrically connected to a control unit 6 described later, and outputs an electrical signal corresponding to the detection value, calculated value, position information, etc. of various sensors, devices, apparatuses, etc. as described above to the control unit 6 as detection data.

[0031] The control unit 6 is an electronic control device mainly composed of, for example, a microcomputer, and comprehensively controls the vehicle Ve. Various data detected or measured by the above detection unit 5 are input to the control unit 6. For this purpose, the control unit 6 has a data acquisition unit 6a described later. Then, the control unit 6 transmits various data input to the data acquisition unit 6a to an external server 101 described later via a communication module 7 described later. At the same time, the control unit 6 performs calculations using the input various data, pre-stored data, calculation formulas, etc. Then, the control unit 6 outputs the calculation result as a control command signal and is configured to control the operations of each part of the vehicle Ve respectively. In FIG. 1, an example is shown in which one control unit 6 is provided for one vehicle Ve, but a plurality of control units 6 may be provided for each device or equipment to be controlled or for each control content.

[0032] The communication module 7 performs data transmission and reception between the control unit 6 of the vehicle Ve and a server 101 provided outside the vehicle Ve described later. The communication module 7, for example, mounts a dedicated wireless communication system (not shown) called DCM [Data Communication Module] on the vehicle Ve, and transmits and receives various data between the control unit 6 and the server 101 using a dedicated communication line. In the embodiment of the present invention, general communication equipment (not shown) may be used to transmit and receive data using a general mobile communication line. Alternatively, for example, data transmission and reception may be performed using wired communication equipment installed in a vehicle Ve dealership, repair shop, etc.

[0033] In the learning system and learning device using a convolutional neural network according to an embodiment of the present invention, as will be described later, the control unit 6 described above transmits and receives data to and from a server 101 provided outside the vehicle Ve, and executes machine learning in cooperation with the server 101. Specifically, using a convolutional neural network, the state of change over time of a predetermined element of the vehicle Ve (for example, the deterioration state of the battery 4 as described above) is estimated. Therefore, the learning system and learning device using a convolutional neural network according to an embodiment of the present invention include, as shown in FIG. 2, the in-vehicle control unit 6 described above and a server 101 installed outside the vehicle Ve.

[0034] Specifically, the control unit (ECU) 6 of the vehicle Ve has the data acquisition unit 6a and the transmission data creation unit 6b described above.

[0035] The data acquisition unit 6a acquires, for each vehicle Ve, predetermined data necessary for generating learning data (matrix data described later) for a convolutional neural network. The various data detected by the detection unit 5 described above are acquired as vehicle information for generating learning data for a convolutional neural network as needed.

[0036] The transmission data creation unit 6b processes the various data acquired by the data acquisition unit 6a above into transmission data adapted to the communication module 7 as learning data to be used in a convolutional neural network, and transmits it to the server 101 via the communication module 7.

[0037] Note that FIG. 2 shows a situation where two control units 6 transmit and receive data to and from the server 101, respectively. That is, it shows the control units 6 mounted on two vehicles Ve, respectively, and one server 101 set externally. The learning system and learning device using the convolutional neural network in the embodiment of the present invention execute machine learning (convolution operation) using various data collected from the vehicle Ve. In order to improve the learning accuracy of the convolutional neural network, it is desirable to collect as much data as possible acquired over as wide a range as possible. Therefore, the learning system and learning device using the convolutional neural network in the embodiment of the present invention are not limited to two vehicles Ve as shown in FIG. 2, and a large number of data are collected from the control units 6 mounted on a large number of vehicles Ve, respectively.

[0038] On the other hand, the server 101 installed outside the vehicle Ve has, for example, a data storage unit 101a, a data generation unit 101b, an arithmetic unit 101c, and a learning unit 101d.

[0039] The data storage unit 101a stores various data and information received from the control unit 6 of each vehicle Ve, and various data and information processed by arithmetic operations in the server 101, etc. in a storage medium (not shown) as a database related to vehicle information.

[0040] The data generation unit 101b generates input data used in the convolutional neural network in the embodiment of the present invention from a large number of data stored in the data storage unit 101a above, that is, vehicle information. In the convolutional neural network in the embodiment of the present invention, a convolution operation is executed based on matrix data in which predetermined information is arranged as components of a matrix. Each row of the matrix data of the data generation unit 101b constitutes the matrix data with time-series data of vehicle information that changes continuously in time in the arrangement direction of each column of the matrix data.

[0041] Specifically, as shown in FIGS. 3(a) and 4, the data generation unit 101b generates the matrix data M that serves as the input data for the convolution operation by, for example, storing each data of the vehicle information in components or cells obtained by dividing the entire input data in a grid pattern. In that case, the data generation unit 101b arranges and stores the time-series data of a predetermined item that changes continuously over time in the array direction (or the order direction of the array, or the column number direction) of each column of the matrix data.

[0042] Further, as shown in FIG. 4, the data generation unit 101b divides the time-series data of the vehicle information that constitutes the above matrix data M into two types of data with different items: the first data that is the main factor for the convolution operation and the second data that is the secondary factor for the convolution operation. The first data includes data that has a high influence on the convolution operation of the convolutional neural network and is at least related to the main factor of the change over time of a predetermined element of the vehicle Ve. For example, when estimating the degradation state of the battery 4 as described above, the time-series data regarding the voltage of the battery 4 (for example, the starting voltage), which has a high influence on the degradation state of the battery 4, that is, a high relevance to the main factor of the change over time (degradation) of the performance of the battery 4, is arranged in the matrix data M as the first data. On the other hand, the second data is data that has a lower influence on the convolution operation than the above first data and that indirectly or comprehensively affects the degradation of the battery 4. For example, time-series data regarding the driving distance of the vehicle Ve, the driving time of the vehicle Ve, the outside air temperature, and the temperature of the battery 4, etc., other than the voltage of the battery 4 described above, is arranged in the matrix data M as the second data. Therefore, the first data and the second data are arranged alternately in the row direction (vertical direction) of the matrix data M.

[0043] Furthermore, as shown in FIG. 4, the data generation unit 101b arranges at least one row of first data for each row corresponding to the coefficients of the kernel (or filter, or mask) K in the convolution operation of the convolutional neural network. The kernel K is a convolutional layer also referred to as a "filter", "mask", or "window" in the convolution operation, and is a set (matrix) of data partitioned into rows and columns corresponding to predetermined coefficients of the above matrix data and arranged in a grid pattern. The coefficient of the kernel K is a numerical value that determines the size or data volume of the kernel K. For example, if the coefficient of the kernel K is "n", the kernel K becomes an n-row × n-column matrix. Then, the data generation unit 101b arranges the first data and the second data in the above matrix data M such that at least one row of the kernel K becomes the first data. That is, the first data and the second data in the matrix data M are alternately arranged so that at least one row of the first data is always included in the kernel K in the convolution operation. As shown in FIG. 3(b) and FIG. 4, when using a kernel K with a coefficient of "3", the matrix data M is generated such that at least one row of the first data is arranged every three rows. At the same time, the second data is arranged in the remaining rows other than the rows where the first data is arranged to generate the matrix data M.

[0044] The operation unit 101c executes a convolution operation in the convolutional neural network based on the matrix data M (time-series data of vehicle information) generated by the data generation unit 101b as described above. The convolutional neural network in the embodiment of the present invention is characterized in that it performs a convolution operation using the time-series data divided into the first data and the second data as described above, and the matrix data M generated by arranging the time-series data such that at least one row of the first data is included for each size (number of rows) of the kernel K. The operation method of the convolution operation is executed in the same manner as that of the conventional convolutional neural network. For example, it is possible to execute a convolution operation using the same operation method as the convolutional neural network described in Patent Document 1 mentioned above.

[0045] The learning unit 101d estimates the state of change of the vehicle Ve over time based on the result of performing the convolution operation in the above-described operation unit 101c. For example, it estimates the deterioration state of the battery 4 as described above, that is, the state of change of the performance of the battery 4 over time. As a result of estimating the state of such a change over time, if an abnormality such as the need for measures such as repair or replacement is detected, the abnormality detection flag is turned on, and predetermined measures for the abnormality (for example, display of a warning, notification, communication, etc.) are executed.

[0046] As described above, the learning system and the learning device using the convolutional neural network in the embodiment of the present invention mainly aim to accurately predict or estimate the state of change of the vehicle Ve over time by applying the convolutional neural network. For this purpose, the learning system and the learning device using the convolutional neural network in the embodiment of the present invention are configured to execute the control shown in the following respective flowcharts.

[0047] In the flowchart shown in FIG. 5, first, in the control unit 6 of the vehicle Ve, in step S1, the data (vehicle information) acquired for each vehicle Ve is transmitted to the server 101 by wireless communication. The vehicle information is acquired, for example, each time the ignition switch (not shown) is turned on (IG-ON) in each vehicle Ve. As described above, the transmission of the vehicle information from the control unit 6 of the vehicle Ve to the external server 101 is not limited to wireless communication, and may be performed using a wired communication device or communication facility.

[0048] Subsequently, in the server 101, in step S2, for the vehicle information received from each vehicle Ve, the ID number of each vehicle Ve (for example, VIN: Vehicle Identification Number set for each vehicle Ve) is assigned for each IG-ON time, and the vehicle information is stored in the server 101.

[0049] In step S3, the vehicle information acquired each time IG-ON is performed as described above is stored in server 101 as individual data for each vehicle Ve. That is, the individual data for each vehicle Ve is temporarily stored in server 101.

[0050] In step S4, for the data of each vehicle Ve, as shown in FIGS. 3 and 4 described above, matrix data M is generated by arranging time-series data of a predetermined item that changes continuously in time in the array direction (or the order direction of the array, or the column number direction) of each column. The time-series data in the embodiment of this invention is composed of first data that is a major factor in the convolution operation and second data that is a minor factor in the convolution operation. That is, the time-series data is composed of first data with a high degree of influence on the convolution operation and second data with a lower degree of influence on the convolution operation than the first data. In the example shown in FIG. 4 described above, the "starting voltage of battery 4" with a high degree of correlation with deterioration Against is arranged in matrix data M as the first data. In addition, the "travel distance of vehicle Ve", "outside air temperature", and "parking time of vehicle Ve" are arranged in matrix data M as the second data.

[0051] In step S5, a kernel K with a coefficient (kernel size) of "n" is read from the matrix data M. At that time, the first data is arranged every n rows of the kernel K. The second data is arranged in the remaining rows of the kernel K other than those where the first data of the kernel K is arranged. Note that the arrangement order in the row direction of each data (item) of the vehicle information in the matrix data M and the arrangement order in the row direction of each data (item) of the vehicle information in the kernel K do not necessarily have to match. In any case, when the kernel K is extracted from the matrix data M, the matrix data M and the kernel K are generated so that at least one row of the first data is necessarily included in the kernel K. In the example shown in FIG. 4 described above, a kernel K with a coefficient of "3" is used, and therefore, the matrix data M is generated so that at least one row of the first data is arranged every 3 rows.

[0052] In step S6, convolution operations and neural network operations are performed using the matrix data M and the kernel K generated as described above. That is, the operations by the convolutional neural network in the embodiment of the present invention are performed. In this convolutional neural network, for example, the matrix data M and the kernel K as described above are read out as two-dimensional data arrays for each vehicle Ve, and the number of batches of vehicle information and the number of kernels K are multiplied by the two-dimensional data to form a four-dimensional data array. Note that in this convolution operation, the kernel K is extracted from the entire matrix data M by sequentially moving (striding) the position of the kernel K with respect to the matrix data M, as indicated by the arrow in FIG. 6, for example.

[0053] Then, in step S7, learning is performed based on the operation result by the convolutional neural network executed in step S6 as described above. That is, the state of change of the vehicle Ve over time is estimated. For example, the deterioration state of the battery 4 (the state of change of the performance of the battery 4 over time) as described above is estimated. At that time, for example, as shown in FIG. 4 described above, the abnormal flag extraction range F is set according to the number of kernels K (stride number) in the matrix data M. As the stride number of the kernel K increases, the abnormal flag extraction range F also expands. Then, as described above, if an abnormality in the deterioration state of the battery 4 is detected as a result of estimating the state of change of the vehicle Ve over time, the abnormality detection flag is set to ON. When this step S7 is executed, then, the routine shown in the flowchart of FIG. 5 is once terminated.

[0054] As described above, in the learning system and learning device using the convolutional neural network in the embodiment of the present invention , the above-mentioned specificBy the method for generating matrix data for a convolutional neural network, matrix data M and kernel K used in the convolutional neural network are generated. Thereby, the first data (main factor) and the second data (sub-factor) as described above are comprehensively incorporated (convolved), and learning by convolutional operation is executed. In the example described above, the deterioration state of the battery 4 mounted on the vehicle Ve is the learning target, and the starting voltage of the battery 4 is taken into the convolutional neural network as the first data. Further, for example, factors other than the first data related to the deterioration state of the battery 4, such as the usage state and usage environment of the vehicle Ve, are taken into the convolutional neural network as the second data. Therefore, in the convolutional neural network according to the embodiment of the present invention, compared with the conventional learning using only the main factor such as the first data or the conventional learning with only a simple correction of the main factor, learning with higher accuracy can be performed.

[0055] Therefore, according to the learning system and learning device using the convolutional neural network according to the embodiment of the present invention, by applying the convolutional neural network, the change state of a device, member, or substance that changes continuously over time, for example, the deterioration state of the battery 4 mounted on the vehicle Ve, can be appropriately and accurately estimated.

Explanation of Signs

[0056] 1 Driving force source (POWER) 2 Driving wheels 3 Starter motor 4 Battery (power storage device) 5 Detection unit 5a Battery voltage sensor 5b Odometer sensor 5c Outside air temperature sensor 5d Battery temperature sensor 5e Timer 5f Counter 5g Vehicle speed sensor (or wheel speed sensor) 5h Rotation speed sensor 6 Control Unit (ECU) 6a Data acquisition section (of the control unit) 6b Transmission data creation section (of the control unit) 7 Communication module (DCM) 8 Engine (driving power source) 9 Transmission 10 Differential gear 11 Drive shaft 101 Server 101a Data storage section (of the server) 101b Data generation section (of the server) 101c Calculation section (of the server) 101d Learning section (of the server) F Abnormal flag extraction range K Kernel (or filter, or mask) M Matrix data Ve Vehicle.

Claims

1. A learning system using a convolutional neural network that includes a control unit mounted on a vehicle and a server installed outside the vehicle, and that executes a convolutional operation based on matrix data in which predetermined information is arranged as components of a matrix, and estimates a state of change over time of a power storage device mounted on the vehicle, wherein the control unit, acquires vehicle information detecting behaviors and states of respective parts of the vehicle, and transmits the vehicle information to the server, wherein the server, constitutes the matrix data with time-series data of the vehicle information in which each row of the matrix data changes continuously in time in the arrangement direction of each column of the matrix data, constitutes the time-series data with first data related to a starting voltage of the power storage device output to start an engine, and second data including at least any one of a travel distance of the vehicle, an outside air temperature, a temperature of the power storage device, a travel time of the vehicle, a parking time of the vehicle, and a number of times of starting the engine, executes the convolutional operation using a kernel that partitions the matrix data by the rows and the columns by a predetermined coefficient, and arranges at least one row of the first data for each row by the coefficient, arranges the second data in the remaining rows excluding the rows arranging the first data, and executes the convolutional operation using the arranged first data and second data to estimate the state of change over time, characterized in that it is a learning system using a convolutional neural network.

2. A learning device using a convolutional neural network that executes a convolutional operation based on matrix data in which vehicle information detecting behaviors and states of respective parts of a vehicle is arranged as components of a matrix, and estimates a state of change over time of a power storage device mounted on the vehicle, a detection unit that detects behaviors and states of the respective parts, and a data acquisition unit that acquires, as the vehicle information, data detected by the detection unit, The time-series data of the vehicle information in which each row of the matrix data changes continuously in time in the array direction of each column of the matrix data constitutes the matrix data, and the first data related to the starting voltage of the power storage device output for starting the engine, and the second data including at least any one of the driving distance of the vehicle, the outside air temperature, the temperature of the power storage device, the driving time of the vehicle, the parking time of the vehicle, and the number of engine starts constitute the time-series data, and for each row of the coefficient of the kernel that partitions the matrix data by the coefficient of the rows and columns, at least one row of the first data is arranged, and the second data is arranged in the remaining rows excluding the rows in which the first data is arranged, a data generation unit; An arithmetic unit that executes the convolution operation using the arranged first data, the second data, and the kernel; A learning unit that estimates the state of change over time of the power storage device based on the result of executing the convolution operation. A learning device using a convolutional neural network, characterized by the above.

Images