Whole vehicle thermal management system based on reinforcement learning control

Abstract

The invention relates to a whole vehicle thermal management system based on reinforcement learning control. A first thermal management module is arranged to execute thermal management of a passenger compartment, and a second thermal management module is arranged to execute thermal management of a driving mechanism; a third heat management module is arranged to execute heat management of the driving battery, and a first heat exchanger is arranged to adjust hot air distribution between the first heat management module and the third heat management module; a second heat exchanger is arranged to adjust hot air distribution between the first heat management module and the second heat management module. On the basis, a control module is further arranged as a control core, the first heat management module, the second heat management module, the third heat management module, the first heat exchanger and the second heat exchanger are controlled to operate through a reinforcement learning mechanism, it can be guaranteed that energy is reasonably utilized, the situation of energy waste is avoided, and then the use efficiency of the pure electric vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the field of thermal management control technology, and in particular to a vehicle thermal management system based on reinforcement learning control. Background Art

[0002] Electric vehicles, with their advantages of low energy consumption and zero emissions, have become an important means to achieve future energy conservation goals. The control strategy of the thermal management system is the key to ensuring the thermal comfort of the electric vehicle cabin and the thermal safety of the battery and motor. However, due to the limitation of battery energy density, the driving range of electric vehicles is relatively short, especially when heating loads are required in cold weather.

[0003] Electric vehicles do not have engine waste heat for heating. In cold environments, the energy consumption of air conditioning accounts for a large proportion (more than 30%) of the total energy consumption provided by the battery. Therefore, improving the energy efficiency of air conditioning and reducing heating energy consumption are of great significance to extending the driving range of electric vehicles.

[0004] However, in the prior art, the thermal management module of the passenger compartment of electric vehicles, the thermal management module of the power battery, and the thermal management module of the drive motor all have technical problems such as unreasonable energy utilization and high energy waste, which makes it impossible to fully utilize the thermal management energy of the entire vehicle. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to overcome the technical problems of unreasonable energy utilization and high energy waste in the passenger compartment thermal management module, power battery thermal management module, and drive motor thermal management module in electric vehicles. In order to overcome the defects of the above-mentioned prior art, the present invention provides a whole vehicle thermal management system based on reinforcement learning control.

[0006] The present invention provides a vehicle thermal management system based on reinforcement learning control, comprising a first solenoid valve, a second solenoid valve and a third solenoid valve. In addition, the vehicle thermal management system further comprises:

[0007] a first thermal management module for conveying thermal airflow to the passenger compartment;

[0008] A second thermal management module, used for conveying hot air flow to drive the motor;

[0009] A third thermal management module, used to deliver hot air flow to drive the battery;

[0010] a first heat exchanger, configured to adjust the distribution of hot air flow between the first thermal management module and the third thermal management module;

[0011] a second heat exchanger, configured to adjust the distribution of hot air flow between the first thermal management module and the second thermal management module;

[0012] a control module configured to control the operation of the first solenoid valve, the second solenoid valve, the third solenoid valve, the first heat exchanger and the second heat exchanger according to the difference between the current temperature of the passenger compartment and the target temperature and the operating temperature range of the drive motor through a reinforcement learning mechanism, so as to adjust the flow order of the thermal airflow;

[0013] in,

[0014] The first solenoid valve and the second solenoid valve are configured to adjust the flow direction of the hot air delivered by the first thermal management module and the second thermal management module, and the first solenoid valve and the third solenoid valve are configured to adjust the flow direction of the hot air delivered by the first thermal management module and the third thermal management module;

[0015] The outlet of the second solenoid valve is communicated with the inlet of the first solenoid valve, the outlet of the first solenoid valve (13) is communicated with the inlet of the third solenoid valve and the first thermal management module, the inlet of the second solenoid valve is communicated with the second heat exchanger, and the outlet of the third solenoid valve is communicated with the first heat exchanger;

[0016] The first thermal management module is in communication with the third thermal management module via the first heat exchanger, and the first thermal management module is in communication with the second thermal management module via the second heat exchanger;

[0017] The control module is electrically connected to the first solenoid valve, the second solenoid valve, the third solenoid valve, the first heat exchanger, and the second heat exchanger.

[0018] The vehicle thermal management system based on reinforcement learning control disclosed in the present invention aims at the above problems. By setting a first thermal management module to deliver hot air flow to perform thermal management of the passenger compartment, setting a second thermal management module to deliver hot air flow to perform thermal management of the drive mechanism; setting a third thermal management module to deliver hot air flow to perform thermal management of the drive battery, and also by setting a first heat exchanger to adjust the distribution of hot air flow between the first thermal management module and the third thermal management module; setting a second heat exchanger to adjust the distribution of hot air flow between the first thermal management module and the second thermal management module. On this basis, by setting a control module as the control core, the first thermal management module, the second thermal management module, the third thermal management module, the first heat exchanger and the second heat exchanger are controlled by the reinforcement learning mechanism to adjust the hot air flow and optimize the thermal management. Since the control module uses the reinforcement learning control mechanism, the energy of each system of the thermal management of the passenger compartment of the pure electric vehicle, the thermal management of the power battery, and the thermal management of the drive motor can be cascaded, ensuring reasonable energy utilization and avoiding energy waste, thereby increasing the use efficiency of the pure electric vehicle. Moreover, while ensuring these thermal management requirements, it can also make rational use of waste heat to strengthen the learning mechanism to achieve the purpose of optimizing passenger compartment comfort or optimizing mileage.

[0019] In a possible implementation, the first thermal management module includes a compressor, a condenser, and an evaporator;

[0020] The compressor is electrically connected to the control module;

[0021] The outlet of the compressor is communicated with the inlet of the condenser, the outlet of the condenser is communicated with the inlet of the first solenoid valve, and the outlet of the first solenoid valve is communicated with the inlet of the evaporator;

[0022] The first heat exchanger is in communication with an inlet of the compressor;

[0023] The inlet of the second solenoid valve is communicated with the outlet of the condenser, the outlet of the second solenoid valve is communicated with the second heat exchanger, and the second heat exchanger is communicated with the outlet of the evaporator;

[0024] This solution can ensure that the thermal management of the passenger compartment is carried out in an orderly manner, and by electrically connecting the compressor to the control module, it can further ensure the controllability of the thermal management of the passenger compartment and avoid energy waste.

[0025] In a possible implementation manner, the second thermal management module includes a drive motor, a first water pump, a cooling fan and a first three-way valve;

[0026] The second heat exchanger is provided with four interfaces, the first interface of the second heat exchanger is communicated with the third interface, and the second interface of the second heat exchanger is communicated with the fourth interface;

[0027] The first interface of the second heat exchanger is in communication with the outlet of the evaporator, and the second interface of the second heat exchanger is in communication with the outlet of the second solenoid valve;

[0028] The third interface of the second heat exchanger is communicated with the air inlet of the drive motor, the air outlet of the drive motor is communicated with the inlet of the first water pump, the air inlet and the air outlet of the drive motor are communicated through a pipeline, the outlet of the first water pump is communicated with the first valve port of the first three-way valve, and the second valve port of the first three-way valve is communicated with the fourth interface of the second heat exchanger;

[0029] The inlet of the cooling fan is communicated with the outlet of the first water pump, and the outlet of the cooling fan is communicated with the second valve port of the first three-way valve;

[0030] This solution can ensure the orderly and efficient operation of the thermal management of the drive mechanism, help avoid energy waste during the thermal management process of the drive mechanism, and further increase the efficiency of pure electric vehicles.

[0031] In a possible implementation, the third thermal management module includes a second water pump, a drive battery, a PTC heater, and a second three-way valve;

[0032] The PTC heater is electrically connected to the control module;

[0033] The first heat exchanger is provided with four interfaces, the first interface of the first heat exchanger is communicated with the fourth interface, and the second interface of the first heat exchanger is communicated with the third interface;

[0034] The first interface of the first heat exchanger is communicated with the inlet of the compressor, and the second interface of the first heat exchanger is communicated with the outlet of the third solenoid valve;

[0035] The third interface of the first heat exchanger is communicated with the first valve port of the second three-way valve, and the second valve port of the second three-way valve is communicated with the fourth interface of the first heat exchanger;

[0036] The third valve port of the second three-way valve is communicated with the inlet of the second water pump, the outlet of the second water pump is communicated with the air inlet of the driving battery, the exhaust port of the driving battery is communicated with the inlet of the PTC heater, and the outlet of the PTC heater is communicated with the fourth interface of the first heat exchanger;

[0037] This solution can ensure the orderly and efficient operation of the drive battery thermal management, help avoid energy waste during the drive battery thermal management process, and further increase the efficiency of pure electric vehicles.

[0038] In a possible implementation manner, the vehicle thermal management system further includes a four-way reversing valve, which is electrically connected to the control module so as to adjust the connection mode between the interface a, the interface b, the interface c and the interface d of the four-way reversing valve through the control module;

[0039] The interface a of the four-way reversing valve is respectively connected to the first valve port of the first three-way valve and the inlet of the cooling fan;

[0040] The interface b of the four-way reversing valve is connected to the outlet of the first water pump;

[0041] The interface c of the four-way reversing valve is in communication with the fourth interface of the first heat exchanger;

[0042] The interface d of the four-way reversing valve is communicated with the inlet of the PTC heater;

[0043] This solution establishes a connection between the second thermal management module and the third thermal management module by setting a four-way reversing valve, thereby ensuring controllable switching between thermal management of the drive mechanism and thermal management of the drive battery.

[0044] In a possible implementation, the control module is configured to perform control by executing the following steps:

[0045] A1: obtaining the current temperature in the passenger compartment, determining whether the difference between the current temperature and the target temperature is less than a first preset value, and if so, suspending control; if not, executing the next step;

[0046] A2: Determine whether the drive motor is operating at a suitable temperature. If yes, turn off the PTC heater and connect the four-way reversing valve interface a to interface d, and interface b to interface c, and then execute the next step; if no, execute step A5;

[0047] A3: Open the first solenoid valve and the second solenoid valve, and then proceed to the next step;

[0048] A4: obtaining the current temperature in the passenger compartment, and determining whether the difference between the current temperature in the passenger compartment and the target temperature is not less than the first preset value but less than the second preset value;

[0049] If yes, close the first solenoid valve, open the second solenoid valve, and then execute step A7;

[0050] If not, then go back to step A3;

[0051] A5: Open the first solenoid valve, close the second solenoid valve, and then proceed to the next step;

[0052] A6: Determine whether the driving battery is operating within its suitable temperature range;

[0053] If yes, then go back to step A5;

[0054] If not, connect the interface a of the four-way reversing valve to the interface b, connect the interface c to the interface d, start the PTC heater, and then return to execute step A2;

[0055] A7: Calculate the difference between the temperature in the passenger compartment and the target temperature within a predetermined time, and obtain the integral value of the difference within the predetermined time by a numerical integration calculation method, and obtain the current energy consumption of the compressor; and use the integral value and the current energy consumption as the current state;

[0056] A8: According to the reinforcement learning mechanism, the current state is used to obtain a reward result through the stored reward function;

[0057] A9: According to the reward result, the speed of the compressor is adjusted once, and the air intake volume of the first heat exchanger and the second heat exchanger are adjusted once respectively;

[0058] A10: Obtaining the current temperature in the passenger compartment, and determining whether the difference between the current temperature and the target temperature is less than a first preset value;

[0059] If yes, then go back to step A2;

[0060] If not, then go back to step A7;

[0061] By setting the control module to execute the above steps, it is possible to reasonably utilize waste heat while ensuring the thermal management requirements of each system, so as to achieve the purpose of optimizing passenger compartment comfort or optimizing mileage.

[0062] In a possible implementation manner, the suitable temperature in step A2 is (35-65)°C.

[0063] In a possible implementation, the reward function in step A8 is set to:

[0064] A positive reward is given when the temperature in the passenger compartment at this time falls within a specified comfortable range or the energy consumption of the compressor is reduced, otherwise a negative reward is given;

[0065] The program's incentive structure helps strike a balance between maintaining a comfortable cabin temperature and minimizing energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of the structure of a vehicle thermal management system based on reinforcement learning control disclosed in an embodiment of the present invention;

[0067] Figure 2 This is a flow chart of the operation of the control module disclosed in the embodiment of the present invention.

[0068] Description of reference numerals:

[0069] 1. Compressor, 2. Condenser, 3. Evaporator, 4. First heat exchanger, 5. Second heat exchanger, 6. Drive motor, 7. First water pump, 8. Second water pump, 9. Drive battery, 10. PTC heater, 11. Cooling fan, 12. Four-way reversing valve, 13. First solenoid valve, 14. Second solenoid valve, 15. Third solenoid valve, 16. First three-way valve, 17. Second three-way valve, 18. Control module. DETAILED DESCRIPTION

[0070] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0071] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense, and should be understood as the existence of a heat exchange relationship between the two connected. For example, it can be a fixed pipeline connection, or a connection in which the pipeline is detachably connected, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0072] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "electrical connection" and "electrical connection relationship" should be understood in a broad sense, that is, referring to a connection method with an electrical relationship, for example, it can be a circuit connection through a conductive wire, or it can be an electrical connection through a radio signal channel (channel), or a combination of the two. In addition, "electrical connection" and "electrical connection relationship" can be based on mechanical connection (such as a conductive wire set in a connecting key); it can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0073] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0074] The present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] See also Figure 1-2 As shown, the embodiment of the present application discloses a vehicle thermal management system based on reinforcement learning control. Figure 1As a structural diagram, the vehicle thermal management system includes a first solenoid valve 13, a second solenoid valve 14, a third solenoid valve 15, a first thermal management module, a second thermal management module, a third thermal management module, a first heat exchanger 4, a second heat exchanger 5 and a control module 18, wherein the first thermal management module is connected to the third thermal management module through the first heat exchanger 4, and the first thermal management module is connected to the second thermal management module through the second heat exchanger 5; the control module 18 is electrically connected to the first solenoid valve 13, the second solenoid valve 14, the third solenoid valve 15, the first heat exchanger 4 and the second heat exchanger 5. Thus, the first solenoid valve 13 and the second solenoid valve 14 can control the distribution of thermal airflow between the motor and the passenger compartment. The first solenoid valve 13 and the third solenoid valve 15 can control the distribution of thermal airflow between the battery and the passenger compartment.

[0076] See also Figure 1 As shown, in this embodiment, the first thermal management module is mainly controlled by the first solenoid valve 13, and is used to transport the hot air flow of the passenger compartment to perform thermal management of the passenger compartment. The first thermal management module includes a compressor 1, a condenser 2 and an evaporator 3; the compressor 1 is electrically connected to the control module 18; the outlet of the compressor 1 is connected to the inlet of the condenser 2, the outlet of the condenser 2 is connected to the inlet of the first solenoid valve 13, the outlet of the first solenoid valve 13 is connected to the inlet of the evaporator 3, and the outlet of the first solenoid valve 13 is also connected to the inlet of the third solenoid valve 15; the outlet of the third solenoid valve 15 is connected to the first heat exchanger 4, and the first heat exchanger 4 is connected to the inlet of the compressor 1; the outlet of the second solenoid valve 14 is connected to the outlet of the condenser 2 or to the inlet of the first solenoid valve 13, the inlet of the second solenoid valve 14 is connected to the second heat exchanger 5, and the second heat exchanger 5 is connected to the outlet of the evaporator 3.

[0077] Please continue to see Figure 1 As shown, in this embodiment, the second thermal management module is mainly controlled by the second solenoid valve 14 and is used to deliver the hot air flow of the drive motor 6 to perform thermal management of the drive motor 6 . The second thermal management module includes a driving motor 6, a first water pump 7, a cooling fan 11 and a first three-way valve 16; the second heat exchanger 5 is provided with four interfaces, the first interface of the second heat exchanger 5 is connected to the third interface, and the second interface of the second heat exchanger 5 is connected to the fourth interface, wherein the first interface of the second heat exchanger 5 is connected to the outlet of the evaporator 3, and the second interface of the second heat exchanger 5 is connected to the inlet of the second solenoid valve 14; the third interface of the second heat exchanger 5 is connected to the air inlet of the driving motor 6, the air outlet of the driving motor 6 is connected to the inlet of the first water pump 7, the air inlet and the air outlet of the driving motor 6 are connected through a pipeline, the outlet of the first water pump 7 is connected to the first valve port of the first three-way valve 16, and the second valve port of the first three-way valve 16 is connected to the fourth interface of the second heat exchanger 5; the inlet of the cooling fan 11 is connected to the outlet of the first water pump 7, and the outlet of the cooling fan 11 is connected to the second valve port of the first three-way valve 16.

[0078] Please continue to see Figure 1 As shown, in this embodiment, the third thermal management module is controlled by the third solenoid valve 15 and is used to perform thermal management of the drive battery. The third thermal management module includes a second water pump 8, a drive battery 9, a PTC heater 10 and a second three-way valve 17; the PTC heater 10 is electrically connected to the control module 5. The first heat exchanger 4 is provided with four interfaces, the first interface of the first heat exchanger 4 is connected to the fourth interface, and the second interface of the first heat exchanger 4 is connected to the third interface, wherein the first interface of the first heat exchanger 4 is connected to the inlet of the compressor 1, and the second interface of the first heat exchanger 4 is connected to the outlet of the third solenoid valve 15; the third interface of the first heat exchanger 4 is connected to the first valve port of the second three-way valve 17, and the second valve port of the second three-way valve 17 is connected to the fourth interface of the first heat exchanger 4; the third valve port of the second three-way valve 17 is connected to the inlet of the second water pump 8, the outlet of the second water pump 8 is connected to the air inlet of the drive battery 9, the exhaust port of the drive battery 9 is connected to the inlet of the PTC heater 10, and the outlet of the PTC heater 10 is connected to the fourth interface of the first heat exchanger 4.

[0079] In addition, please continue to see Figure 1 As shown, in this embodiment, the vehicle thermal management system further includes a four-way reversing valve 12, which is electrically connected to the control module 18, so as to adjust the connection mode between the interface a, interface b, interface c and interface d of the four-way reversing valve 12 through the control module 18. The interface a of the four-way reversing valve 12 is respectively connected to the first valve port of the first three-way valve 16 and the inlet of the cooling fan 11; the interface b of the four-way reversing valve 12 is connected to the outlet of the first water pump 7; the interface c of the four-way reversing valve 12 is connected to the fourth interface of the first heat exchanger 4; and the interface d of the four-way reversing valve 12 is connected to the inlet of the PTC heater 10.

[0080] See also Figure 1 and Figure 2 As shown, in this embodiment, the control module 18 is configured to control the operation of the first solenoid valve 13, the second solenoid valve 14, the third solenoid valve 15, the first heat exchanger 4, the PTC heater 10 and the second heat exchanger 5 through a reinforcement learning mechanism according to the difference between the current temperature of the passenger compartment and the target temperature and the operating temperature range of the drive mechanism to optimize thermal management. It includes a temperature sensor, and the module is specifically configured to perform control by executing the following steps:

[0081] A1: obtaining the current temperature in the passenger compartment by calling the temperature sensor contained therein, and determining whether the difference between the current temperature and the target temperature is less than a first preset value. If so, suspending control; if not, executing the next step.

[0082] A2: Determine whether the drive motor 6 is operating at an appropriate temperature, which is (35-65)°C. If yes, turn off the PTC heater 10, connect the port a of the four-way reversing valve 12 to the port d, and the port b to the port c, and then execute the next step; if no, execute step A5.

[0083] A3: Open the first solenoid valve 13 and the second solenoid valve 14, and then proceed to the next step.

[0084] A4: Obtaining the current temperature in the passenger compartment, and determining whether the difference between the current temperature in the passenger compartment and the target temperature is not less than a first preset value but less than a second preset value;

[0085] If yes, close the first solenoid valve 13, open the second solenoid valve 14, and then execute step A7;

[0086] If not, the process returns to step A3.

[0087] A5: Open the first solenoid valve 13, close the second solenoid valve 14, and then proceed to the next step.

[0088] A6: Determine whether the driving battery 9 is operating within its suitable temperature range;

[0089] If yes, go back to step A5;

[0090] If not, connect the port a of the four-way reversing valve 12 to the port b, connect the port c to the port d, start the PTC heater 10, and then return to step A2.

[0091] A7: Calculate the difference between the five passenger compartment temperatures and the target temperature within the next 5 seconds, and obtain the integral value of the difference within the predetermined time by a numerical integration method, and simultaneously obtain the current energy consumption of compressor 1; use the integral value and the current energy consumption as the current state.

[0092] A8: According to the reinforcement learning mechanism, the current state is used to obtain the reward result through the stored reward function. The reward function in this embodiment is set to give a positive reward when the temperature in the passenger compartment at this time falls within the specified comfort range or the energy consumption of the compressor 1 decreases, otherwise a negative reward is given.

[0093] A9: According to the reward result, the speed of the compressor 1 is adjusted once, and the air intake volume of the first heat exchanger 4 and the second heat exchanger 5 are adjusted once respectively.

[0094] A10: Obtain the current temperature in the passenger compartment, and determine whether the difference between the current temperature and the target temperature is less than a first preset value;

[0095] If yes, go back to step A2;

[0096] If not, the process returns to step A7.

[0097] So far, under the action of the control module 18, when the drive motor 6 works at a suitable temperature (35-65°C) and the passenger compartment temperature does not reach the target temperature, the dual heat source heat pump mode can be started, that is, the first solenoid valve 13 is opened to allow the passenger compartment heat pump air conditioner to continue to heat the passenger compartment, and the second solenoid valve 14 is opened to allow the waste heat generated by the drive motor 6 to be supplied to the passenger compartment through the second heat exchanger 5. When the drive motor 6 does not work at a suitable temperature, the heat pump air conditioning system is started, that is, the first solenoid valve 13 is opened to allow the heat pump air conditioner to heat the passenger compartment, and the second solenoid valve 14 is closed to turn off the motor waste heat recovery mode. When the passenger compartment temperature is close to the target temperature and the drive motor 6 works at a suitable temperature, the waste heat recovery mode of the drive motor 6 is started, that is, the first solenoid valve 13 is closed to turn off the heat pump air conditioner, and the second solenoid valve 14 is opened to continue to use the waste heat generated by the drive motor 6 to heat the passenger compartment, thereby satisfying the comfort of the passenger compartment and optimizing energy.

[0098] from Figure 1 It can be seen that the four-way reversing valve 12 can control the hot air flow distribution between the driving battery 9 and the driving motor 6. When the driving battery 9 does not reach the appropriate working temperature and the driving motor 6 does not reach the appropriate working temperature, the a and b interfaces of the four-way reversing valve 12 are connected, the c and d interfaces are connected, and the PTC heater 10 is started to provide heat for the driving battery 9. When the driving motor 6 reaches the appropriate working temperature, the PTC heater 10 is turned off, the a and d interfaces of the four-way reversing valve 12 are connected, and the b and c interfaces are connected, so that the waste heat generated by the driving motor 6 is supplied to the driving battery 9 for use, thereby achieving the purpose of optimizing energy.

[0099] In summary, the vehicle thermal management system disclosed in this embodiment performs thermal management of the passenger compartment by setting a first thermal management module, performs thermal management of the drive mechanism by setting a second thermal management module; performs thermal management of the drive battery by setting a third thermal management module, adjusts the thermal airflow distribution between the first thermal management module and the third thermal management module by setting a first heat exchanger 4; and adjusts the thermal airflow distribution between the first thermal management module and the second thermal management module by setting a second heat exchanger 5. On this basis, by setting a control module 18 as the control core, the operation of the first thermal management module, the second thermal management module, the third thermal management module, the first heat exchanger and the second heat exchanger is controlled by a reinforcement learning mechanism to optimize thermal management. In addition, the energy of each system of thermal management of the passenger compartment of pure electric vehicles, thermal management of power batteries, and thermal management of drive motors can be utilized in stages to ensure reasonable energy utilization and avoid energy waste, thereby increasing the use efficiency of pure electric vehicles. In addition, under the guarantee of these thermal management requirements, waste heat can be reasonably utilized to achieve the purpose of optimizing the comfort of the passenger compartment or optimizing the mileage by strengthening the learning mechanism.

[0100] In the description of the embodiments of the present application, it should be noted that in the description of the present application, terms such as "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is only for the convenience of description, and does not indicate or imply that the device or component must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present application.

[0101] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0102] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A vehicle thermal management system based on reinforcement learning control, characterized in that: It comprises a first solenoid valve (13), a second solenoid valve (14) and a third solenoid valve (15); The vehicle thermal management system further includes: a first thermal management module for conveying thermal airflow to the passenger compartment; a second thermal management module, for conveying a hot air flow to drive the motor (6); a third thermal management module, used for conveying a hot air flow of the driving battery (9); A first heat exchanger (4) for adjusting the distribution of hot air flow between the first thermal management module and the third thermal management module; a second heat exchanger (5), used for adjusting the distribution of hot air flow between the first thermal management module and the second thermal management module; A control module (18) is configured to control the operation of the first solenoid valve (13), the second solenoid valve (14), the third solenoid valve (15), the first heat exchanger (4) and the second heat exchanger (5) according to the difference between the current temperature of the passenger cabin and the target temperature and the operating temperature range of the drive motor (6) through a reinforcement learning mechanism, so as to adjust the flow order of the thermal airflow; in, The first solenoid valve (13) and the second solenoid valve (14) are configured to adjust the flow direction of the hot air flow delivered by the first thermal management module and the second thermal management module, and the first solenoid valve (13) and the third solenoid valve (15) are configured to adjust the flow direction of the hot air flow delivered by the first thermal management module and the third thermal management module; The outlet of the second solenoid valve (14) is communicated with the inlet of the first solenoid valve (13), the outlet of the first solenoid valve (13) is communicated with the inlet of the third solenoid valve (15) and the first thermal management module, the inlet of the second solenoid valve (14) is communicated with the second heat exchanger (5), and the outlet of the third solenoid valve (15) is communicated with the first heat exchanger (4); The first thermal management module is in communication with the third thermal management module via the first heat exchanger (4), and the first thermal management module is in communication with the second thermal management module via the second heat exchanger (5); The control module (18) is electrically connected to the first solenoid valve (13), the second solenoid valve (14), the third solenoid valve (15), the first heat exchanger (4) and the second heat exchanger (5).

2. The vehicle thermal management system based on reinforcement learning control according to claim 1 is characterized in that: The first thermal management module comprises a compressor (1), a condenser (2) and an evaporator (3); The compressor (1) is electrically connected to the control module (18); The outlet of the compressor (1) is communicated with the inlet of the condenser (2), the outlet of the condenser (2) is communicated with the inlet of the first solenoid valve (13), and the outlet of the first solenoid valve (13) is communicated with the inlet of the evaporator (3); The first heat exchanger (4) is in communication with an inlet of the compressor (1); The inlet of the second solenoid valve (14) is in communication with the second heat exchanger (5), and the second heat exchanger (5) is in communication with the outlet of the evaporator (3).

3. The vehicle thermal management system based on reinforcement learning control according to claim 2 is characterized in that: The second thermal management module comprises the drive motor (6), a first water pump (7), a cooling fan (11) and a first three-way valve (16); The second heat exchanger (5) is provided with four interfaces, the first interface of the second heat exchanger (5) is in communication with the third interface, and the second interface of the second heat exchanger (5) is in communication with the fourth interface; The first interface of the second heat exchanger (5) is in communication with the outlet of the evaporator (3), and the second interface of the second heat exchanger (5) is in communication with the inlet of the second solenoid valve (14); The third interface of the second heat exchanger (5) is communicated with the air inlet of the drive motor (6), the air outlet of the drive motor (6) is communicated with the air inlet of the first water pump (7), the air inlet and the air outlet of the drive motor (6) are communicated through a pipeline, the outlet of the first water pump (7) is communicated with the first valve port of the first three-way valve (16), and the second valve port of the first three-way valve (16) is communicated with the fourth interface of the second heat exchanger (5); The inlet of the cooling fan (11) is communicated with the outlet of the first water pump (7), and the outlet of the cooling fan (11) is communicated with the second valve port of the first three-way valve (16).

4. The vehicle thermal management system based on reinforcement learning control according to claim 3 is characterized in that: The third thermal management module comprises a second water pump (8), the driving battery (9), a PTC heater (10) and a second three-way valve (17); The PTC heater (10) is electrically connected to the control module (5); The first heat exchanger (4) is provided with four interfaces, the first interface of the first heat exchanger (4) is in communication with the fourth interface, and the second interface of the first heat exchanger (4) is in communication with the third interface; The first interface of the first heat exchanger (4) is in communication with the inlet of the compressor (1), and the second interface of the first heat exchanger (4) is in communication with the outlet of the third solenoid valve (15); The third interface of the first heat exchanger (4) is in communication with the first valve port of the second three-way valve (17), and the second valve port of the second three-way valve (17) is in communication with the fourth interface of the first heat exchanger (4); The third valve port of the second three-way valve (17) is connected to the inlet of the second water pump (8), the outlet of the second water pump (8) is connected to the air inlet of the driving battery (9), the exhaust port of the driving battery (9) is connected to the inlet of the PTC heater (10), and the outlet of the PTC heater (10) is connected to the fourth interface of the first heat exchanger (4).

5. The vehicle thermal management system based on reinforcement learning control according to claim 4 is characterized in that: The vehicle thermal management system further comprises a four-way reversing valve (12), wherein the four-way reversing valve (12) is electrically connected to the control module (18) so as to adjust the connection mode between the interface a, the interface b, the interface c and the interface d of the four-way reversing valve (12) through the control module (18); The interface a of the four-way reversing valve (12) is respectively connected to the first valve port of the first three-way valve (16) and the inlet of the cooling fan (11); The interface b of the four-way reversing valve (12) is in communication with the outlet of the first water pump (7); The interface c of the four-way reversing valve (12) is in communication with the fourth interface of the first heat exchanger (4); The interface d of the four-way reversing valve (12) is in communication with the inlet of the PTC heater (10).

6. The vehicle thermal management system based on reinforcement learning control according to claim 5 is characterized in that: The control module (18) is configured to perform control by executing the following steps: A1: obtaining the current temperature in the passenger compartment, determining whether the difference between the current temperature and the target temperature is less than a first preset value, and if so, suspending control; if not, executing the next step; A2: Determine whether the drive motor (6) is operating at an appropriate temperature. If so, turn off the PTC heater (10) and connect the interface a of the four-way reversing valve (12) to the interface d, and the interface b to the interface c, and then execute the next step; if not, execute step A5; A3: Open the first solenoid valve (13) and the second solenoid valve (14), and then proceed to the next step; A4: obtaining the current temperature in the passenger compartment, and determining whether the difference between the current temperature in the passenger compartment and the target temperature is not less than the first preset value but less than the second preset value; If yes, close the first solenoid valve (13), open the second solenoid valve (14), and then execute step A7; If not, then go back to step A3; A5: Open the first solenoid valve (13), close the second solenoid valve (14), and then proceed to the next step; A6: Determine whether the driving battery (9) is operating within its suitable temperature range; If yes, then go back to step A5; If not, then connect the interface a of the four-way reversing valve (12) to the interface b, connect the interface c to the interface d, start the PTC heater (10), and then return to execute step A2; A7: within a predetermined time thereafter, calculating the difference between a plurality of temperatures in the passenger compartment and the target temperature, and obtaining the integral value of the difference within the predetermined time by a numerical integration calculation method, and simultaneously obtaining the current energy consumption of the compressor (1); and taking the integral value and the current energy consumption as the current state; A8: According to the reinforcement learning mechanism, the current state is used to obtain a reward result through the stored reward function; A9: According to the reward result, the speed of the compressor (1) is adjusted once, and the air intake volume of the first heat exchanger (4) and the second heat exchanger (5) are adjusted once respectively; A10: Obtaining the current temperature in the passenger compartment, and determining whether the difference between the current temperature and the target temperature is less than a first preset value; If yes, then go back to step A2; If not, the process returns to step A7.

7. The vehicle thermal management system based on reinforcement learning control according to claim 6 is characterized in that: The suitable temperature in step A2 is (35-65)°C.

8. The vehicle thermal management system based on reinforcement learning control according to claim 6 or 7, characterized in that: The reward function in step A8 is set to: When the temperature in the passenger compartment at this time falls within a specified comfortable range or the energy consumption of the compressor (1) is reduced, a positive reward is given; otherwise, a negative reward is given.

Images