Control method of vehicle thermal management system and electronic equipment

Abstract

The invention provides a control method of a vehicle thermal management system and electronic equipment, and the method comprises the steps: obtaining a current-frame depth map and a current-frame color map collected by a camera in a vehicle for a preset riding area, and a passenger physiological signal collected by a radar sensor, and further obtaining a vehicle thermal management result according to the current-frame depth map and the current-frame color map; the relative distance between the target passenger and the radar sensor is determined according to the passenger physiological signal, the current frame physiological feature of the target passenger is determined according to the passenger physiological signal, the working mode of the thermal management system is determined according to the relative distance and the current frame physiological feature, and the temperature offset in the working mode is determined according to the current frame physiological feature. The thermal management system is controlled, multi-modal data fusion is achieved, the temperature control requirement of a passenger is dynamically determined, and the comfort level and energy efficiency of the passenger are improved.

Description

Technical Field

[0001] The present application relates to the technical field of vehicle thermal management systems, and in particular to a control method and electronic equipment for a vehicle thermal management system. Background Art

[0002] Currently, in vehicle thermal management systems, zoned temperature control can be implemented to improve the comfort of different occupants in the same vehicle.

[0003] Traditional zoned temperature control typically uses fixed zones for different temperature settings, such as left and right zones for the front row and independent control for the rear row, with corresponding temperature settings for each zone. However, this approach cannot dynamically adjust to the occupants' actual position and needs, which can affect their comfort. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, the present application aims to provide a control method and electronic equipment for a vehicle thermal management system to solve the problem in the related art that the thermal management system cannot be dynamically controlled according to the actual position and actual needs of the occupants, thereby improving the comfort of the occupants.

[0005] An embodiment of the present application provides a method for controlling a vehicle thermal management system, the method comprising: Obtaining a current frame depth image and a current frame color image collected by a camera in the vehicle for a preset seating area, and an occupant physiological signal collected by a radar sensor for the preset seating area; determining a relative distance between a target occupant and the radar sensor based on the current frame depth map and the current frame color map, and determining a current frame physiological feature of the target occupant based on the occupant physiological signal; An operating mode of the thermal management system in the vehicle is determined based on the relative distance and the current frame physiological characteristics, and a temperature offset in the operating mode is determined based on the current frame physiological characteristics.

[0006] Optionally, determining the relative distance between the target occupant and the radar sensor based on the current frame depth map and the current frame color map includes: Determine a 3D bounding box of the target occupant based on the current frame depth map and the current frame color map; Determine the coordinates of key points in the 3D bounding box, and convert the key point coordinates from the world coordinate system to the coordinate system of the radar sensor; The relative distance between the target occupant and the radar sensor is determined according to the converted key point coordinates.

[0007] Optionally, determining a 3D bounding box of the target occupant based on the current frame depth map and the current frame color map includes: Converting the current frame depth map into a three-dimensional point cloud; Projecting a three-dimensional coordinate point in the three-dimensional point cloud onto the current frame color image to obtain a pixel coordinate point corresponding to the three-dimensional coordinate point, and determining a pixel value of the pixel coordinate point as the pixel value corresponding to the three-dimensional coordinate point; A 3D bounding box of the target occupant is determined based on each 3D coordinate point and the corresponding pixel value in the 3D point cloud.

[0008] Optionally, before converting the current frame depth map into a three-dimensional point cloud, the method further includes: Filter the depth map of the current frame to obtain the filtering result of the current frame; Obtain a depth map of a previous frame, and weight the current frame filtering result and the depth map of the previous frame based on a preset weight factor; The current frame depth map is updated according to the weighted result.

[0009] Optionally, filtering the depth map of the current frame to obtain a filtering result of the current frame includes: Get the preset neighborhood diameter, preset depth standard deviation, and preset distance standard deviation; For each coordinate point in the depth map of the current frame, determining a neighborhood point and a corresponding depth value based on the preset neighborhood diameter, determining a depth value difference based on the depth value of the coordinate point and the depth value of the neighborhood point, and determining a distance difference based on the coordinate point and the neighborhood point; The coordinate points are filtered based on the preset depth standard deviation, the preset distance standard deviation, the depth value difference, and the distance difference.

[0010] Optionally, the current frame physiological characteristics include current frame respiratory frequency, current frame heart rate, current frame heart rate variability, current frame chest vibration displacement, and current frame skin energy. Determining the current frame physiological characteristics of the target occupant based on the occupant physiological signal includes: determining a current frame respiratory frequency based on a respiratory signal in the occupant's physiological signal; determining a current frame heart rate and a current frame heart rate variability based on a heartbeat signal in the occupant's physiological signal; Based on the skin vibration signal in the occupant's physiological signal, the chest vibration displacement of the current frame and the skin energy of the current frame are determined.

[0011] Optionally, determining an operating mode of a thermal management system in the vehicle based on the relative distance and the physiological characteristics of the current frame includes: obtaining a bandwidth of the radar sensor, and determining a range resolution of the radar sensor based on the bandwidth; determining a range cell of the target occupant relative to the radar sensor based on the relative distance and the range resolution; A current state matrix is ​​constructed according to the distance unit and the physiological characteristics of the current frame, and based on the current state matrix and the state matrices corresponding to the preset modes, the operating mode of the thermal management system is determined in each preset mode.

[0012] Optionally, determining the temperature offset in the working mode based on the physiological characteristics of the current frame includes: Determining, based on the physiological features of the current frame and the physiological features of the previous frame, a change amount of each feature between the two moments; The temperature offset in the working mode is determined based on the variation of each characteristic.

[0013] Optionally, determining the temperature offset in the operating mode based on the change in each characteristic includes: determining an occupant label of the target occupant based on the current frame color image, and determining a weight of each feature corresponding to the target occupant based on the occupant label; The change amount of each feature is weighted based on the weight of each feature to obtain the temperature offset in the working mode.

[0014] An embodiment of the present application further provides an electronic device, comprising: processor and memory; The processor is used to execute the steps of the vehicle thermal management system control method provided in any embodiment of the present application by calling the program or instructions stored in the memory.

[0015] An embodiment of the present application also provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction enables a computer to execute the steps of the vehicle thermal management system control method provided by any embodiment of the present application.

[0016] In summary, the present application proposes a control method for a vehicle thermal management system, which obtains the current frame depth map and the current frame color map collected by the camera in the vehicle for a preset seating area, and the physiological signals of the occupants collected by the radar sensor for the preset seating area, and then determines the relative distance between the target occupant and the radar sensor based on the current frame depth map and the current frame color map, and determines the current frame physiological characteristics of the target occupant based on the occupant physiological signals, thereby determining the working mode of the thermal management system through the relative distance and the current frame physiological characteristics, and determining the temperature offset under the working mode through the current frame physiological characteristics to achieve control of the thermal management system. The method can identify the distance between the occupant and the radar through the depth map and the color map, improve the accuracy of the occupant distance, and determine the working mode and temperature offset in combination with the physiological characteristics. Considering that the occupant's physiological characteristics and distance can reflect the occupant's demand for temperature control, the occupant's temperature control demand is dynamically determined through multimodal data fusion, achieving more accurate zoned temperature control and improving occupant comfort and energy efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a flow chart of a control method for a vehicle thermal management system provided by an embodiment of the present application; Figure 2 This is a schematic diagram of a sensor configuration provided by an embodiment of the present application; Figure 3 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0020] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] As mentioned in the background art, in response to the problems in the prior art, this application proposes a control method for a vehicle thermal management system. Figure 1This is a flow chart of a control method for a vehicle thermal management system provided by an embodiment of the present application. This method can be executed by a controller of a thermal management system in a vehicle, or by a vehicle controller in a vehicle. Figure 1 , the control method of the vehicle thermal management system specifically includes: S110 , obtaining a current frame depth map and a current frame color map collected by a camera in the vehicle for a preset seating area, and an occupant physiological signal collected by a radar sensor for the preset seating area.

[0022] The preset seating areas may be pre-demarcated seating areas within the vehicle. For example, a five-seater vehicle may be divided into five preset seating areas: the left front seating area, the right front seating area, the left rear seating area, the center rear seating area, and the right rear seating area. The division of the preset seating areas may be related to the vehicle model.

[0023] In this embodiment of the present application, a camera and radar sensor can be pre-configured for each pre-set seating area. The camera can be a ToF (Time-of-Flight) camera, and the radar sensor can be a millimeter-wave radar. In addition to ToF cameras, millimeter-wave radars, and other related sensors, temperature and pressure sensors and infrared thermal imaging sensors can also be configured. The temperature and pressure sensors can be used to detect temperatures at different locations in the thermal management system to determine whether the thermal management system is functioning properly and for debugging and calibration. Each relevant sensor can use a conventional 5V voltage sensor.

[0024] like Figure 2 As shown, Figure 2 This is a sensor configuration diagram provided in an embodiment of the present application. The passenger compartment can be divided into five preset seating areas, among which the relevant sensors of the left front seating area and the right front seating area can be set in the roof or instrument panel area; the relevant sensors of the left rear seating area, the middle rear seating area and the right rear seating area can be set in the roof or the rear air-conditioning control area.

[0025] Specifically, for each preset seating area, the camera corresponding to the preset seating area can be used to capture images of the preset seating area to obtain a current frame depth map and a current frame color map; and the radar sensor corresponding to the preset seating area can be used to collect physiological signals of the occupants in the preset seating area.

[0026] Among them, the current frame depth map is a two-dimensional image that records the distance from each point in the preset riding area to the camera in pixels, which is used to convert the three-dimensional distance information of the space into two-dimensional grayscale. Each pixel point stores the vertical distance from the camera to the point in the preset riding area, that is, the depth value (such as the unit can be millimeters, meters, etc.).

[0027] The current frame color image may be an RGB image of the current frame, including three channels: red, green, and blue. The pixel value of each pixel includes the values ​​under the three channels: red, green, and blue.

[0028] The occupant's physiological signals may include breathing signals, heartbeat signals, and skin vibration signals.

[0029] S120 : Determine the relative distance between the target occupant and the radar sensor based on the current frame depth map and the current frame color map, and determine the current frame physiological characteristics of the target occupant based on the occupant's physiological signals.

[0030] Specifically, the target occupant in the preset seating area can be identified through the current frame depth map and the current frame color map, that is, the recorded depth information and color information, and then the relative distance between the target occupant and the radar sensor can be determined.

[0031] In some embodiments, determining the relative distance between the target occupant and the radar sensor based on the current frame depth image and the current frame color image includes the following steps: Step 11: Determine the 3D bounding box of the target occupant based on the current frame depth image and the current frame color image; Step 12: Determine the coordinates of the key points in the 3D bounding box and convert the key point coordinates from the world coordinate system to the coordinate system of the radar sensor; Step 13: Determine the relative distance between the target occupant and the radar sensor based on the converted key point coordinates.

[0032] In step 11, the target occupant in the preset seating area may be located based on the current frame depth map and the current frame color map to obtain a 3D bounding box of the target occupant.

[0033] Regarding step 11 above, in a specific embodiment, determining the 3D bounding box of the target occupant based on the current frame depth image and the current frame color image includes the following steps: Step 111: Convert the current frame depth map into a three-dimensional point cloud; Step 112: Project the three-dimensional coordinate point in the three-dimensional point cloud onto the current frame color image to obtain the pixel coordinate point corresponding to the three-dimensional coordinate point, and determine the pixel value of the pixel coordinate point as the pixel value corresponding to the three-dimensional coordinate point; Step 113: Determine the 3D bounding box of the target occupant based on the 3D coordinate points and corresponding pixel values ​​in the 3D point cloud.

[0034] In an embodiment of the present application, in order to deal with the edge blur problem between the seat back and the occupant's body, before constructing a three-dimensional point cloud based on the current frame depth map, the current frame depth map can also be bilaterally filtered to retain the edge details of the human body while smoothing the noise, and the result of the previous frame filtering can be used for smoothing.

[0035] In one example, before converting the current frame depth map into a three-dimensional point cloud, the following steps are further included: Step 1101: Filter the depth map of the current frame to obtain a filtering result of the current frame; Step 1102: Obtain a depth map of the previous frame, and weight the current frame filtering result and the depth map of the previous frame based on a preset weight factor; Step 1103: Update the current frame depth map according to the weighted result.

[0036] In step 1101, a bilateral filtering process may be performed on the depth map of the current frame to obtain a filtering result of the current frame. In the embodiment of the present application, the filtering process may be performed in the depth value space and the coordinate space.

[0037] With respect to the above step 1101, optionally, filtering the depth map of the current frame to obtain a filtering result of the current frame includes the following steps: Step 11011: Obtain a preset neighborhood diameter, a preset depth standard deviation, and a preset distance standard deviation; Step 11012: for each coordinate point in the depth map of the current frame, determine the neighborhood point and the corresponding depth value based on the preset neighborhood diameter, determine the depth value difference based on the depth value of the coordinate point and the depth value of the neighborhood point, and determine the distance difference based on the coordinate point and the neighborhood point; Step 11013: Filter the coordinate points based on the preset depth standard deviation, the preset distance standard deviation, the depth value difference, and the distance difference.

[0038] The preset neighborhood diameter is the preset pixel distance, the preset depth standard deviation is the tolerance for controlling the depth value difference, and the preset distance standard deviation is the weight for controlling the spatial distance. In step 11011, the preset neighborhood diameter, the preset depth standard deviation, and the preset distance standard deviation can be obtained, such as the preset neighborhood diameter. , preset depth standard deviation , preset distance standard deviation .

[0039] Specifically, in step 11012, for each coordinate point in the current frame depth map, the neighborhood range of the coordinate point can be determined in the current frame depth map by presetting the neighborhood diameter, and then multiple neighborhood points are determined within the neighborhood range to obtain the depth values ​​corresponding to the neighborhood points, and then the depth value difference between the depth value of the coordinate point and the depth value of the neighborhood point is calculated, and the distance difference between the coordinates of the coordinate point and the coordinates of the neighborhood point is calculated.

[0040] Furthermore, in step 11013, the coordinate point may be filtered according to a preset depth standard deviation, a preset distance standard deviation, a depth value difference, and a distance difference.

[0041] Specifically, the weights of each neighboring point of the coordinate point can be calculated by presetting the depth standard deviation, the preset distance standard deviation, the depth value difference, and the distance difference, and then the depth values ​​of each neighboring point are weighted according to the weights. The weighted results of all neighboring points are used to update the depth value of the coordinate point to obtain the current frame filtering result. The weights of the neighboring points are shown in the following formula: ; Where, are the coordinates of the coordinate point, is the coordinate of the neighboring point of the coordinate point, is the weight of the neighborhood points, is the depth value of the coordinate point, is the depth value of the neighborhood point, is the preset distance variance, is the preset depth variance.

[0042] Through the above steps 11011 to 11013, the coordinate points can be filtered using the neighborhood points of each coordinate point based on the spatial distance difference and the depth value difference, which can handle the edge blur problem between the seat back and the human body and output the current frame filtering result that is smooth and retains the edge of the human body.

[0043] After obtaining the current frame filtering result, further, in steps 1102-1103, the depth map of the previous frame can be obtained, and the current frame filtering result and the depth map of the previous frame are weighted based on a preset weight factor, so as to update the current frame depth map according to the weighted result.

[0044] For example, the current frame depth map can be updated as follows: ; Where, is the updated depth map of the current frame, is the preset weight factor, is the filtering result of the current frame, It is the depth map of the previous frame (the depth map of the previous frame updated based on the filtering result).

[0045] Through the above steps 1101 to 1103, the depth map of the previous frame can be used to smooth the filtering result of the current frame. Through time domain filtering, the historical frame data is used to suppress the random noise and instantaneous jitter of the current frame, which can stabilize the occupant's head point cloud during vehicle driving and ensure the accuracy of subsequent distance detection.

[0046] After bilateral filtering and smoothing the depth map of the current frame, the depth map of the current frame can be converted into a three-dimensional point cloud in step 111. Specifically, the three-dimensional coordinates of each point in the current frame depth map can be obtained by the pixel coordinates and depth value of each pixel in the image, thereby constructing a three-dimensional point cloud, as shown in the following formula: ; Where, is the pixel coordinate of the pixel point, is the depth value of the pixel, is the focal length of the camera in the x and y directions, is the optical center of the camera, is the coordinate of the 3D point cloud.

[0047] After converting the current frame depth map into a three-dimensional point cloud, further, in step 112, the three-dimensional coordinate point in the three-dimensional point cloud can be projected onto the current frame color map to obtain the pixel coordinate point corresponding to the three-dimensional coordinate point in the current frame color map, and then the pixel value of the pixel coordinate point is assigned to the three-dimensional coordinate point, that is, the pixel value of the pixel coordinate point is determined as the pixel value corresponding to the three-dimensional coordinate point.

[0048] Furthermore, in step 113, the coordinate values ​​and corresponding pixel values ​​of each 3D point in the 3D point cloud can be input into a pre-trained bounding box detection model (such as the YOLO model) to obtain a 2D bounding box of the target occupant. The 2D bounding box can be composed of the bounding box center coordinates, the bounding box dimensions (height and width), the bounding box depth, and the bounding box yaw angle.

[0049] Considering that the bounding box detection model may generate multiple overlapping 2D bounding boxes for the same object, we can also use the non-maximum suppression method to eliminate redundant 2D bounding boxes and retain only the most accurate 2D bounding box. The redundant 2D bounding boxes can be eliminated by calculating the intersection-over-union ratio between each 2D bounding box. The intersection-over-union ratio can be calculated using the following formula: ; Where, is the intersection-over-union ratio between bounding box A and bounding box B, is the area of ​​the intersection between bounding box A and bounding box B, is the area of ​​the union of bounding box A and bounding box B.

[0050] After determining the 2D bounding box of the target occupant, the regional point cloud corresponding to the 2D bounding box can be extracted from the 3D point cloud, and then the 3D bounding box corresponding to the regional point cloud can be generated using the minimum bounding box (AABB or OBB). For example, the regional point cloud corresponding to the 2D bounding box can be expressed as follows: ; Where, The region point cloud representing the corresponding area of ​​the 2D bounding box, are the coordinates of the point in the 2D bounding box, for The corresponding coordinates in the 3D point cloud.

[0051] Taking AABB (axis-aligned bounding box) as an example, we can take the maximum and minimum values ​​of the X, Y, and Z directions of the regional point cloud to construct a 3D bounding box, that is, through 、 、 、 、 、 Determine the 3D bounding box.

[0052] Through the above steps 111 to 113, a three-dimensional point cloud can be constructed using the current frame depth map, and then the 3D bounding box of the target occupant can be identified in combination with the current frame color map. Compared with directly identifying the human body boundary through the depth map, this method combines the current frame depth map with the current frame color map, and uses the pixel values ​​in the current frame color map as additional information of the three-dimensional coordinate points. It can take into account the color continuity of human clothing, more accurately distinguish the human body boundary and the seat back, and obtain a more accurate 3D bounding box.

[0053] After obtaining the target occupant's 3D bounding box, the pose of the 3D bounding box can be estimated and visualized in step 12. This involves determining the key points within the 3D bounding box, constructing a human skeleton tree (using the SMPL model), and calculating key angles and kinematic chains. This allows for real-time rendering of the 3D human pose and scene using Open3D. Once the key points are known, the coordinates of the key points within the 3D bounding box can be converted from the world coordinate system to the radar sensor's coordinate system.

[0054] For example, the key point coordinates can be calibrated by parameters (including rotation matrix R and translation vector t). Convert to the coordinate system of the radar sensor to obtain the new key point coordinates : ; After converting to the coordinate system of the radar sensor, further, in step 13, the center of mass coordinates of the human body or the center point coordinates of the 3D bounding box in the key point coordinates can be used to calculate the distance between the center of mass coordinates of the human body or the center point coordinates and the radar sensor to obtain the relative distance.

[0055] For example, the relative distance between the target occupant and the radar sensor can be calculated using the coordinates of the center of mass of the human body or the center point coordinates of the 3D bounding box. : ; In an embodiment of the present application, in addition to determining the relative distance between the target occupant and the radar sensor through the current frame depth map and the current frame color map, the current frame physiological characteristics of the target occupant can also be extracted from the occupant's physiological signals.

[0056] In some embodiments, the current frame physiological characteristics include the current frame respiratory frequency, the current frame heart rate, the current frame heart rate variability, the current frame chest vibration displacement, and the current frame skin energy. Determining the current frame physiological characteristics of the target occupant based on the occupant physiological signal includes the following steps: Step 21: determining the respiratory frequency of the current frame based on the respiratory signal in the occupant's physiological signal; Step 22: determining the current frame heart rate and the current frame heart rate variability based on the heartbeat signal in the occupant's physiological signal; Step 23: Determine the chest vibration displacement and skin energy of the current frame based on the skin vibration signal in the occupant's physiological signal.

[0057] Specifically, the radar sensor can collect breathing signals, heartbeat signals, and skin vibration signals that penetrate clothing. It can first perform a 0.1Hz~0.5Hz band-pass filter on the breathing signal, a 0.8Hz~2Hz band-pass filter on the heartbeat signal, and extract the high-frequency band of 2Hz~10Hz in the skin vibration signal.

[0058] In step 21, the peak interval in the respiratory signal can be identified, and the respiratory frequency of the current frame can be calculated based on the peak interval. is the interval between two adjacent peaks, the respiratory frequency of the current frame , the unit is times / minute; the peak needs to meet the following conditions: ,and, ,and, ; Where, is the peak value of the respiratory signal at time t, 、 are the peak values ​​of the respiratory signal at time t-1 and time t+1 respectively.

[0059] In step 22, the heartbeat signal can be subjected to Hilbert transform to extract the envelope, and the R wave peak can be captured by a dynamically adjusted threshold, and then the current frame heart rate and the current frame heart rate variability can be determined based on the R wave peak. , is the local mean of the signal, reflecting the baseline level of the signal in the current time window, is the local standard deviation of the signal, reflecting the fluctuation intensity of the signal, is the adjustment coefficient, which is used to control the sensitivity of the threshold to the fluctuation intensity.

[0060] Specifically, the heart rate variability of the current frame may include the heartbeat interval standard deviation and the heartbeat interval root mean square. Among them, the heartbeat interval root mean square RMSSD = , is the total number of heartbeats, is the i+1th normal heartbeat interval, It is the average of all normal heartbeat intervals.

[0061] Among them, in step 23, the chest vibration displacement of the current frame and the skin energy of the current frame can be determined by the skin vibration signal. After the radar sensor receives the reflected signal, it can obtain the skin vibration signal containing distance and speed information by mixing. The skin vibration signal can be a complex signal, that is, the skin vibration signal , 、 are the real and imaginary parts respectively, and j is the imaginary unit.

[0062] Specifically, the phase change can be extracted through the real and imaginary parts of the skin vibration signal. , , the phase change can reflect the chest vibration displacement, 、 are the horizontal and vertical coordinates of the midpoint of the skin vibration signal, the phase Indicates the angle of the point.

[0063] Furthermore, after the skin vibration signal is band-pass filtered at 2 Hz to 10 Hz, a high-frequency vibration signal can be obtained. The total energy of the signal can be calculated from the high-frequency vibration signal, and the skin energy of the current frame can be calculated from the total energy of the signal, as shown in the following formula: ; Where, is the skin energy of the current frame, specifically the root mean square of the total signal energy, is the number of signal sampling points, is the square sum of the signal, representing the total energy of the signal.

[0064] Through the above steps 21 to 23, the current frame respiratory frequency, current frame heart rate, current frame heart rate variability, current frame chest vibration displacement and current frame skin energy can be extracted, and the influence of parameters such as respiratory frequency, heart rate, chest vibration displacement and skin energy on temperature requirements can be taken into account. For example, respiratory frequency, heart rate, heart rate variability, chest vibration displacement can reflect the occupant's movement state, and skin energy can reflect the occupant's skin temperature. The occupant's movement state and occupant's skin temperature can affect the occupant's temperature requirement, which facilitates the subsequent control of the thermal management system based on such parameters, improves the control accuracy of the thermal management system, and makes the operation of the thermal management system more in line with the occupant's needs.

[0065] S130 : Determine an operating mode of a thermal management system in the vehicle based on the relative distance and the current frame physiological characteristics, and determine a temperature offset in the operating mode based on the current frame physiological characteristics.

[0066] After obtaining the relative distance between the target occupant and the radar sensor, as well as the current-frame physiological characteristics of the target occupant, the operating mode can be determined by the relative distance and the current-frame physiological characteristics, and the temperature offset under the operating mode can be determined by the current-frame physiological characteristics. The thermal management system can then be controlled to operate in this operating mode, and the adjusted temperature amount is the temperature offset.

[0067] In some embodiments, determining an operating mode of a thermal management system in a vehicle based on relative distance and physiological characteristics of a current frame includes the following steps: Step 31: Obtain the bandwidth of the radar sensor, and determine the range resolution of the radar sensor based on the bandwidth; Step 32: Determine the range unit of the target occupant relative to the radar sensor based on the relative distance and the range resolution; Step 33: construct a current state matrix according to the distance unit and the physiological characteristics of the current frame, and determine the working mode of the thermal management system in each preset mode based on the current state matrix and the state matrices corresponding to each preset mode.

[0068] In step 31, the range resolution of the radar sensor can be calculated based on the bandwidth and channel capacity of the radar sensor, as shown in the following formula: ; Where, is the distance resolution, is the channel capacity, is the bandwidth of the radar sensor.

[0069] Furthermore, in step 32, the relative distance may be converted into a distance unit according to the distance resolution, as shown in the following formula: ; Where, is the distance unit, Indicates rounding down. is the relative distance.

[0070] Furthermore, in step 33, the distance unit and the physiological characteristics of the current frame can be used to construct a current state matrix, which can reflect the distance and physiological state of the target occupant. The current state matrix is ​​then compared with the state matrices under each pre-calibrated preset mode to determine the state matrix with the highest similarity to the current state matrix, and the preset mode corresponding to the state matrix is ​​used as the working mode.

[0071] It should be noted that in the process of constructing the current state matrix, for data from different sources, namely the distance unit and the physiological characteristics of the current frame, time synchronization and spatial synchronization can be performed first. Time synchronization means that all sensors need to connect to the server to synchronize the system time, and each data packet is accompanied by a local timestamp accurate to milliseconds. Spatial synchronization requires matching spatial feature points for data integration.

[0072] Through steps 31-33 above, the relative distance is first converted into distance units. The current state matrix is ​​then constructed based on the physiological characteristics of each current frame. This matrix is ​​then compared with the pre-calibrated matrices for each mode to determine the operating mode of the thermal management system. This approach accurately determines the operating mode of the thermal management system by taking into account the impact of parameters such as distance, heart rate, heartbeat, and skin vibration energy on temperature requirements, ensuring that the thermal management system's operation better meets user needs. Furthermore, converting the relative distance to the radar sensor into distance units reduces the amount of data, thereby improving the efficiency of the comparison with the pre-calibrated state matrices.

[0073] After determining the operating mode of the thermal management system, considering that the current frame physiological characteristics can accurately reflect the target occupant's demand for temperature adjustment, the temperature offset under this operating mode can be further determined based on the current frame physiological characteristics to achieve precise temperature control under this operating mode.

[0074] For example, weights may be assigned to the various features in the physiological features of the current frame, and then weighted according to the weights to obtain the temperature offset.

[0075] In some embodiments, determining a temperature offset in the working mode based on the physiological characteristics of the current frame includes: Based on the physiological characteristics of the current frame and the physiological characteristics of the previous frame, the change amount of each characteristic between the two moments is determined; and based on the change amount of each characteristic, the temperature offset in the working mode is determined.

[0076] Specifically, the difference between the physiological features of the current frame and the physiological features of the previous frame can be calculated to obtain the change of each feature between the two moments.

[0077] Furthermore, the change amount of each feature can be weighted according to the weights pre-assigned to each feature to obtain the temperature offset, as shown in the following formula: ; in, is the temperature offset, 、 、 are the weights corresponding to breathing, heart rate and skin energy, is the change in respiratory rate, is the change in heart rate, is the change in skin energy.

[0078] The above embodiment determines the temperature offset by the difference between the physiological characteristics of the current frame and the physiological characteristics of the previous frame, and can accurately adjust the temperature in combination with the changing characteristics of the occupant's physiological characteristics, so that the temperature of the air outlet of the thermal management system is more in line with the needs of the occupants, further improving the comfort of the occupants.

[0079] In one example, determining a temperature offset in an operating mode based on a change in each characteristic includes: The occupant label of the target occupant is determined based on the current frame color image, and the weight of each feature corresponding to the target occupant is determined based on the occupant label; the change amount of each feature is weighted based on the weight of each feature to obtain the temperature offset in the working mode.

[0080] Specifically, facial recognition can be performed on the current frame color image, and the target occupant's occupant label can be determined based on the facial recognition results. After obtaining the occupant label, the weights of each feature corresponding to the target occupant can be queried based on the occupant label. The weights of each feature corresponding to each occupant can be determined through supervised learning by recording the occupant's physiological data at a known temperature, or by training a linear regression or regression forest algorithm on the occupant's manually adjusted temperature.

[0081] After the weights of the characteristics corresponding to the target occupant are obtained through query, the weights of the characteristics can be used to weight the changes of the characteristics to obtain the temperature offset.

[0082] Through the above example, the weights of various characteristics can be calibrated according to the characteristics of different occupants to meet the requirements of different occupants for passenger compartment temperature, further improve the accuracy of temperature control of the thermal management system, and thus enhance the occupant experience.

[0083] For example, assuming that the body temperature of the occupant in the right front seating area is 36.8°C, the occupant's respiratory rate can be extracted as 32 times / minute through the occupant's physiological signal (the normal respiratory rate is 6 to 30 times / minute), indicating that the occupant may have just performed strenuous exercise. At this time, the air outlet temperature should not be too low, and the air volume should not be too high. A corresponding working mode and temperature offset can be output accordingly. For example, if the right front seating area is output at a low gear, the temperature offset is +2°C, and the final output air volume temperature is 24°C.

[0084] An embodiment of the present application provides a control method for a vehicle thermal management system. The method obtains a current frame depth map and a current frame color map collected by a camera in the vehicle for a preset seating area, and an occupant physiological signal collected by a radar sensor for the preset seating area. The method then determines the relative distance between the target occupant and the radar sensor based on the current frame depth map and the current frame color map, and determines the current frame physiological characteristics of the target occupant based on the occupant physiological signal. The operating mode of the thermal management system is determined by the relative distance and the current frame physiological characteristics, and the temperature offset under the operating mode is determined by the current frame physiological characteristics to achieve control of the thermal management system. The method can identify the distance between the occupant and the radar through the depth map and the color map, improve the accuracy of the occupant distance, and determine the operating mode and temperature offset in combination with the physiological characteristics. Considering that the occupant's physiological characteristics and distance can reflect the occupant's demand for temperature control, the occupant's temperature control demand is dynamically determined through multimodal data fusion, achieving more accurate zoned temperature control and improving occupant comfort and energy efficiency.

[0085] Compared with related technologies, the control method of the vehicle thermal management system provided in the embodiment of the present application can effectively improve energy utilization. The air outlet of the traditional zone temperature control has an energy loss of about 50%. The embodiment of the present application provides a method that can dynamically focus the airflow, reducing the loss by about 15%. In addition, the method provided in the embodiment of the present application has high positioning accuracy. The traditional zone temperature control uses seat pressure sensors to identify the approximate position of the occupants. The embodiment of the present application uses cameras and radar sensors, and through multimodal fusion, the accuracy can be controlled to 3mm level. In addition, the method provided in the embodiment of the present application has a strong degree of personalized matching. The air conditioning mode of the traditional zone temperature control is fixed and can only be executed according to a fixed level. The embodiment of the present application can be controlled according to the real-time status of the occupants and a variety of pre-calibrated temperature control strategies to improve occupant comfort.

[0086] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, the electronic device 400 includes one or more processors 401 and a memory 402 .

[0087] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0088] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the vehicle thermal management system control method of any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters and threshold values ​​may also be stored in the computer-readable storage medium.

[0089] In one example, electronic device 400 may further include an input device 403 and an output device 404, which are interconnected via a bus system and / or other connection mechanisms (not shown). Input device 403 may include, for example, a keyboard, a mouse, etc. Output device 404 may output various information to the outside, including warning information, braking force, etc. Output device 404 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0090] Of course, to simplify, Figure 3 Only some of the components related to the present application in the electronic device 400 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the electronic device 400 may further include any other appropriate components according to specific application scenarios.

[0091] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle thermal management system control method provided by any embodiment of the present application.

[0092] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0093] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the vehicle thermal management system control method provided by any embodiment of the present application.

[0094] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0095] It should be noted that the terms used in this application are only for describing specific embodiments and are not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates an exception, the words "one", "an", "a kind of" and / or "the" do not specifically refer to the singular and may also include the plural. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method or device comprising the elements.

[0096] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the internal parts of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0097] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. The above is only the preferred implementation method of this application. It should be pointed out that due to the limitations of textual expression, there are objectively infinite specific structures. For ordinary technicians in this technical field, without departing from the principles of this application, they can also make several improvements, modifications or changes, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of this application.

Claims

1. A control method for a vehicle thermal management system, characterized in that: include: Obtaining a current frame depth image and a current frame color image collected by a camera in the vehicle for a preset seating area, and an occupant physiological signal collected by a radar sensor for the preset seating area; determining a relative distance between a target occupant and the radar sensor based on the current frame depth map and the current frame color map, and determining a current frame physiological feature of the target occupant based on the occupant physiological signal; An operating mode of the thermal management system in the vehicle is determined based on the relative distance and the current frame physiological characteristics, and a temperature offset in the operating mode is determined based on the current frame physiological characteristics.

2. The method according to claim 1, characterized in that Determining a relative distance between a target occupant and the radar sensor based on the current frame depth map and the current frame color map includes: Determine a 3D bounding box of the target occupant based on the current frame depth map and the current frame color map; Determine the coordinates of key points in the 3D bounding box, and convert the key point coordinates from the world coordinate system to the coordinate system of the radar sensor; The relative distance between the target occupant and the radar sensor is determined according to the converted key point coordinates.

3. The method according to claim 2, characterized in that Determining a 3D bounding box of the target occupant based on the current frame depth map and the current frame color map includes: Converting the current frame depth map into a three-dimensional point cloud; Projecting a three-dimensional coordinate point in the three-dimensional point cloud onto the current frame color image to obtain a pixel coordinate point corresponding to the three-dimensional coordinate point, and determining a pixel value of the pixel coordinate point as the pixel value corresponding to the three-dimensional coordinate point; A 3D bounding box of the target occupant is determined based on each 3D coordinate point and the corresponding pixel value in the 3D point cloud.

4. The method according to claim 3, characterized in that Before converting the current frame depth map into a three-dimensional point cloud, the method further includes: Filter the depth map of the current frame to obtain the filtering result of the current frame; Obtain a depth map of a previous frame, and weight the current frame filtering result and the depth map of the previous frame based on a preset weight factor; The current frame depth map is updated according to the weighted result.

5. The method according to claim 4, characterized in that The filtering process of the current frame depth map to obtain the current frame filtering result includes: Get the preset neighborhood diameter, preset depth standard deviation, and preset distance standard deviation; For each coordinate point in the depth map of the current frame, determining a neighborhood point and a corresponding depth value based on the preset neighborhood diameter, determining a depth value difference based on the depth value of the coordinate point and the depth value of the neighborhood point, and determining a distance difference based on the coordinate point and the neighborhood point; The coordinate points are filtered based on the preset depth standard deviation, the preset distance standard deviation, the depth value difference, and the distance difference.

6. The method according to claim 1, characterized in that The current frame physiological characteristics include a current frame respiratory frequency, a current frame heart rate, a current frame heart rate variability, a current frame chest vibration displacement, and a current frame skin energy. Determining the current frame physiological characteristics of the target occupant based on the occupant physiological signal includes: determining a current frame respiratory frequency based on a respiratory signal in the occupant's physiological signal; determining a current frame heart rate and a current frame heart rate variability based on a heartbeat signal in the occupant's physiological signal; Based on the skin vibration signal in the occupant's physiological signal, the chest vibration displacement of the current frame and the skin energy of the current frame are determined.

7. The method according to claim 1, characterized in that Determining an operating mode of a thermal management system in the vehicle based on the relative distance and the current frame physiological characteristics includes: obtaining a bandwidth of the radar sensor, and determining a range resolution of the radar sensor based on the bandwidth; determining a range cell of the target occupant relative to the radar sensor based on the relative distance and the range resolution; A current state matrix is ​​constructed according to the distance unit and the physiological characteristics of the current frame, and based on the current state matrix and the state matrices corresponding to the preset modes, the operating mode of the thermal management system is determined in each preset mode.

8. The method according to claim 1, characterized in that Determining a temperature offset in the working mode based on the current frame physiological feature includes: Determining, based on the physiological features of the current frame and the physiological features of the previous frame, a change amount of each feature between the two moments; The temperature offset in the working mode is determined based on the variation of each characteristic.

9. The method according to claim 8, characterized in that The determining of the temperature offset in the working mode based on the variation of each characteristic includes: determining an occupant label of the target occupant based on the current frame color image, and determining a weight of each feature corresponding to the target occupant based on the occupant label; The change amount of each feature is weighted based on the weight of each feature to obtain the temperature offset in the working mode.

10. An electronic device, characterized in that: The electronic device comprises: processor and memory; The processor is configured to execute the steps of the vehicle thermal management system control method according to any one of claims 1 to 9 by calling the program or instruction stored in the memory.

Images