Cabin state adjusting method and related equipment

Abstract

The invention discloses a cabin state adjusting method, which comprises the following steps that: a vehicle control unit authenticates and positions a user according to a digital key, determines the user in combination with visual detection, and acquires a first image shot by a camera positioned on a main driving side when detecting a door opening operation on the main driving side; then, the vehicle control unit carries out human body key point detection on the first image, and pixel coordinates of key points of the user are obtained; then, the vehicle control unit determines a first vertical plane according to the pixel coordinates of the key points and the parameter information of the camera, and determines a second vertical plane according to the target detection frame, the depth image and the parameter information of the camera; and finally, the vehicle control unit extracts height characteristics of the user according to a fused vertical plane obtained by fusing the first vertical plane and the second vertical plane, and adjusts parameters of facilities in the cabin so as to adjust the state of the cabin. According to the method, the height characteristics of the user can be accurately extracted, so that the cabin state is accurately adjusted, and the use experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle control, and in particular to a cabin state adjustment method, a cabin state adjustment device, a vehicle controller and a computer readable storage medium. BACKGROUND

[0002] With the development of information technology, users' demand for intelligent use of vehicles is also increasing. An important demand for intelligent use of vehicles is intelligent cabin, specifically, the cabin state can be self-adaptively adjusted according to the user entering the vehicle. For example, the intelligent control system of the cabin in the vehicle perceives that the user opens the door remotely through the key, and can open the vehicle-mounted air conditioner in advance, and adjust the seat of the driving position to a state suitable for the user to drive.

[0003] Among them, the cabin state adjustment refers to that the vehicle analyzes the physiological state or behavior mode of the user through multi-modal interaction perception, and adjusts the facilities in the cabin based on the physiological state or behavior mode. The cabin state has an important influence on the convenience and comfort of the user during use. For example, the vehicle can detect the height of the user, and adjust the height of the facilities such as steering wheel and seat according to the height of the user, so as to facilitate the user to use.

[0004] However, the accuracy of the current cabin state adjustment is not high, resulting in poor user experience of using vehicles. SUMMARY

[0005] Therefore, the present application provides a cabin state adjustment method and related equipment to solve the problem that the accuracy of the current cabin state adjustment is not high, resulting in poor user experience of using vehicles.

[0006] In a first aspect, the present application provides a cabin state adjustment method, which comprises:

[0007] The vehicle controller detects the door opening operation of the main driving side, and obtains the first image photographed by the camera located at the main driving side. Then, the vehicle controller detects the human key points in the first image through a pose detection algorithm to obtain the pixel coordinates of the key points of the user in the first image. Next, the vehicle controller determines the first vertical plane according to the pixel coordinates of the key points and the parameter information of the camera, and determines the second vertical plane according to the target detection frame in the first image, the depth image constructed based on the first image and the parameter information of the camera. Among them, the first vertical plane and the second vertical plane are both perpendicular to the ground plane. Finally, the vehicle controller extracts at least one of the full body height, the half body height, the thigh length, the calf length, the arm length or the small arm length of the user as the height feature of the user according to the fusion vertical plane obtained by fusing the first vertical plane and the second vertical plane, and adjusts the parameters of the facilities in the cabin to adjust the cabin state.

[0008] On the one hand, the vehicle controller accurately extracts the height feature of the user by acquiring the user image and using the key point coordinates of the user detected according to the user image, the depth image constructed based on the user image, and the parameter information of the camera, and then adjusts the cabin state, thereby improving the accuracy of cabin adjustment. On the other hand, the method can automatically adjust the cabin state according to the extracted user feature, avoiding the tedious process of manually adjusting the cabin state each time the user enters the cabin, improving the efficiency of state adjustment, and improving the user experience. In addition, the method collects images before the user enters the cabin, protecting the user's in-cabin privacy.

[0009] In some possible implementation manners, the vehicle controller determines the first vertical plane according to the pixel coordinates of the key point and the parameter information of the camera. The determination of the first vertical plane can be performed in the following manner. The vehicle controller determines a projection ray of the key point in a world coordinate system according to the pixel coordinates of the key point and the parameter information of the camera, wherein the projection ray passes through the key point and has a camera optical center as an apex. Then, the vehicle controller determines the first vertical plane according to the projection ray of the key point in the world coordinate system. The first vertical plane can be used as a projection plane of the key point in the world coordinate system, and then the position information of the key point in the world coordinate system is obtained. In this way, the position information of the key point in the real world can be obtained by determining a projection plane according to the pixel coordinates of the key point in the image and the parameter information of the camera, and projecting the key point in the image into the world coordinate system.

[0010] In some possible implementation manners, the key point of the user includes a ground contact key point, for example, an ankle key point of the user. In this way, the vehicle controller can determine a projection ray of the ground contact key point of the user in a world coordinate system according to the pixel coordinates of the ground contact key point of the user and the parameter information of the camera, wherein the projection ray of the ground contact key point of the user in the world coordinate system passes through the ground contact key point and has a camera optical center as an apex. Then, the vehicle controller can determine the first vertical plane according to the intersection of the projection ray of the ground contact key point of the user in the world coordinate system and the ground plane. Since the coordinates of the ground contact key point of the user are relatively easy to determine and verify, the accuracy of determining the first vertical plane according to the ground contact key point of the user and the parameter information of the camera is relatively high, and the method is feasible.

[0011] In some possible implementation manners, the vehicle controller determines the second vertical plane according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera in the following manner. The vehicle controller first determines the projection point of a pixel in the target detection frame in the first image in a world coordinate system according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera. Then, the vehicle controller clusters the projection points, for example, can obtain a plurality of grouping clusters by using a density-based spatial clustering of applications with noise (DBSCAN) algorithm. Next, the vehicle controller determines a target cluster according to the average distance between the projection points in each grouping cluster and the camera optical center, wherein the target cluster is the cluster with the smallest average distance. Finally, the vehicle controller determines the second vertical plane according to the target cluster. For example, the vehicle controller can fit the point set in the target cluster by using a least square method to obtain the second vertical plane. In this way, the vehicle controller can fully utilize the depth information of the image to determine a projection plane, and project the key points in the image to the world coordinate system to obtain the position information of the key points in reality.

[0012] In some possible implementation manners, the vehicle controller can obtain a fusion vertical plane by fusing the first vertical plane and the second vertical plane. For example, the vehicle controller can determine a first normal vector of the first vertical plane and a second normal vector of the second vertical plane, and obtain a fusion normal vector by averaging the first normal vector and the second normal vector, for example, an arithmetic average. The vehicle controller can determine the fusion vertical plane according to the intersection line of the first vertical plane and the second vertical plane and the fusion normal vector. In this way, fusing the vertical planes determined by two methods can reduce the error caused by a single detection method, and a more accurate vertical plane can be obtained.

[0013] In some possible implementation manners, the vehicle controller extracts the height feature of the user according to the fusion vertical plane in the following manner. The vehicle controller determines the projection point of the key point on the fusion vertical plane according to the projection ray of the key point in the world coordinate system and the fusion vertical plane, and then extracts the height feature of the user according to the projection point of the key point on the fusion vertical plane. The height feature includes at least one of the full-body height, the half-body height, the thigh length, the calf length, the arm length, or the small arm length.

[0014] In some possible implementation manners, the in-cabin facility includes at least one of a seat, a steering wheel, a rearview mirror, and a head-up display (HUD). In this way, the vehicle controller can adjust parameters of the in-cabin facility, such as a height of the seat, an inclination angle of the seat, a relative position of the seat and the steering wheel, a height of the steering wheel, an angle of the rearview mirror, a projection height of the HUD, and a scaling scale of the HUD, according to the height feature of the user, to adjust the in-cabin state and improve user comfort.

[0015] In some possible implementation manners, the vehicle controller can further acquire a dressing feature of the user according to the first image. Then, the vehicle controller can adjust parameters of an air conditioner in the cabin according to the dressing feature of the user, to adjust a state of the air conditioner in the cabin. The parameters of the air conditioner include at least one of an air duct, an air volume, an air direction, a temperature, and a circulation type. In this way, the state of the air conditioner in the cabin is automatically adjusted according to the dressing of the user, and user comfort is improved, for example, high-intensity cooling can be avoided when the user is dressed in cool clothes.

[0016] In some possible implementation manners, the vehicle controller can store the manually adjusted parameters as preferred parameters of the user in response to the user manually adjusting the parameters of the in-cabin facility, and adjust the in-cabin state by using the preferred parameters when the user enters the cabin again. In this way, the personalized needs of the user can be met, and the in-cabin state is automatically adjusted when the user enters the cabin again, improving the efficiency of adjustment.

[0017] In some possible implementation manners, the vehicle controller can receive a plurality of fused preferred parameters sent by a cloud server. Each fused preferred parameter is obtained by fusing a plurality of preferred parameters in each group in a plurality of groups, each group corresponds to one fused preferred parameter, and the plurality of groups are obtained based on attribute information provided by a plurality of users. Then, the vehicle controller can adjust the in-cabin state according to the plurality of fused preferred parameters and the attribute information of the user. In this way, the vehicle controller can recommend parameter settings to the user according to the preferred parameters corresponding to the attribute of the user, and the method is universal.

[0018] In some possible implementation manners, the vehicle controller can automatically adjust the in-cabin state corresponding to a moving position of the user according to the movement of the user. Specifically, when the user is detected by the depth camera to move from a first position in the cabin to a second position, the parameters of the facility in the second position are adjusted according to the parameters of the facility in the first position. Then, the vehicle controller can adjust the in-cabin state according to the parameters of the facility in the second position. In this way, the method is more flexible, and user experience is improved.

[0019] In a second aspect, the present application provides a cockpit state adjustment device, which comprises various modules for executing the cockpit state adjustment method in the first aspect or any possible implementation manner of the first aspect, and specifically comprises:

[0020] a communication module, configured to acquire a first image captured by a camera located at the driver side when detecting a door opening operation at the driver side;

[0021] a detection module, configured to perform human key point detection on the first image by using a pose detection algorithm to obtain pixel coordinates of the key points of the user in the first image;

[0022] In some possible implementation manners, the detection module is further configured to determine a first vertical plane according to the pixel coordinates of the key points and parameter information of the camera, and determine a second vertical plane according to the target detection frame in the first image, a depth image constructed based on the first image and the parameter information of the camera, wherein the first vertical plane and the second vertical plane are both perpendicular to the ground plane.

[0023] a feature extraction module, configured to extract a height feature of the user according to a fusion vertical plane obtained by fusing the first vertical plane and the second vertical plane, wherein the height feature comprises at least one of a full-body height, a half-body height, a thigh length, a calf length, an arm length or a small arm length.

[0024] an adjustment module, configured to adjust parameters of facilities in the cockpit according to the height feature of the user, so as to adjust the cockpit state.

[0025] In some possible implementation manners, the detection module is specifically configured to:

[0026] determine a projection ray of the key point in a world coordinate system according to the pixel coordinates of the key point and the parameter information of the camera, wherein the projection ray passes through the key point and has a camera optical center as an apex, and then determine the first vertical plane according to the projection ray of the key point in the world coordinate system. The first vertical plane can be used as a projection plane of the key point in the world coordinate system, and thus the position information of the key point in the world coordinate system can be obtained.

[0027] In some possible implementation manners, the key points of the user include a ground contact key point, for example, an ankle key point of the user. The detection module is specifically configured to:

[0028] determine a projection ray of the ground contact key point of the user in the world coordinate system according to the pixel coordinates of the ground contact key point of the user and the parameter information of the camera, wherein the projection ray of the ground contact key point of the user in the world coordinate system passes through the ground contact key point and has the camera optical center as an apex, and then determine the first vertical plane according to the intersection of the projection ray of the ground contact key point of the user in the world coordinate system and the ground plane.

[0029] In some possible implementation manners, the detection module is specifically configured to:

[0030] The projection point of the pixel in the target detection frame in the first image in the world coordinate system is determined according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera. Then, the projection points are clustered to obtain a plurality of grouping cluster groups. Next, the target cluster is determined according to the average distance between the projection points in each grouping cluster group and the camera optical center, wherein the target cluster is the cluster group with the minimum average distance. Finally, the second vertical plane is determined according to the target cluster.

[0031] In some possible implementation manners, the detection module can also be configured to:

[0032] The fusion vertical plane is obtained by fusing the first vertical plane and the second vertical plane. For example, the first normal vector of the first vertical plane and the second normal vector of the second vertical plane can be determined, and the first normal vector and the second normal vector are averaged, for example, arithmetically averaged, to obtain a fusion normal vector. Then, the fusion vertical plane can be determined according to the intersection line of the first vertical plane and the second vertical plane and the fusion normal vector.

[0033] In some possible implementation manners, the feature extraction module is specifically configured to:

[0034] The projection point of the key point on the fusion vertical plane is determined according to the projection ray of the key point in the world coordinate system and the fusion vertical plane, and then the height feature of the user is extracted according to the projection point of the key point on the fusion vertical plane. The height feature includes at least one of the full-body height, the half-body height, the thigh length, the calf length, the arm length, or the small arm length.

[0035] In some possible implementation manners, the feature extraction module can also be configured to:

[0036] The dressing feature of the user is obtained according to the first image, and the dressing feature of the user can be used to adjust the parameters of the air conditioner in the cabin and adjust the state of the air conditioner in the cabin. In this way, the use comfort of the user can be improved, for example, high-intensity cooling can be avoided when the user is dressed in cool clothes.

[0037] In some possible implementation manners, the adjustment module can also be configured to:

[0038] In response to the user manually adjusting the parameters of the facilities in the cabin, the manually adjusted parameters are stored as the preference parameters of the user, and the preference parameters are used to adjust the state of the cabin when the user enters the cabin again. In this way, the individual needs of the user can be met, and the state of the cabin can be automatically adjusted when the user enters the cabin again, improving the efficiency of adjustment.

[0039] In some possible implementation manners, the adjustment module can also be configured to:

[0040] The cabin state is adjusted according to the plurality of fused preference parameters and the attribute information of the user. Each fused preference parameter is obtained by fusing a plurality of preference parameters in each group in a plurality of groups in a cloud server, each group corresponds to one fused preference parameter, and the plurality of groups are obtained based on the attribute information provided by the user.

[0041] In some possible implementation manners, the adjusting module can also be configured to:

[0042] The cabin state corresponding to the moving position is automatically adjusted according to the movement of the user. Specifically, when the user is detected by the depth camera to move from a first position to a second position in the cabin, the parameters of the facilities in the second position are adjusted according to the parameters of the facilities in the first position, and the cabin state is adjusted based on the parameters of the facilities in the second position.

[0043] In a third aspect, the present application provides a vehicle control unit. The vehicle control unit comprises a processor and a memory. The memory is configured to store computer instructions, and the processor is configured to execute the method of the first aspect or any possible implementation manner of the first aspect according to the computer instructions.

[0044] In a fourth aspect, the present application provides a computer readable medium, and the computer readable storage medium stores instructions. When the instructions are run on a computer device, the computer device executes the method of the first aspect or any possible implementation manner of the first aspect.

[0045] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when run on a computer, cause the computer to execute the method of the first aspect or any possible implementation manner of the first aspect.

[0046] On the basis of the implementation manners of the aspects described above, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of a cabin state adjustment method disclosed by an embodiment of the present application;

[0048] Figure 2 A schematic diagram of user trajectory determination disclosed by an embodiment of the present application;

[0049] Figure 3 A schematic diagram of coordinate system conversion disclosed by an embodiment of the present application;

[0050] Figure 4 A schematic diagram of key point projection disclosed by an embodiment of the present application;

[0051] Figure 5A schematic diagram of a method for adjusting facility parameters in a cabin disclosed in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of a method for adjusting air conditioning system parameters disclosed in an embodiment of the present application;

[0053] Figure 7 A schematic diagram of a parameter migration method disclosed in an embodiment of the present application;

[0054] Figure 8 A flowchart of a cabin state adjustment method disclosed in an embodiment of the present application;

[0055] Figure 9 A structural schematic diagram of a cabin state adjustment device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the above objectives, features and advantages of the present application more apparent, further detailed description of the embodiments of the present application will be given below with reference to the accompanying drawings and specific embodiments.

[0057] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to be limiting on the present application. The terms "first", "second" in the embodiments of the present application are only for the purpose of description and cannot be understood as indicating or implying relative importance, operation time sequence or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more of the features.

[0058] First, some technical terms involved in the embodiments of the present application are introduced.

[0059] Cabin state adjustment refers to that a vehicle analyzes a user's physiological state or behavior pattern through multi-modal interactive perception, and adjusts facilities in the cabin based on the physiological state or behavior pattern. For example, the vehicle can detect the height of the user, and adjust the height of the steering wheel, seat and other facilities according to the height of the user, so as to improve the driving experience or riding experience of the user.

[0060] The traditional cabin state adjustment mode relies on fixed option buttons or manual adjustment by the user, and cannot automatically complete personalized adjustment flexibly according to different user needs, so that the cabin state adjustment process is cumbersome and affects the user experience.

[0061] An improved mode is to call the setting parameters of the cabin according to the user identity information, and adjust the cabin state according to the setting parameters. This mode calls the setting parameters of the cabin through the user identity information, and adjusts the cabin state to the corresponding memory state according to the setting parameters, so that when the user enters the cabin for the first time, the cabin state can be adjusted to the state set by the user, improving the user experience.

[0062] However, this method cannot obtain accurate user position information and user characteristics, and still relies on the user to manually adjust the parameters of the facilities in the cabin when entering the cabin for the first time, thereby failing to automatically accurately adjust the cabin state for the user entering the cabin, resulting in poor user experience.

[0063] Therefore, the present application provides a cabin state adjustment method and related device to accurately determine the position information of the user and extract the user characteristics, and then automatically accurately adjust the cabin state for the user, thereby improving the user experience.

[0064] Specifically, the vehicle controller detects the door opening operation on the driver side, and obtains a first image captured by a camera located on the driver side. Then, the vehicle controller detects the human key points in the first image through a pose detection algorithm to obtain the pixel coordinates of the key points of the user in the first image. Next, the vehicle controller determines a first vertical plane according to the pixel coordinates of the key points and the parameter information of the camera, and determines a second vertical plane according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera. The first vertical plane and the second vertical plane are both perpendicular to the ground plane. Finally, the vehicle controller extracts at least one of the full body height, the half body height, the thigh length, the calf length, the arm length or the small arm length of the user as the height characteristics of the user according to the fusion vertical plane obtained by fusing the first vertical plane and the second vertical plane, and adjusts the parameters of the facilities in the cabin to adjust the cabin state.

[0065] On the one hand, the vehicle controller accurately extracts the height characteristics of the user by obtaining the user image and using the key point coordinates of the user detected from the user image, the depth image constructed based on the user image, and the parameter information of the camera, and then adjusts the cabin state, thereby improving the accuracy of the cabin adjustment. On the other hand, this method can automatically adjust the cabin state according to the extracted user characteristics, avoiding the tedious process of manually adjusting the cabin state every time the user enters the cabin, improving the efficiency of state adjustment, and improving the user experience. In addition, this method collects images before the user enters the cabin, protecting the user's cabin privacy.

[0066] In order to make the technical solutions of the present application clearer and easier to understand, the method provided by the present application will be introduced below in combination with specific embodiments.

[0067] Figure 1 A flowchart of a cabin state adjustment method disclosed in an embodiment of the present application. The method comprises:

[0068] S102: The vehicle controller obtains a first image captured by a camera located on the driver side.

[0069] In the embodiments of the present application, the first image can be collected by the vehicle-mounted camera to automatically adjust the cabin state according to the height characteristics of the user in the parking scene. The vehicle controller can send a request for collecting images to the vehicle-mounted camera in response to the door opening operation to obtain the first image. The vehicle-mounted camera can be a monocular camera, a 360-degree panoramic camera, a surround-view fisheye camera, etc., and the door opening operation can be that the vehicle controller detects that the user pulls the driver's side door from the controller area network (CAN), or triggers a passive entry (PE) event signal, etc.

[0070] In some possible implementations, the vehicle controller can authenticate the user according to the digital key. The digital key, also known as Bluetooth key or virtual key, is a new type of key that uses modern communication technologies such as Bluetooth, near field communication (NFC), ultra-wide band (UWB), etc. to realize vehicle unlocking and control. The digital key can be loaded on a smart device, and by establishing a wireless connection between the smart device and the vehicle, identity verification and key exchange can be completed, so that encrypted communication can be performed and instructions can be executed. In this way, the door opening operation can be simplified on the basis of ensuring security, and the worry of losing the physical key can be eliminated.

[0071] The user can open the digital key permission of the vehicle to other users by generating a new digital key and a corresponding identification (ID), so that other users can use the digital key for authentication. The vehicle controller can perform unlocking, starting or other vehicle-related operations in response to the authentication result of the digital key. In this way, the needs of valet parking, friends borrowing cars, and driving for hire can be met.

[0072] In some possible implementations, the vehicle controller can also locate the user according to the digital key and determine the user in combination with the image obtained by the camera. Specifically, refer to Figure 2The vehicle controller can obtain the user's first trajectory based on the location information from the digital key, such as location information obtained via Bluetooth or UWB. It then uses time-lapse images captured by the onboard camera at specific time intervals to detect multiple candidate targets and determine the user's second trajectory from these candidate targets' trajectories. For example, the vehicle controller can obtain the bounding boxes of all candidate targets in the image and obtain multiple candidate target trajectories by tracking the center point of the lower edge of each bounding box. The controller can then compare the average Euclidean distance between the candidate target trajectories and the first trajectory and determine the trajectory with the smallest average Euclidean distance as the user's second trajectory. This allows for accurate user identification, secondary matching, and authentication, improving vehicle security.

[0073] Based on the above method, the first image acquired by the vehicle controller can be an image including a specific user. For example, in response to a door opening operation, the vehicle controller can extract an image including the user from a series of delayed images as the first image, and then perform detection and feature extraction. In this way, user detection and feature extraction can be performed in a targeted manner, improving efficiency.

[0074] S104: The vehicle controller performs human key point detection on the first image to obtain the pixel coordinates of the user's key points in the first image.

[0075] In the embodiments of this application, human key points are important location points on the human body structure, which usually represent joints or specific parts of the human body, such as the head, shoulders, elbows, wrists, hips, knees, and ankles. Human body key points can be used to estimate human posture and are applicable to various scenarios such as action recognition, behavior analysis, and motion capture.

[0076] The vehicle controller can use a pose detection algorithm to detect human key points in the first image and obtain the pixel coordinates of the user's key points. In this way, the user's location and body feature information can be identified based on the coordinates of the key points.

[0077] S106: The vehicle controller determines the first vertical plane based on the pixel coordinates of the key points and the parameter information of the camera.

[0078] In the embodiments of the present application, the vehicle controller can determine the first vertical plane according to the pixel coordinates of the key points and the parameter information of the camera, so as to project the key points in the image into the world coordinate system to obtain the position information of the key points in reality. The parameter information of the camera includes camera external parameters, camera internal parameters and distortion parameters and the like. Among them, the camera internal parameters (Intrinsic Parameters) are related to the physical properties of the camera, which can one-to-one map the pixel coordinates of the points in the pixel coordinate system (Pixel Coordinate System) and the coordinates in the camera coordinate system (Camera Coordinate System), and complete the conversion between the two coordinate systems. The camera external parameters (Extrinsic Parameters) are related to the position information of the camera in the world coordinate system, which can one-to-one map the points in the camera coordinate system and the points in the world coordinate system (World Coordinate System), and complete the conversion between the two coordinate systems. The distortion parameter is a physical quantity describing the degree of distortion generated by the camera in the imaging process, which can be used to realize coordinate de-distortion and improve the imaging quality. As shown in FIG. 1, Figure 3 w -X w Y w Z w is the world coordinate system, which can describe the position of an object in reality, for example, the actual position of the camera. Point P is a point in the world coordinate system, that is, a point in reality. O c -X c Y c Z c is the camera coordinate system, which is a three-dimensional rectangular coordinate system with the focusing center of the camera as the origin and the optical axis as the Z axis. O-xy is the image coordinate system, which is located in the imaging plane, and the origin of the image coordinate system is the intersection of the camera optical axis and the imaging plane. Point p is the imaging point of point P in the image, and its coordinates are (x, y). f is the camera focal length, which is equal to the distance between o and O c . uv is the pixel coordinate system, which is also located in the image plane, and is a two-dimensional rectangular coordinate system with pixels as the unit, with the upper left corner of the image as the origin. The horizontal coordinate u and the vertical coordinate v of the pixel are the column number and the row number of the pixel in the image array, respectively.

[0079] ​The image captured by the camera can also have distortion. Distortion refers to the phenomenon of shape distortion or size distortion in the image. For example, barrel distortion can cause objects in the center of the image to visually curve outward, which is common in wide-angle lenses. Pincushion distortion can cause objects in the center of the image to visually curve inward, which is common in telephoto lenses. Therefore, the vehicle controller needs to de-distort the coordinates based on the distortion parameters of the camera to obtain more accurate position information, which is beneficial for extracting user features.

[0080] The vehicle controller can determine a projection ray of the key point in the world coordinate system according to the pixel coordinates of the key point and the parameter information of the camera, wherein the projection ray passes through the key point and the vertex is the camera optical center. Then, the vehicle controller determines the first vertical plane according to the projection ray of the key point in the world coordinate system. The first vertical plane can be used as the projection plane of the key point in the world coordinate system, and thus the position information of the key point in the world coordinate system is obtained.

[0081] In some possible implementations, the key point of the user includes a ground contact key point, such as an ankle key point of the user. In this way, the vehicle controller can determine a projection ray of the ground contact key point of the user in the world coordinate system according to the pixel coordinates of the ground contact key point of the user and the parameter information of the camera, wherein the projection ray of the ground contact key point of the user in the world coordinate system passes through the ground contact key point and the vertex is the camera optical center. Then, the vehicle controller can determine the first vertical plane according to the intersection of the projection ray of the ground contact key point of the user in the world coordinate system and the ground plane. Since the coordinates of the ground contact key point of the user are relatively easy to determine and verify, the accuracy of determining the first vertical plane by the ground contact key point of the user and the parameter information of the camera is high, and it is feasible.

[0082] S108: The vehicle controller determines a second vertical plane according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera.

[0083] In the embodiments of the present application, the vehicle controller can determine a second vertical plane according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera, so as to project the key point in the image to the world coordinate system and obtain the position information of the key point in reality. The parameter information of the camera includes camera external parameters, camera internal parameters, and distortion parameters, etc., which are used for coordinate de-distortion and coordinate conversion.

[0084] The vehicle controller can determine a projection point of a pixel in the target detection frame in the first image in a world coordinate system according to the target detection frame in the first image, a depth image constructed based on the first image, and parameter information of the camera. The depth image refers to an image composed of distance information from each point in the image or local image to the camera, which can be obtained based on depth estimation. Depth estimation can be divided into monocular depth estimation, binocular depth estimation, multi-view stereo (MVS), etc. In the embodiment of the present application, the first image is obtained by a camera on the driver's side, and monocular depth estimation can be selected, for example, the depth image is obtained by estimating the first image through a monocular depth estimation self-supervised model MonoDepth2. The target detection frame is an identification frame for identifying the user, and the parameter information of the camera includes camera external parameters, camera internal parameters, distortion parameters, etc., which are used to realize coordinate de-distortion and coordinate conversion. It should be noted that the present application does not limit the algorithm or model used to obtain the depth image, nor does it limit how to train the depth estimation model.

[0085] In some possible implementations, the vehicle controller can cluster the projection points, for example, a density-based spatial clustering algorithm with noise application (DBSCAN) can be used to obtain a plurality of grouping clusters. It should be noted that the present application does not limit the use of any clustering algorithm.

[0086] After obtaining the plurality of grouping clusters, the vehicle controller can determine a target cluster according to an average distance between the projection points in each grouping cluster and the camera optical center, wherein the target cluster is the cluster with the smallest average distance. For example, the average distance can be the average Euclidean distance, which is not limited by the present application.

[0087] Next, the vehicle controller can determine a second vertical plane according to the target cluster. For example, the vehicle controller can fit the point set in the target cluster by the least square method to obtain the second vertical plane. In this way, the vehicle controller can make full use of the depth information of the image to determine a projection plane, and project the key points in the image to the world coordinate system to obtain the position information of the key points in reality.

[0088] It should be noted that steps S106 and S108 have no sequence, and the first vertical plane and the second vertical plane do not imply relative importance or operation time sequence. The vehicle controller can first perform S106 to obtain the first vertical plane, or first perform S108 to obtain the second vertical plane.

[0089] S110: The vehicle controller extracts the height feature of the user according to the fusion vertical plane obtained by fusing the first vertical plane and the second vertical plane.

[0090] In some possible implementation manners, the vehicle controller can obtain the fusion vertical plane by fusing the first vertical plane and the second vertical plane. For example, the vehicle controller can determine a first normal vector of the first vertical plane and a second normal vector of the second vertical plane, and average the first normal vector and the second normal vector, for example, arithmetically, to obtain a fusion normal vector. The vehicle controller can determine the fusion vertical plane according to the intersection line of the first vertical plane and the second vertical plane and the fusion normal vector. In this way, the fusion vertical plane determined by fusing two kinds of methods can reduce the error caused by a single detection method, and a more accurate vertical plane can be obtained.

[0091] In some possible implementation manners, the vehicle controller can determine the projection point of the key point on the fusion vertical plane according to the projection ray of the key point in the world coordinate system and the fusion vertical plane, and then extract the height feature of the user according to the projection point of the key point on the fusion vertical plane.

[0092] As shown in FIG. 1, Figure 4 The intersection point of the projection ray determined by the pixel coordinates of the key point and the fusion vertical plane is the projection point of the key point on the fusion vertical plane. The vehicle controller can extract the height feature of the user according to the projection point of the key point on the fusion vertical plane. It should be noted that Figure 4 Only one kind of key point selection and projection diagram is shown, and in the application of the method, appropriate key points should be selected according to actual conditions.

[0093] The height feature includes at least one of the full-body height, the half-body height, the thigh length, the calf length, the arm length, or the small arm length. For example, the key point can be a ground key point, a head key point, a shoulder key point, a hand key point, an elbow key point, a hip key point, a knee key point, and the like, the full-body height can be the Euclidean distance between the head key point and the ground key point in the vertical direction, the half-body height can be the Euclidean distance between the shoulder key point and the hip key point in the vertical direction, the thigh length can be the Euclidean distance between the hip key point and the knee key point in the vertical direction, the calf length can be the Euclidean distance between the knee key point and the ground key point in the vertical direction, the arm length can be the Euclidean distance between the shoulder key point and the hand key point, and the small arm length can be the Euclidean distance between the elbow key point and the hand key point.

[0094] S112: The vehicle controller adjusts the parameters of the facilities in the cabin according to the height feature of the user, to adjust the cabin state.

[0095] In some possible implementation manners, the in-cabin facilities include at least one of a seat, a steering wheel, a rearview mirror, and a head-up display (HUD). As shown in Figure 5 illustrated, the vehicle controller can adjust parameters of the in-cabin facilities, for example, a height of the seat, an inclination angle of the seat, a relative position of the seat and the steering wheel in a horizontal direction, a height of the steering wheel, an angle of the rearview mirror, a projection height of the HUD, and a scaling scale of the HUD, according to the height feature of the user, to adjust a state of the cabin and improve user comfort.

[0096] In some possible implementation manners, the vehicle controller can further acquire a dressing feature of the user according to the first image. As shown in Figure 6 illustrated, the vehicle controller can adjust parameters of an air conditioning system in the cabin according to the dressing feature of the user, the heart rate and blood oxygen data, and the temperature of the day, to adjust a state of the air conditioning in the cabin. The heart rate and blood oxygen data of the user can be obtained according to a smart device associated with the digital key, for example, a smart watch or a smart bracelet. The parameters of the air conditioning system include at least one of an air duct, an air volume, an air direction, a temperature, and a circulation type. In this way, the state of the air conditioning in the cabin is automatically adjusted according to the dressing of the user, and user comfort is improved, for example, high-intensity cooling can be avoided when the user is dressed in cool clothes.

[0097] The method provided by the embodiments of the present application also supports secondary adjustment of the parameters of the in-cabin facilities determined by the vehicle controller by the user. Specifically, the vehicle controller can further store the manually adjusted parameters as preferred parameters of the user in response to manual adjustment of the parameters of the in-cabin facilities by the user, and adjust the state of the cabin by using the preferred parameters when the user enters the cabin again. For example, the user can manually add a cushion, a pillow, or the like, and accordingly manually adjust the parameters of the in-cabin facilities. In this way, personalized needs of the user can be met, and the state of the cabin is automatically adjusted when the user enters the cabin again, improving the efficiency of adjustment. The vehicle controller can store multiple parameters obtained by multiple times of adjustment by the user, and present the multiple parameters when the user enters the cabin again, for the user to select. It should be noted that the vehicle controller can determine whether the user is entering the cabin for the first time according to an identity of the digital key.

[0098] In the method provided by the embodiments of the present application, the vehicle controller can also upload the user's preference parameters to the cloud server, and recommend parameter settings to the user according to the fusion preference parameters returned by the cloud server. The cloud server can group according to the attribute information (such as age, gender, etc.) provided by multiple users, for example, group the received data based on the K-Means clustering algorithm (K-Means). Then, the cloud server fuses the user's preference parameters in each group respectively, for example, adopts mean fusion to obtain multiple fused preference parameters, and each group corresponding to a fused preference parameter according to the attribute information provided by the user. In this way, the vehicle controller can identify the attributes of the user who first gets into the vehicle, and recommend parameter settings to the user according to the preference parameters corresponding to the user attributes, so that the method has universality.

[0099] In some possible implementation manners, the vehicle controller can also acquire images captured by the vehicle-mounted camera on the co-driver side or the rear row side in response to a door opening operation of the co-driver or the rear row, and perform target detection and feature extraction to adjust the parameters of the facilities in the cabin and adjust the cabin state. The method of performing target detection and feature extraction is as described in the foregoing embodiments, which will not be described here again.

[0100] Referring to Figure 7 , the vehicle controller can also determine the specific position where the user sits in the cabin in combination with sensors in the cabin, such as seat occupancy sensors, depth cameras, millimeter wave radars, etc., and record the parameters of the facilities in the cabin adjusted according to the user features. In this way, the user's trajectory is tracked by the depth camera, and when the user's position is detected to change, the parameters on the new position are migrated, and the parameters of the facilities in the cabin at the original position are copied to the new position where the user sits, so as to realize automatic adjustment of the cabin state. The depth camera detects the position of the user according to the depth value, and does not acquire specific images of the user in the cabin, thereby guaranteeing the in-cabin privacy of the user to the maximum extent.

[0101] Based on the above description, the embodiments of the present application provide a cabin state adjustment method. On the one hand, the method accurately extracts the height feature of the user by acquiring the user image and using the key point coordinates of the user detected according to the user image, the depth image constructed based on the user image, and the parameter information of the camera, and then adjusts the cabin state, thereby improving the accuracy of cabin adjustment. On the other hand, the method can automatically adjust the cabin state according to the extracted user features, thereby avoiding the cumbersome process of manually adjusting the cabin state every time the user gets into the cabin, improving the efficiency of state adjustment, and improving the user experience. In addition, the method collects images before the user gets into the cabin, thereby protecting the in-cabin privacy of the user.

[0102] The overall flow of the cabin state adjustment method of the present application is introduced below in combination with a scenario.

[0103] Referring to Figure 8 A flowchart of a cabin state adjustment method disclosed by the present application is shown. The method is applied to the scenario that a user enters a cabin from outside a vehicle in a parking state. The method mainly includes a key authentication and positioning phase, a visual detection phase, a local personalization phase, and a cloud personalization fusion phase.

[0104] In the key authentication and positioning phase, the vehicle controller can authenticate the user according to the digital key and position the user according to the positioning information of the digital key, such as the positioning information obtained by using Bluetooth technology or UWB technology. Then, the vehicle controller can obtain the key trajectory according to the positioning information of the digital key.

[0105] In the visual detection phase, the vehicle controller can obtain an image containing a pedestrian according to the vehicle-mounted camera and detect and track the action trajectory of the pedestrian in the image to obtain a visual trajectory. Then, the vehicle controller can match the visual trajectory with the key trajectory to determine the user. In response to a door opening operation, the vehicle controller can obtain an image including the user and extract feature information of the user using the image. The feature information of the user can include at least one of a height feature and a clothing feature.

[0106] In the local personalization phase, the vehicle controller can adjust the parameters of the facilities in the cabin according to the feature information extracted in the visual detection phase. For example, at least one of a seat, a steering wheel, a rearview mirror, and an air conditioner. The vehicle controller can also update the local personalization model according to the user's manual secondary adjustment.

[0107] In the cloud personalization fusion phase, the vehicle controller can periodically upload the parameters of the local personalization model and receive the fused model parameters from the cloud to update the local personalization. The cloud server can group the uploaded models based on the attribute information of the user, such as the user's gender, age, etc., and fuse the model parameters in each group to obtain the fused parameters.

[0108] In this way, the cabin adjustment method provided by the embodiments of the present application can accurately extract user features and automatically adjust the cabin state according to the extracted user features when the user enters the cabin from outside the vehicle in a parking scenario. In addition, the embodiments of the present application can fuse the model parameters in each group based on the user attribute information to obtain the fused parameters, and automatically recommend adjustment parameters for the user according to the fused parameters and the user attribute information.

[0109] The present application also provides a cabin state adjustment device. The device of the present application is described in detail below in combination with the drawings.

[0110] Referring to Figure 9 As shown in a structural schematic diagram of a cabin state adjustment device, the cabin state adjustment device 900 can include:

[0111] The communication module 902 is configured to acquire a first image captured by a camera located at the driver side when detecting a door opening operation at the driver side.

[0112] The detection module 904 is configured to perform human key point detection on the first image by using a pose detection algorithm to obtain pixel coordinates of key points of the user in the first image.

[0113] The detection module 904 is further configured to determine a first vertical plane according to the pixel coordinates of the key points and parameter information of the camera, and determine a second vertical plane according to a target detection frame in the first image, a depth image constructed based on the first image, and the parameter information of the camera, wherein the first vertical plane and the second vertical plane are both perpendicular to a ground plane.

[0114] The feature extraction module 906 is configured to extract a height feature of the user according to a fusion vertical plane obtained by fusing the first vertical plane and the second vertical plane, wherein the height feature includes at least one of a full-body height, a half-body height, a thigh length, a calf length, an arm length, or a small arm length.

[0115] The adjustment module 908 is configured to adjust parameters of facilities in the cabin according to the height feature of the user to adjust a cabin state.

[0116] In some possible implementation manners, the detection module 904 is specifically configured to:

[0117] determine a projection ray of the key point in a world coordinate system according to the pixel coordinates of the key point and the parameter information of the camera, wherein the projection ray passes through the key point and has a camera optical center as an apex, and then determine the first vertical plane according to the projection ray of the key point in the world coordinate system. The first vertical plane can be used as a projection plane of the key point in the world coordinate system, and thus the position information of the key point in the world coordinate system is obtained. In this way, the position information of the key point in the real world can be obtained by projecting the key point in the image into the world coordinate system according to the pixel coordinates of the key point in the image and the parameter information of the camera.

[0118] In some possible implementation manners, the key points of the user include a foot key point, for example, an ankle key point of the user. The detection module 904 is specifically configured to:

[0119] The projection ray of the key point of the user in the world coordinate system is determined according to the pixel coordinates of the key point of the user and the parameter information of the camera, wherein the projection ray of the key point of the user in the world coordinate system passes through the key point of the user, and the vertex is the camera optical center. Then, the first vertical plane is determined according to the intersection of the projection ray of the key point of the user in the world coordinate system and the ground plane. Since the coordinates of the key point of the user are relatively easy to determine and verify, the accuracy of the first vertical plane determined by the key point of the user and the parameter information of the camera is relatively high, and the method is feasible.

[0120] In some possible implementation manners, the detection module 904 is specifically configured to:

[0121] The projection point of the pixel in the target detection frame in the first image in the world coordinate system is determined according to the target detection frame in the first image, the depth image constructed based on the first image, and the parameter information of the camera. Then, the projection points are clustered, for example, a density-based spatial clustering of applications with noise (DBSCAN) algorithm can be used to obtain a plurality of grouping clusters. Next, the target cluster is determined according to the average distance between the projection points in each grouping cluster and the camera optical center, wherein the target cluster is the cluster with the minimum average distance. Finally, the second vertical plane is determined according to the target cluster. For example, the least square method can be used to fit the point set in the target cluster to obtain the second vertical plane. In this way, the depth information of the image can be fully utilized to determine a projection plane, and the key points in the image are projected into the world coordinate system to obtain the position information of the key points in the real world.

[0122] In some possible implementation manners, the detection module 904 can also be configured to:

[0123] The fusion vertical plane is obtained by fusing the first vertical plane and the second vertical plane. For example, the first normal vector of the first vertical plane and the second normal vector of the second vertical plane can be determined, and the first normal vector and the second normal vector are averaged, for example, the arithmetic mean, to obtain the fusion normal vector. Then, the fusion vertical plane can be determined according to the intersection line of the first vertical plane and the second vertical plane and the fusion normal vector. In this way, the vertical plane determined by fusing two methods can reduce the error caused by a single detection method, and a more accurate vertical plane is obtained.

[0124] In some possible implementation manners, the feature extraction module 906 is specifically configured to:

[0125] According to the projection ray of the key point in the world coordinate system and the fusion vertical plane, the projection point of the key point on the fusion vertical plane is determined, and then the height feature of the user is extracted according to the projection point of the key point on the fusion vertical plane. The height feature includes at least one of the full-body height, the half-body height, the thigh length, the calf length, the arm length, or the small arm length.

[0126] In some possible implementation manners, the feature extraction module 906 can also be configured to:

[0127] The dressing feature of the user is acquired according to the first image, and the dressing feature of the user can be used to adjust the parameter of the air conditioner in the cabin, and adjust the state of the air conditioner in the cabin. In this way, the use comfort of the user can be improved, for example, high-intensity refrigeration can be avoided when the user wears cool clothes.

[0128] In some possible implementation manners, the adjustment module 908 can also be configured to:

[0129] In response to the user manually adjusting the parameter of the facility in the cabin, the manually adjusted parameter is stored as the preferred parameter of the user, and the preferred parameter is used to adjust the cabin state when the user enters the cabin again. In this way, the individual needs of the user can be met, and the cabin state can be automatically adjusted when the user enters the cabin again, improving the efficiency of adjustment.

[0130] In some possible implementation manners, the adjustment module 908 can also be configured to:

[0131] The cabin state is adjusted according to the plurality of fused preferred parameters and the attribute information of the user. Each fused preferred parameter is obtained by fusing a plurality of preferred parameters in each group in a plurality of groups in the cloud server, each group corresponds to one fused preferred parameter, and the plurality of groups are obtained based on the attribute information provided by the plurality of users.

[0132] In some possible implementation manners, the adjustment module 908 can also be configured to:

[0133] The cabin state of the corresponding moving position is automatically adjusted according to the movement of the user. Specifically, when the user is detected to move from a first position to a second position in the cabin based on the depth camera, the parameter of the facility in the second position is adjusted according to the parameter of the facility in the first position, and the cabin state is adjusted based on the parameter of the facility in the second position.

[0134] Based on the foregoing cabin state adjustment method and cabin state adjustment device, the application further provides a controller. The controller may be, for example, a vehicle control unit (VCU) or an electronic control unit (ECU). The controller comprises a processor and a memory. The memory stores computer readable instructions, and the processor is configured to execute the computer readable instructions to perform the foregoing cabin state adjustment method. In some examples, the controller is configured to implement the functions of the foregoing cabin state adjustment device.

[0135] The modules described above as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed over multiple virtual modules. Some or all of the modules can be selected as needed to achieve the purpose of the embodiment.

[0136] In addition, the functional modules in each embodiment of the application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module.

[0137] The integrated module, if realized in the form of a software functional module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application essentially contribute to the part or the whole or part of the technical solutions, which can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the processes of the methods described in the embodiments of the application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0138] The foregoing description of the disclosed embodiments enables a person skilled in the art to implement or use the application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for adjusting cockpit status, characterized in that, The method includes: When an opening operation is detected on the driver's side, the first image captured by the camera located on the driver's side is obtained; The first image is used to detect human key points by a pose detection algorithm to obtain the pixel coordinates of the user's key points in the first image. Based on the pixel coordinates of the key points and the parameter information of the camera, a first vertical plane is determined, and based on the target detection box in the first image, the depth image constructed based on the first image, and the parameter information of the camera, a second vertical plane is determined. Both the first vertical plane and the second vertical plane are perpendicular to the ground plane. Based on the fused vertical plane obtained by fusing the first vertical plane and the second vertical plane, the user's height features are extracted. The height features include at least one of the following: total height, half-body height, thigh length, calf length, arm length, or forearm length. Based on the user's height characteristics, the parameters of the cabin facilities are adjusted to regulate the cabin status.

2. The method according to claim 1, characterized in that, Determining the first vertical plane based on the pixel coordinates of the key points and the parameter information of the camera includes: The projection ray of the key point in the world coordinate system is determined based on the pixel coordinates of the key point and the parameter information of the camera. The projection ray passes through the key point and its vertex is the optical center of the camera. The first perpendicular plane is determined based on the projection rays of the key points in the world coordinate system.

3. The method according to claim 2, characterized in that, The key point includes the user's grounding key point, and the method includes: Based on the pixel coordinates of the user's grounding key point and the parameter information of the camera, the projection ray of the user's grounding key point in the world coordinate system is determined. The projection ray of the grounding key point in the world coordinate system passes through the grounding key point, and its vertex is the optical center of the camera. The first vertical plane is determined by the intersection of the projection ray of the user's grounding key point in the world coordinate system and the ground plane.

4. The method according to claim 1, characterized in that, Determining the second vertical plane based on the target detection bounding box in the first image, the depth image constructed based on the first image, and the parameter information of the camera includes: The projection points of pixels within the target detection box in the first image in the world coordinate system are determined based on the target detection box in the first image, the depth image constructed based on the first image, and the parameter information of the camera. Cluster the projection points of pixels within the target detection box in the first image in the world coordinate system to determine multiple grouping clusters; The target cluster is determined based on the average distance between the projection point in each group and the optical center of the camera, and the target cluster is the group with the smallest average distance. The second vertical plane is determined based on the target cluster.

5. The method according to claim 2, wherein extracting the user's height feature based on the fused vertical plane obtained by fusing the first vertical plane and the second vertical plane includes: The projection point of the key point on the fusion vertical plane is determined based on the projection ray of the key point in the world coordinate system and the fusion vertical plane. The user's height features are extracted based on the projection points of the key points onto the fusion vertical plane.

6. The method according to claim 1, characterized in that, The method further includes: The user's clothing characteristics are obtained from the first image; Based on the user's clothing characteristics, the parameters of the cabin air conditioning system are adjusted to regulate the state of the cabin air conditioning. The parameters of the air conditioning system include at least one of the following: air duct, air volume, air direction, temperature, and circulation type.

7. The method according to claim 1, characterized in that, The method further includes: In response to a user manually adjusting parameters of the cabin facilities, the manually adjusted parameters are stored as the user's preference parameters, which are used to adjust the cabin state when the user re-enters the cabin.

8. The method according to claim 7, characterized in that, The method further includes: The system receives multiple fused preference parameters sent by a cloud server. Each fused preference parameter is obtained by fusing multiple preference parameters within each of multiple groups. Each group corresponds to one fused preference parameter. The multiple groups are obtained based on attribute information provided by multiple users. The cockpit state is adjusted based on the multiple fused preference parameters and the user's attribute information.

9. The method according to claim 1, characterized in that, The method further includes: When the depth camera detects that the user has moved from a first position to a second position in the cockpit, the parameters of the facilities at the second position are adjusted according to the parameters of the facilities at the first position. The cabin state is adjusted according to the parameters of the facility at the second location.

10. A cockpit state adjustment device, characterized in that, The device includes: A communication module is used to acquire a first image captured by a camera located on the driver's side when an opening operation on the driver's side is detected; The detection module is used to detect human key points in the first image using a pose detection algorithm to obtain the pixel coordinates of the user's key points in the first image. The detection module is further configured to determine a first vertical plane based on the pixel coordinates of the key points and the parameter information of the camera, and to determine a second vertical plane based on the target detection box in the first image, the depth image constructed based on the first image, and the parameter information of the camera, wherein both the first vertical plane and the second vertical plane are perpendicular to the ground plane. The feature extraction module is used to extract the user's height features based on the fused vertical plane obtained by fusing the first vertical plane and the second vertical plane. The height features include at least one of the following: total height, half-body height, thigh length, calf length, arm length, or forearm length. The adjustment module is used to adjust the parameters of the cabin facilities according to the user's height characteristics in order to adjust the cabin state.

Images