Vehicle air conditioning control method, device, system, vehicle and readable storage medium

Abstract

The application discloses a vehicle air conditioner control method, device and system, a vehicle and a readable storage medium, and relates to the technical field of vehicles, to flexibly adjust the vehicle air conditioner and improve the user's vehicle riding experience. The method comprises the following steps: acquiring video frame data in the vehicle, wherein the video frame data comprises a target person; processing the video frame data by using a target detection model to obtain position information of the target person; determining a behavior feature of the target person; the behavior feature comprises a first behavior feature and a second behavior feature; the first behavior feature is used for indicating a gesture action feature of the target person, and the second behavior feature is used for indicating a body action feature of the target person; adjusting the wind direction and the air volume of a target air outlet based on the first behavior feature, and adjusting the set temperature of the vehicle air conditioner based on the second behavior feature; the target air outlet is an air outlet closest to the target person among a plurality of air outlets of the vehicle.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and specifically to a vehicle air conditioning control method, device, system, vehicle, and readable storage medium. Background Technology

[0002] With the rapid development of the automotive industry, vehicles can be equipped with increasingly rich in-vehicle infotainment functions, integrating digital instrument clusters, central control screens, and advanced driver assistance systems. Regarding human-machine interaction with the vehicle's air conditioning, buttons or taps on the central control screen can be used to control the air conditioning (such as adjusting the set temperature, air vent direction, and fan speed). Alternatively, the vehicle's air conditioning can be controlled via voice control.

[0003] However, because the vehicle has multiple air vents, the driver cannot precisely control the air volume and direction of each vent, and it relies on the driver's active control, lacking autonomous intelligent adjustment and flexibility. Summary of the Invention

[0004] The purpose of this invention is to provide a vehicle air conditioning control method, device, system, vehicle, and readable storage medium to flexibly adjust the vehicle air conditioning and improve the user's riding experience.

[0005] In a first aspect, a vehicle air conditioning control method is provided, the method comprising: acquiring video frame data inside the vehicle, the video frame data including a target person; processing the video frame data using a target detection model to obtain the location information of the target person; determining the behavioral characteristics of the target person; the behavioral characteristics including a first behavioral characteristic and a second behavioral characteristic; the first behavioral characteristic being used to indicate the hand gesture characteristics of the target person, and the second behavioral characteristic being used to indicate the body movement characteristics of the target person; adjusting the airflow direction and air volume of the target air conditioning vent based on the first behavioral characteristic, and adjusting the set temperature of the vehicle air conditioning based on the second behavioral characteristic; the target air conditioning vent being the vent closest to the target person among multiple air conditioning vents of the vehicle.

[0006] Based on the technical solution provided in this application, after processing video frame data inside the vehicle using a target detection model to obtain the location information of the target person in the video frame data, the gesture and body movement characteristics of the target person can be determined. The airflow direction and volume of the target air conditioning vent can be adjusted based on the target person's gesture characteristics, and the set temperature of the vehicle's air conditioning can be adjusted based on the target person's body movement characteristics. Since the target air conditioning vent is the vent closest to the target person among the vehicle's multiple air conditioning vents, the airflow and direction of each vent can be precisely controlled. Furthermore, because the target person's behavioral characteristics are autonomously adjusted based on the video frame data inside the vehicle, it does not rely on the driver's active control. This allows for intelligent adjustment of the vehicle's air conditioning set temperature, as well as the airflow and direction of the air conditioning vents, improving the flexibility of vehicle air conditioning control and enhancing the user experience.

[0007] Furthermore, the video data is processed using a target detection model to obtain the location information of the target person, including: determining multiple recognition boxes and their corresponding recognition box information; the recognition box information includes the location information of the recognition box and the category of the person to be identified within the recognition box; determining the target recognition box from the multiple recognition boxes based on the category of the person to be identified; the target recognition box includes the target person; and determining the location information of the target recognition box as the location information of the target person in the video frame.

[0008] Furthermore, determining the first behavioral feature and the second behavioral feature includes: determining the location information of key points of the target person in each frame of the video frame data; performing convolution processing on the location information of the key points of the target person to obtain the spatial features of the key points of the target person; determining the first behavioral feature and the second behavioral feature based on the target classification header and the spatial features of the key points of the target person; the target classification header includes a global average pooling layer and a fully connected layer.

[0009] Furthermore, the airflow direction and air volume of the target air conditioning vent are adjusted based on the first behavioral characteristics, including: determining the direction of the target person's hand gesture; adjusting the airflow direction of the target air conditioning vent when the direction is determined to be a first direction; and adjusting the air volume of the target air conditioning vent when the direction is determined to be a second direction; wherein the first direction and the second direction are different directions.

[0010] Furthermore, adjusting the vehicle air conditioning set temperature based on the second behavioral characteristics includes: determining the target person's body movement characteristics and a first mapping relationship, wherein the first mapping relationship includes different body movement characteristics and corresponding body movement types; the body movement type includes a first body movement type and a second body movement type; when the target person's body movement type is determined to be the first body movement type, performing a first adjustment operation on the vehicle air conditioning set temperature, wherein the set temperature after the first adjustment operation is higher than the set temperature before the adjustment; when the target person's body movement type is determined to be the second body movement type, performing a second adjustment operation on the vehicle air conditioning set temperature, wherein the set temperature after the second adjustment operation is lower than the set temperature before the adjustment.

[0011] Secondly, a vehicle air conditioning control device is provided, comprising: an acquisition unit, a processing unit, and a determination unit; the acquisition unit is used to acquire video frame data inside the vehicle, the video frame data including a target person; the processing unit is used to process the video frame data using a target detection model to obtain the location information of the target person; the determination unit is used to determine the behavioral characteristics of the target person; the behavioral characteristics include a first behavioral characteristic and a second behavioral characteristic; the first behavioral characteristic is used to indicate the hand gesture characteristics of the target person, and the second behavioral characteristic is used to indicate the body movement characteristics of the target person; the processing unit is further used to adjust the airflow direction and air volume of the target air conditioning vent based on the first behavioral characteristic, and to adjust the set temperature of the vehicle air conditioning based on the second behavioral characteristic; the target air conditioning vent is the vent closest to the target person among multiple air conditioning vents of the vehicle.

[0012] Furthermore, the processing unit is specifically used for: determining multiple recognition boxes and corresponding recognition box information; the recognition box information includes the position information of the recognition box and the category of the person to be identified within the recognition box; determining the target recognition box from the multiple recognition boxes according to the category of the person to be identified; the target recognition box includes the target person; and determining the position information of the target recognition box as the position information of the target person in the video frame.

[0013] Furthermore, the unit is specifically used to: determine the location information of key points of the target person in each frame of the video frame data; perform convolution processing on the location information of the key points of the target person to obtain the spatial features of the key points of the target person; determine the first behavioral feature and the second behavioral feature based on the target classification header and the spatial features of the key points of the target person; the target classification header includes a global average pooling layer and a fully connected layer.

[0014] Furthermore, the processing unit is specifically used for: determining the direction of the target person's hand gesture; adjusting the airflow direction of the target air conditioning vent when the determined direction is a first direction; and adjusting the airflow volume of the target air conditioning vent when the determined direction is a second direction; wherein the first direction and the second direction are different directions.

[0015] Furthermore, the processing unit is specifically configured to: determine the target person's body movement characteristics and a first mapping relationship, wherein the first mapping relationship includes different body movement characteristics and corresponding body movement types; the body movement type includes a first body movement type and a second body movement type; when the target person's body movement type is determined to be the first body movement type, perform a first adjustment operation on the set temperature of the vehicle's air conditioning, wherein the set temperature after the first adjustment operation is higher than the set temperature before the adjustment; when the target person's body movement type is determined to be the second body movement type, perform a second adjustment operation on the set temperature of the vehicle's air conditioning, wherein the set temperature after the second adjustment operation is lower than the set temperature before the adjustment.

[0016] Thirdly, a server is provided, comprising: a processor; a memory for storing processor-executable instructions; the processor being configured to execute instructions, functions performed in the first aspect or any possible design of the first aspect.

[0017] Fourthly, a vehicle air conditioning control system is provided, the vehicle air conditioning control system including a server, the server being used to perform methods as described in the first aspect or any possible design of the first aspect.

[0018] Fifthly, a vehicle is provided, including a vehicle air conditioning control system as provided in the fourth aspect.

[0019] Sixthly, a vehicle air conditioning control device is provided, which can realize the functions performed by the vehicle air conditioning control device in the above aspects or possible designs. The functions can be implemented by hardware. For example, in one possible design, the vehicle air conditioning control device may include a processor and a communication interface. The processor can be used to support the vehicle air conditioning control device in realizing the functions involved in the first aspect or any possible design of the first aspect.

[0020] In another possible design, the vehicle air conditioning control device may further include a memory for storing necessary computer execution instructions and data. When the vehicle air conditioning control device is running, the processor executes the computer execution instructions stored in the memory to cause the vehicle air conditioning control device to perform the first aspect or any of the possible vehicle air conditioning control methods described above.

[0021] In a seventh aspect, a computer-readable storage medium is provided, which may be a readable non-volatile storage medium storing computer instructions or programs that, when executed on a computer, enable the computer to perform the vehicle air conditioning control method described in the first aspect or any of the possible methods described above.

[0022] Eighthly, a computer program product containing instructions is provided, which, when run on a computer, enables the computer to execute the vehicle air conditioning control method of the first aspect or any possible design of the above aspects.

[0023] The beneficial effects of this invention are:

[0024] (1) After processing the video frame data inside the vehicle using a target detection model to obtain the location information of the target person in the video frame data, the gesture and body movement characteristics of the target person can be determined. Based on the gesture and body movement characteristics of the target person, the airflow direction and volume of the target air conditioning vent can be adjusted, and the set temperature of the vehicle air conditioning can be adjusted based on the body movement characteristics of the target person. Since the target air conditioning vent is the vent closest to the target person among the multiple air conditioning vents of the vehicle, the airflow and direction of each vent of the vehicle can be precisely controlled. Furthermore, since the target person's behavior characteristics are autonomously adjusted based on the video frame data inside the vehicle, it does not need to rely on the driver's active control. The set temperature of the vehicle air conditioning, as well as the airflow and direction of the vehicle air conditioning vents, can be intelligently adjusted, improving the flexibility of controlling the vehicle air conditioning and enhancing the user experience.

[0025] (2) The location information of the target personnel can be determined according to the target detection model, and then the target air conditioner outlet can be accurately determined and adjusted, thereby improving the user experience.

[0026] (3) The hand gestures and body movements of the target person can be determined based on the location information of the key points of the target person. Then, the air direction and air volume of the air conditioner vent can be adjusted based on the hand gestures of the target person, and the set temperature of the vehicle air conditioner can be adjusted based on the behavior characteristics of the target person.

[0027] (4) The direction and volume of the air vents of the target air conditioner can be adjusted according to the direction of the target person's hand gestures, which improves the flexibility of controlling the vehicle's air conditioner.

[0028] (5) The user’s body sensation type can be determined based on the target person’s body movement characteristics, and the vehicle’s air conditioning temperature can be adjusted proactively, thus improving the user experience.

[0029] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0031] Figure 1 This is a schematic diagram of the structure of a vehicle air conditioning control system provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the structure of another vehicle air conditioning control system provided in the embodiments of this application;

[0033] Figure 3 This is a schematic diagram of the structure of a vehicle air conditioning control device provided in an embodiment of this application;

[0034] Figure 4 A schematic flowchart illustrating a vehicle air conditioning control method provided in an embodiment of this application;

[0035] Figure 5 A schematic flowchart illustrating another vehicle air conditioning control method provided in this application embodiment;

[0036] Figure 6 This is a schematic diagram of the structure of another vehicle air conditioning control device provided in the embodiments of this application. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0038] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar personnel and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0039] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0040] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0041] The terms / nouns used in the embodiments of this invention are explained below.

[0042] 1. YOLOX model

[0043] It can be built on YOLOv5. Backbone and Neck also use New CSP-Darknet53, SPPF, and New CSP-PAN. The main change is that YOLOX uses different Headers for different predicted feature maps, that is, the parameters are not shared, which greatly improves the convergence speed of the network.

[0044] In the input phase, the input image is first resized to 640×640 and normalized to ensure a mean of 0 and a standard deviation of 1. Then, the input image undergoes Mosaic data augmentation to enrich the dataset and reduce the GPU load. Subsequently, adaptive anchor box calculation and adaptive image scaling are used to adjust the aspect ratio of the input image, improving inference speed.

[0045] In the Backbone section, the feature maps are first downsampled using a Focus structure to reduce computational cost while preserving as much original image information as possible. Then, CBL and CSP structures are used to enhance the CNN's learning ability, maintaining both lightweight design and accuracy while reducing computational and memory costs. Finally, the SPP module connects feature maps of different scales together.

[0046] In the Neck section, an FPN+PAN structure is adopted, which means that strong semantic features are conveyed from top to bottom and strong localization features are conveyed from bottom to top in a collaborative manner. Parameters of different detection layers are aggregated from different backbone layers, so that the target's location information and category information are preserved to the greatest extent.

[0047] In the Prediction part, since the detection head of the network has cls, reg, and obj branches that are decoupled, the loss is determined by L. cls L reg and L obj It consists of three parts, and the loss calculation formula is as follows:

[0048]

[0049] Among them, L cls L represents classification loss. reg L represents the localization loss. obj N represents the loss of object, λ represents the balance coefficient of the localization loss, and N represents the loss of object. pos This represents the number of anchor points that are divided into positive samples.

[0050] The target of object detection in this system is only human bodies. YOLOX's official Model Zoo contains many weight files that demonstrate good detection performance; therefore, the official weight parameters can be used directly without repeated training. Object detection is implemented using the mmaction2 and mmdetection frameworks with YOLOX.

[0051] 2. SlowFast Model

[0052] First, high-frequency and low-frequency image data are acquired separately. A dual-channel branch extraction method is used to extract image features: both the Slow and Fast channels use a 3D RestNet model, and 3D convolution operations are performed immediately after capturing several frames. The Slow channel uses a large temporal span, set to 16, meaning approximately 2 frames can be captured per second. The Fast channel uses a very small temporal span τ / α, where α is set to 8, allowing for 15 frames to be captured per second. The Fast channel uses a much smaller convolution width to maintain lightweight operation, set to 1 / 8 of the Slow channel's convolution width; this value is denoted as β. The reason for using a smaller convolution width is that the Fast channel requires 4 times less computation than the Slow channel, despite its higher temporal frequency.

[0053] For feature fusion, data from the Fast channel is fed into the Slow channel via lateral connections, allowing the Slow channel to understand the processing results of the Fast channel. The Fast channel has a shape of {αT,S2,βC}, while the Slow channel has a shape of {T,S2,αβC}. Therefore, the results from the Fast channel are transformed and then incorporated into the Slow channel. Time-strided convolution is used for this data transformation.

[0054] Finally, at the end of each channel, SlowFast performs global average pooling, then combines the results from the two channels and feeds them into a fully connected classification layer that uses softmax to identify actions occurring in the image.

[0055] We used our own captured dataset for training to obtain weight parameters. We then employed the mmaction2 and mmdetection frameworks to call the SlowFast model, performing behavior recognition based on YOLOX object detection, thus constructing an overall YOLOX+SlowFast behavior recognition model.

[0056] With the rapid development of the automotive industry, vehicles can be equipped with increasingly rich in-vehicle infotainment functions, integrating digital instrument clusters, central control screens, and advanced driver assistance systems. Regarding human-machine interaction with the vehicle's air conditioning, buttons or taps on the central control screen can be used to control the air conditioning (such as adjusting the set temperature, air vent direction, and fan speed). Alternatively, the vehicle's air conditioning can be controlled via voice control.

[0057] However, because the vehicle has multiple air vents, the driver cannot precisely control the air volume and direction of each vent, and it relies on the driver's active control, lacking autonomous intelligent adjustment and flexibility.

[0058] In view of this, embodiments of this application provide a vehicle air conditioning control method, the method comprising:

[0059] Acquire video frame data inside the vehicle; process the video frame data using a target detection model to obtain the location information of the target person in the video frame data; determine the behavioral characteristics of the target person; the behavioral characteristics include a first behavioral characteristic and a second behavioral characteristic; the first behavioral characteristic is used to indicate the target person's hand gesture characteristics, and the second behavioral characteristic is used to indicate the target person's body movement characteristics; adjust the airflow direction and air volume of the target air conditioning vent based on the first behavioral characteristic, and adjust the set temperature of the vehicle's air conditioning based on the second behavioral characteristic; the target air conditioning vent is the vent closest to the target person among the multiple air conditioning vents of the vehicle.

[0060] It should be noted that the vehicle air conditioning control system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of vehicle air conditioning control systems and the emergence of other vehicle air conditioning control systems, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0061] The vehicle air conditioning control system provided in this application can be applied to a server. This application does not limit the specific technology, quantity, or form of the server used.

[0062] Figure 1 This is a schematic diagram of the composition of a vehicle air conditioning control system 10 provided in an embodiment of this application, as shown below. Figure 1 As shown, the vehicle air conditioning control system 10 may include a vehicle 11 and a server 12.

[0063] Vehicle 11 and server 12 are connected. For example, vehicle 11 and server 12 can be connected wirelessly or via a wired connection; this embodiment of the invention does not limit the connection.

[0064] Vehicle 11 is used to acquire video frame data from inside the vehicle and send the video frame data to server 12. For example, vehicle 11 can be a new energy vehicle, a hybrid vehicle, or a fuel vehicle. The embodiments of this application do not limit the specific technology, quantity, or form of equipment used in vehicle 11.

[0065] It should be noted that vehicle 11 is equipped with a vehicle air conditioner, which can have multiple air vents. The number and location of these vents can be configured as needed. For example, there can be 7, 8, or 9 air vents. These vents can be located in front of each vehicle seat.

[0066] After acquiring video frame data inside the vehicle, server 12 processes the video frame data using a target detection model to obtain the location information of the target person in the video frame data; further, it determines the behavioral characteristics of the target person, adjusts the airflow direction and air volume of the target air conditioning vent based on the first behavioral characteristics, and adjusts the set temperature of the vehicle air conditioning based on the second behavioral characteristics.

[0067] The server 12 can also be referred to as a vehicle computing platform or a cloud platform. The server 12 can be a single server or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This application does not limit the specific technology, quantity, or form of the server 12.

[0068] Figure 2 This is a schematic diagram of the composition of a vehicle air conditioning control system provided in an embodiment of this application, as shown below. Figure 2 As shown, the vehicle air conditioning control system may include a camera module, a computing platform module, a target detection model module, a behavior recognition model module, an air conditioning control module, and an execution module.

[0069] The camera module can be used to collect video frame data from inside the vehicle. For example, it can be used as a camera in an in-cabin monitoring system (IMS) for automobiles.

[0070] The computing platform module can be used to process video frame data collected from inside the vehicle. For example, it can include normalization processing.

[0071] The target detection model module can be used to detect the location information of target personnel. For example, the target detection model can be the YOLOX model.

[0072] The behavior recognition model module can be used to determine the behavioral characteristics of a target person. For example, the behavior recognition model could be SlowFast.

[0073] The air conditioning control module can be used to generate adjustment commands, which are used to request adjustments to the vehicle's air conditioning set temperature, airflow from the vents, and the direction of the vents.

[0074] The execution module can be used to adjust instructions based on execution.

[0075] It should be noted that, Figure 1 and Figure 2 This is just an example framework diagram. Figure 1 and Figure 2 The names of the modules included are unrestricted, and except for Figure 1 and Figure 2 In addition to the functional modules shown, other modules may also be included, but this application embodiment does not limit this.

[0076] In practical implementation, Figure 1 The server in the middle can be adopted Figure 3 The shown composition structure, or including Figure 3 The components shown. Figure 3 This is a schematic diagram of the structure of a vehicle air conditioning control device 200 provided in an embodiment of this application. The vehicle air conditioning control device 200 can be a server in a vehicle air conditioning control system, or it can be a chip or system-on-a-chip in a server. Figure 3 As shown, the vehicle air conditioning control device 200 includes a processor 201, a communication interface 202, and a communication line 203.

[0077] Furthermore, the vehicle air conditioning control device 200 may also include a memory 204. The processor 201, the memory 204, and the communication interface 202 can be connected via a communication line 203.

[0078] The processor 201 can be a CPU, a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 201 can also be other devices with processing capabilities, such as circuits, devices, or software modules, without limitation.

[0079] Communication interface 202 is used to communicate with other devices or other communication networks. Communication interface 202 can be a module, circuit, communication interface, or any device capable of enabling communication.

[0080] Communication line 203 is used to transmit information between the various components included in the vehicle air conditioning control device 200.

[0081] Memory 204 is used to store instructions executable by processor 201. These instructions may be computer programs.

[0082] The memory 204 can be a read-only memory (ROM) or other type of static storage device that can store static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device that can store information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, etc., without limitation.

[0083] It should be noted that the memory 204 can exist independently of the processor 201 or can be integrated with the processor 201. The memory 204 can be used to store instructions, program code, or some data, etc. The memory 204 can be located inside or outside the vehicle air conditioning control device 200, without limitation. The processor 201 is used to execute the instructions stored in the memory 204 to implement the vehicle air conditioning control method provided in the following embodiments of this application.

[0084] In one example, processor 201 may include one or more CPUs, for example, Figure 3 CPU0 and CPU1 in the CPU.

[0085] As an optional implementation, the vehicle air conditioning control unit 200 includes multiple processors, for example, in addition to Figure 3 In addition to processor 201, it may also include processor 205.

[0086] It should be pointed out that, Figure 3 The composition shown does not constitute a basis for this. Figure 1 The limitations of each device in the process, except Figure 3 In addition to the components shown, Figure 1 The server in the middle can include more Figure 3 More or fewer components, or combinations of certain components, or different arrangements of components.

[0087] In this embodiment of the application, the chip system may be composed of chips or may include chips and other discrete devices.

[0088] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0089] The following is combined with Figure 1 The vehicle air conditioning control system shown herein describes the vehicle air conditioning control method provided in the embodiments of this application.

[0090] This application uses an example of an application to a vehicle air conditioning control device for illustration. For instance, the vehicle air conditioning control device can be... Figure 1 Server 12 in the middle. For example... Figure 4 As shown, the method includes the following S301-S304:

[0091] S301, Obtain video frame data inside the vehicle.

[0092] The video frame data includes the target person, which can be one or more passengers inside the vehicle. The video frame data can be video data of a preset length, such as 10 seconds or 1 minute.

[0093] As one possible implementation, the server can acquire video frame data from inside the vehicle via one or more camera devices inside the vehicle.

[0094] For example, the shooting device can be an IMS camera.

[0095] In some embodiments, video frame data may include front-seat occupants inside the vehicle.

[0096] In other embodiments, the video frame data may include all occupants inside the vehicle.

[0097] In practical applications, the location of the camera can be set as needed. For example, it can be placed above the windshield of the vehicle, on the ceiling inside the vehicle, or in the rear seats, etc.

[0098] S302. Use the target detection model to process the video frame data to obtain the location information of the target personnel.

[0099] The target detection model can be configured as needed. For example, it can be the YOLOX model, YOLO5 model, etc. Target personnel can be passengers inside the vehicle and the driver. Location information can be coordinates based on a preset vehicle coordinate system.

[0100] As one possible implementation, the server can input video frame data into the target detection model to obtain the location information of the target person in the video frame data.

[0101] It should be noted that a detailed introduction to the YOLOX model can be found in the glossary section above, and will not be repeated here.

[0102] In some embodiments, to improve the processing efficiency of the object detection model, the server can normalize the video frame data. For example, the image size can be adjusted to a target specification, the mean of the video frame data can be adjusted to a target mean, and the standard deviation of the video frame data can be adjusted to a target standard deviation.

[0103] The target size, target mean, and target standard deviation can be set as needed. For example, the target size can be 640*640, the target mean can be 0, and the target standard deviation can be 1.

[0104] S303. Determine the behavioral characteristics of the target personnel.

[0105] Among them, behavioral characteristics include first behavioral characteristics and second behavioral characteristics; the first behavioral characteristics are used to indicate the hand gesture characteristics of the target person, and the second behavioral characteristics are used to indicate the body movement characteristics of the target person.

[0106] For example, hand gestures can include raising a hand, pressing a hand down, waving a hand to the left, waving a hand to the right, etc. Body movements can include wiping sweat, rubbing hands together while exhaling, putting on clothes, and trembling.

[0107] As one possible implementation, the server can use behavior recognition models to determine the behavioral characteristics of the target person.

[0108] For example, the server can use a behavior recognition model to determine the location information of key points of the target person in each frame of video frame data, and perform convolution processing on the location information of the key points of the target person to obtain the spatial features of the key points of the target person; further, the first behavior feature and the second behavior feature are determined based on the target classification head and the spatial features of the key points of the target person.

[0109] It should be noted that the behavior recognition model can be a SlowFast model. Keypoints can be multiple skeletal points of the target person. The target classification head includes a global average pooling layer and a fully connected layer.

[0110] In some embodiments, the server can acquire multiple sets of sample data and train a preset SlowFast model based on the multiple sets of sample data to obtain a behavior recognition model.

[0111] The multiple sets of sample data can include multiple sets of sample videos and corresponding sample behaviors.

[0112] The number of sample data can be set as needed. For example, it can be 1000, 9000, etc. The length of the sample video can be 1 minute, 5 minutes, etc.

[0113] S304. Adjust the airflow direction and air volume of the target air conditioning vent based on the first behavioral feature, and adjust the set temperature of the vehicle air conditioning based on the second behavioral feature.

[0114] Among them, the target air conditioning vent is the vent that is closest to the target person among the multiple air conditioning vents of the vehicle.

[0115] As one possible implementation, the server determines the direction of the target person's gesture and determines the airflow direction and volume of the target air conditioner vent based on the direction of the gesture.

[0116] For example, if the direction is determined to be the first direction, adjust the airflow direction of the target air conditioning vent.

[0117] The first direction can be horizontal. The first direction includes horizontal to the left and horizontal to the right.

[0118] In one example, if the server determines that the target person's gesture is horizontal to the left, it adjusts the airflow direction of the target air conditioner vent to the left. If the server determines that the target person's gesture is horizontal to the right, it adjusts the airflow direction of the target air conditioner vent to the right.

[0119] In another example, if the server determines that the target person's gesture is horizontal to the left, it adjusts the airflow direction of the target air conditioner vent to the right. If the server determines that the target person's gesture is horizontal to the right, it adjusts the airflow direction of the target air conditioner vent to the left.

[0120] If the direction is determined to be the second direction, adjust the airflow of the target air conditioning vent.

[0121] Here, the first direction and the second direction are different directions. The second direction can be a vertical direction. The second direction includes the vertically upward direction and the vertically downward direction.

[0122] In one example, if the server determines that the target person's gesture is vertically upward, it increases the airflow from the target air conditioning vent. If the server determines that the target person's gesture is vertically upward, it decreases the airflow from the target air conditioning vent.

[0123] In another example, if the server determines that the target person's gesture is vertically upward, it reduces the airflow from the target air conditioner vent. If the server determines that the target person's gesture is vertically upward, it increases the airflow from the target air conditioner vent.

[0124] Furthermore, the server adjusts the vehicle's air conditioning temperature setting based on the perceived body temperature.

[0125] The types of somatosensory sensations include a first type and a second type. The first type can be a cold sensation, and the second type can be a hot sensation.

[0126] For example, if the target person's perceived body temperature is determined to be a first type, the vehicle's air conditioning temperature setting is adjusted first; if the target person's perceived body temperature is determined to be a second type, the vehicle's air conditioning temperature setting is adjusted second.

[0127] In the first adjustment operation, the set temperature is higher than the set temperature before the adjustment, and in the second adjustment operation, the set temperature is lower than the set temperature before the adjustment.

[0128] It should be noted that the adjustment granularity of the first and second adjustment operations can be set as needed. For example, it can be 1 degree, 2 degrees, etc.

[0129] In some embodiments, the server may also employ a random forest model to adjust the set temperature of the vehicle's air conditioning. For example, after determining the perceived body type of the target person, the server can use this as an input sample and combine it with video frame data of more target people from past time periods for a comprehensive judgment.

[0130] In some embodiments, the server can also identify more information such as the target person's clothing and identity based on machine vision-based driver and passenger attribute recognition methods, and input this information into a random forest model so that each decision tree in the random forest model can make judgments and classifications respectively, obtain their respective classification results, and adjust the set temperature of the vehicle's air conditioning through the majority voting principle.

[0131] Based on the technical solution provided in this application, after processing video frame data inside the vehicle using a target detection model to obtain the location information of the target person in the video frame data, the gesture and body movement characteristics of the target person can be determined. The airflow direction and volume of the target air conditioning vent can be adjusted based on the target person's gesture characteristics, and the set temperature of the vehicle's air conditioning can be adjusted based on the target person's body movement characteristics. Since the target air conditioning vent is the vent closest to the target person among the vehicle's multiple air conditioning vents, the airflow and direction of each vent can be precisely controlled. Furthermore, because the target person's behavioral characteristics are autonomously adjusted based on the video frame data inside the vehicle, it does not rely on the driver's active control. This allows for intelligent adjustment of the vehicle's air conditioning set temperature, as well as the airflow and direction of the air conditioning vents, improving the flexibility of vehicle air conditioning control and enhancing the user experience.

[0132] In some embodiments, such as Figure 5 As shown, S302 in the vehicle air conditioning control method of this application may specifically include the following S401-S403.

[0133] S401. Determine multiple recognition boxes and their corresponding recognition box information.

[0134] The identification box information includes the location information of the identification box and the category of the person to be identified within the identification box.

[0135] As one possible implementation, the server can identify all the people to be identified in the video frame based on the object detection model, and label the bounding boxes of all the people to be identified and the corresponding categories of the people to be identified.

[0136] S402. Determine the target identification box from multiple identification boxes based on the category of the person to be identified.

[0137] The target recognition box includes the target person.

[0138] As one possible implementation, the server can determine the target category from the categories of the person to be identified and remove other bounding boxes except for the target category to obtain the target bounding box.

[0139] It should be noted that the target category is passengers inside the vehicle.

[0140] S403. Determine the location information of the target identification box as the location information of the target person.

[0141] The target recognition box can be either the smallest bounding rectangle of the target person or the outline of the target person.

[0142] Based on the aforementioned technical means, the location information of the target personnel can be determined according to the target detection model, thereby accurately identifying the target air conditioning vent and adjusting the target air conditioning vent, thus improving the user experience.

[0143] The various solutions in the above embodiments of this application can be combined without contradiction.

[0144] This application embodiment can divide the vehicle air conditioning control device or vehicle air conditioning control device into functional modules or functional units according to the above method examples. For example, each function can be divided into a separate functional module or functional unit, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or in software functional modules or functional units. The module or unit division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0145] When dividing each function into modules according to its corresponding function. Figure 6 A schematic diagram of a vehicle air conditioning control device 600 is shown. The vehicle air conditioning control device 600 can be a controller or a chip applied in the controller. The vehicle air conditioning control device 600 can be used to perform the functions of the controller involved in the above embodiments. Figure 6 The vehicle air conditioning control device 600 shown may include: an acquisition unit 601, a processing unit 602, and a determination unit 603; the acquisition unit 601 is used to acquire video frame data inside the vehicle, the video frame data including a target person; the processing unit 602 is used to process the video frame data using a target detection model to obtain the location information of the target person; the determination unit 603 is used to determine the behavioral characteristics of the target person; the behavioral characteristics include a first behavioral characteristic and a second behavioral characteristic; the first behavioral characteristic is used to indicate the hand gesture characteristics of the target person, and the second behavioral characteristic is used to indicate the body movement characteristics of the target person; the processing unit 602 is also used to adjust the airflow direction and air volume of the target air conditioning vent based on the first behavioral characteristic, and to adjust the set temperature of the vehicle air conditioning based on the second behavioral characteristic; the target air conditioning vent is the vent closest to the target person among the multiple air conditioning vents of the vehicle.

[0146] Furthermore, the processing unit 602 is specifically used for: determining multiple recognition boxes and corresponding recognition box information; the recognition box information includes the position information of the recognition box and the category of the person to be recognized within the recognition box; determining the target recognition box from the multiple recognition boxes according to the category of the person to be recognized; the target recognition box includes the target person; and determining the position information of the target recognition box as the position information of the target person in the video frame.

[0147] Furthermore, the determination unit 603 is specifically used to: determine the location information of key points of the target person in each frame of the video frame data; perform convolution processing on the location information of the key points of the target person to obtain the spatial features of the key points of the target person; determine the first behavioral feature and the second behavioral feature based on the target classification header and the spatial features of the key points of the target person; the target classification header includes a global average pooling layer and a fully connected layer.

[0148] Furthermore, the processing unit 602 is specifically used to: determine the direction of the target person's hand gesture; adjust the airflow direction of the target air conditioning vent when the direction is determined to be a first direction; and adjust the airflow volume of the target air conditioning vent when the direction is determined to be a second direction; wherein the first direction and the second direction are different directions.

[0149] Furthermore, the processing unit 602 is specifically used to: determine the target person's body movement characteristics and a first mapping relationship, wherein the first mapping relationship includes different body movement characteristics and corresponding body movement types; the body movement type includes a first body movement type and a second body movement type; when the target person's body movement type is determined to be the first body movement type, perform a first adjustment operation on the set temperature of the vehicle's air conditioning, wherein the set temperature after the first adjustment operation is higher than the set temperature before the adjustment; when the target person's body movement type is determined to be the second body movement type, perform a second adjustment operation on the set temperature of the vehicle's air conditioning, wherein the set temperature after the second adjustment operation is lower than the set temperature before the adjustment.

[0150] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. This program can be stored in the computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be an internal storage unit of the vehicle air conditioning control device or controller (including a data transmitter and / or data receiver) of any of the foregoing embodiments, such as the hard disk or memory of the vehicle air conditioning control device. The computer-readable storage medium can also be an external storage device of the vehicle air conditioning control device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the vehicle air conditioning control device. Further, the computer-readable storage medium can include both internal storage units and external storage devices of the vehicle air conditioning control device. The computer-readable storage medium is used to store the computer program and other programs and data required by the vehicle air conditioning control device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0151] This application also provides a vehicle, including the vehicle air conditioning control system, server, or vehicle air conditioning control device involved in the above method embodiments.

[0152] Furthermore, the actions, terms, etc., involved in the various embodiments of this application can be referenced interchangeably without limitation. The message names or parameter names in the messages exchanged between the various devices in the embodiments of this application are merely examples, and other names may be used in specific implementations without limitation.

[0153] It should be noted that the terms "first" and "second," etc., in the specification, claims, and drawings of this application are used to distinguish different persons, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0154] It should be understood that in this application, "at least one (item)" means one or more, "more than" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related persons, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related persons before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0155] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0157] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0158] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0159] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0160] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vehicle air conditioning control method, characterized in that, include: Acquire video frame data inside the vehicle, the video frame data including the target personnel; The video frame data is processed using a target detection model to obtain the location information of the target person; Determining the behavioral characteristics of the target person includes: determining the location information of key points of the target person in each frame of the video frame data; The location information of the key points of the target person is processed by convolution to obtain the spatial features of the key points of the target person. First behavioral features and second behavioral features are determined based on the spatial features of the target classification head and the key points of the target person; the target classification head includes a global average pooling layer and a fully connected layer; the first behavioral features are used to indicate the hand gesture features of the target person, and the second behavioral features are used to indicate the body movement features of the target person; wherein, the hand gesture features include the direction information of the hand gesture, and the body movement features include limb movement information used to characterize the somatosensory state of the target person; Adjust the airflow direction and volume of the target air conditioner vent based on the first behavioral characteristic; The body sensation type of the target person is determined based on the target person's body movement characteristics and a first mapping relationship. The first mapping relationship includes different body movement characteristics and corresponding body sensation types. The body sensation type includes a first body sensation type and a second body sensation type. If the target person's body sensation type is determined to be the first body sensation type, the set temperature of the vehicle's air conditioning is adjusted first, and the set temperature after the first adjustment is higher than the set temperature before the adjustment. If the target person's body sensation type is determined to be the second body sensation type, the set temperature of the vehicle's air conditioning is adjusted in a second way, and the set temperature after the second adjustment is lower than the set temperature before the adjustment. The target air conditioning vent is the vent closest to the target person among the multiple air conditioning vents of the vehicle.

2. The vehicle air conditioning control method according to claim 1, characterized in that, The step of processing the video frame data using a target detection model to obtain the location information of the target person includes: Multiple recognition boxes and their corresponding recognition box information are determined; the recognition box information includes the position information of the recognition box and the category of the person to be identified within the recognition box; A target identification box is determined from the plurality of identification boxes based on the category of the person to be identified; the target identification box includes the target person. The location information of the target recognition box is determined as the location information of the target person in the video frame.

3. The vehicle air conditioning control method according to claim 1, characterized in that, The adjustment of the airflow direction and volume of the target air conditioner vent based on the first behavioral characteristic includes: Determine the direction of the target person's hand gestures; If the direction is determined to be the first direction, adjust the airflow direction of the target air conditioning outlet; If the direction is determined to be the second direction, adjust the airflow of the target air conditioning vent; the first direction and the second direction are different directions.

4. A vehicle air conditioning control device, characterized in that, The device includes: an acquisition unit, a processing unit, and a determination unit; The acquisition unit is used to acquire video frame data inside the vehicle, and the video frame data includes the target personnel; The processing unit is used to process the video frame data using a target detection model to obtain the location information of the target person; The determining unit is used to determine the behavioral characteristics of the target person, including: determining the location information of key points of the target person in each frame of the video frame data; The location information of the key points of the target person is processed by convolution to obtain the spatial features of the key points of the target person. First behavioral features and second behavioral features are determined based on the spatial features of the target classification head and the key points of the target person; the target classification head includes a global average pooling layer and a fully connected layer; the first behavioral features are used to indicate the hand gesture features of the target person, and the second behavioral features are used to indicate the body movement features of the target person; wherein, the hand gesture features include the direction information of the hand gesture, and the body movement features include limb movement information used to characterize the somatosensory state of the target person; The processing unit is also used to adjust the airflow direction and air volume of the target air conditioner outlet based on the first behavioral feature; The processing unit is further configured to determine the body sensation type of the target person based on the target person's body movement characteristics and a first mapping relationship, wherein the first mapping relationship includes different body movement characteristics and corresponding body sensation types; the body sensation type includes a first body sensation type and a second body sensation type. If the target person's body sensation type is determined to be the first body sensation type, the set temperature of the vehicle's air conditioning is adjusted first, and the set temperature after the first adjustment is higher than the set temperature before the adjustment. If the target person's body sensation type is determined to be the second body sensation type, the set temperature of the vehicle's air conditioning is adjusted in a second way, and the set temperature after the second adjustment is lower than the set temperature before the adjustment. The target air conditioning vent is the vent closest to the target person among the multiple air conditioning vents of the vehicle.

5. The vehicle air conditioning control device according to claim 4, characterized in that, The processing unit is specifically used for: Multiple recognition boxes and their corresponding recognition box information are determined; the recognition box information includes the position information of the recognition box and the category of the person to be identified within the recognition box; The target identification box is determined from the plurality of identification boxes according to the category of the person to be identified; The target identification box includes the target person; The location information of the target recognition box is determined as the location information of the target person in the video frame.

6. A vehicle air conditioning control system, characterized in that, The vehicle air conditioning control system includes a vehicle air conditioning control device. The vehicle air conditioning control device is used to perform the method as described in any one of claims 1 to 3.

7. A vehicle, characterized in that, Includes the vehicle air conditioning control system as described in claim 6.

8. A computer-readable storage medium, characterized in that, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is capable of performing the method as described in any one of claims 1 to 3.

Images