Vehicle multimedia volume adjustment method, apparatus and device, and storage medium

Abstract

Disclosed in the present application are a vehicle multimedia volume adjustment method, apparatus and device, and a storage medium. The method comprises: when it is detected that a multimedia volume in a vehicle is greater than a preset volume, collecting multi-angle image data in the vehicle; on the basis of the multi-angle image data, constructing a three-dimensional human body model corresponding to a first user in the vehicle; on the basis of the three-dimensional human body model, performing state recognition on the first user in the vehicle, so as to obtain a current state corresponding to the first user in the vehicle; and if it is recognized that the current state is a sleep state, making an inquiry to a second user in the vehicle regarding whether to adjust the multimedia volume.

Description

Vehicle multimedia volume adjustment method, device, equipment and storage medium

[0001] This application claims priority to Chinese patent application No. 202410314859.3 filed on March 19, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of vehicle auxiliary technology, and in particular to a vehicle multimedia volume adjustment method, device, equipment and storage medium. Background Art

[0003] In-vehicle multimedia devices are multimedia systems installed in cars or other vehicles. These systems are usually used to provide entertainment, information and connectivity functions to improve the riding experience. However, during the use of in-vehicle multimedia devices, the user in the car may gradually fall asleep. If the multimedia volume of the in-vehicle multimedia device is high at this time, and the driver needs to focus on driving and cannot notice whether the user in the car has fallen asleep in time, the high multimedia volume may disturb the user in the car. Based on this, the industry is currently in urgent need of a method that can adjust the vehicle multimedia volume when the user in the car falls asleep.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Technical issues

[0005] The main purpose of this application is to provide a vehicle multimedia volume adjustment method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot adjust the vehicle multimedia volume when the user in the car falls asleep. Technical Solutions

[0006] To achieve the above objectives, the present application provides a method for adjusting the volume of a vehicle multimedia system, the method comprising the following steps:

[0007] When it is detected that the multimedia volume inside the vehicle is greater than a preset volume, collecting multi-angle image data of the interior of the vehicle;

[0008] constructing a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data;

[0009] Performing status recognition on the first user in the vehicle according to the three-dimensional human body model to obtain current statuses corresponding to the first user in the vehicle;

[0010] If it is identified that the current state is a sleeping state, the second user in the car is asked whether to adjust the multimedia volume.

[0011] In one embodiment, the step of constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the multi-angle image data includes:

[0012] performing data preprocessing on the multi-angle image data to obtain preprocessed image data;

[0013] Feature extraction is performed on the preprocessed image data to obtain human body features of the first user in the car, and a three-dimensional human body model corresponding to the first user in the car is constructed based on the human body features.

[0014] In one embodiment, the step of constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the human body features includes:

[0015] Performing time series segmentation on the human body features to obtain time series segments corresponding to the human body features;

[0016] Classifying the time sequence segments according to the time sequence information corresponding to each of the time sequence segments, and arranging the classified time sequence segments in time sequence to obtain time-sharing human body models, wherein the time-sharing human body models are human body models corresponding to the first user in the car in different time periods;

[0017] A three-dimensional human body model corresponding to the first user in the vehicle is constructed based on the time-sharing human body model.

[0018] In one embodiment, the step of performing data preprocessing on the multi-angle image data to obtain preprocessed image data includes:

[0019] resizing the images to images of the same size;

[0020] Normalizing the pixel values ​​of the images of the same size to obtain a standardized image;

[0021] The standardized image is smoothed or specific features in the standardized image are highlighted by Gaussian filtering or median filtering to obtain preprocessed image data.

[0022] In one embodiment, the step of performing feature extraction on the preprocessed image data to obtain body features of the first user in the vehicle, and constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the body features includes:

[0023] Construct a spatial rectangular coordinate system;

[0024] In the spatial rectangular coordinate system, the same feature points in the human body features are aligned and the different feature points are arranged in relative positions to obtain a three-dimensional human body model corresponding to the first user in the car.

[0025] In one embodiment, the step of identifying the state of the first user in the vehicle based on the three-dimensional human body model to obtain the current state corresponding to each of the first users in the vehicle includes:

[0026] Simplifying the three-dimensional human body model to obtain a simplified three-dimensional human body model;

[0027] Key features are extracted from the simplified three-dimensional human body model, and the state of the first user in the vehicle is identified based on the key features to obtain the current state corresponding to the first user in the vehicle.

[0028] In one embodiment, the key features include eye features, head features, back features, hip features, and leg features, and the step of identifying the state of the first user in the vehicle based on the key features to obtain the current state corresponding to each of the first users in the vehicle includes:

[0029] Performing an initial state recognition on the first user in the vehicle based on the eye feature to obtain a first state prediction value;

[0030] Performing secondary state recognition on the first user in the vehicle based on the head feature and the back feature to obtain a second state prediction value;

[0031] Performing three state recognitions on the first user in the vehicle based on the hip features and the leg features to obtain a third state prediction value;

[0032] The current states corresponding to the first user in the vehicle are determined respectively through the first state prediction value, the second state prediction value and the third state prediction value.

[0033] In one embodiment, the step of determining the current states corresponding to the first user in the vehicle respectively by using the first state prediction value, the second state prediction value, and the third state prediction value includes:

[0034] Assigning a first weight, a second weight, and a third weight to the first state prediction value, the second state prediction value, and the third state prediction value, respectively;

[0035] Adding a first product of the first state prediction value and the first weight, a second product of the second state prediction value and the second weight, and a third product of the third state prediction value and the third weight to obtain a weighted state prediction value;

[0036] If the weighted state prediction value is less than a preset threshold, determining that the current state corresponding to the first user in the vehicle is a non-sleeping state;

[0037] If the weighted state prediction value is greater than or equal to a preset threshold, it is determined that the current state corresponding to the first user in the vehicle is a sleeping state.

[0038] In one embodiment, the step of simplifying the human body three-dimensional model to obtain a simplified human body three-dimensional model includes:

[0039] The human body three-dimensional model is simplified by any one of dimensionality reduction technology, model clipping and model quantization to obtain a simplified human body three-dimensional model.

[0040] In one embodiment, if the current state is identified as a sleeping state, the step of asking the second user in the vehicle whether to adjust the multimedia volume includes:

[0041] When the current state is identified as a sleeping state, inquiring a second user in the vehicle via the headrest speaker and / or the vehicle screen whether to issue a multimedia volume adjustment instruction;

[0042] If so, the multimedia volume is gradually adjusted according to the multimedia volume adjustment instruction.

[0043] In one embodiment, the multi-angle image data is image data of the portrait features of the first user in the car at different angles.

[0044] In one embodiment, the current states corresponding to the first users in the vehicle include a sleeping state and a non-sleeping state.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle multimedia volume adjustment device, the vehicle multimedia volume adjustment device comprising:

[0046] a data acquisition module, configured to acquire multi-angle image data of the interior of the vehicle when detecting that the multimedia volume inside the vehicle is greater than a preset volume;

[0047] A model building module, configured to build a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data;

[0048] a state recognition module, configured to recognize the state of the first user in the vehicle according to the three-dimensional human body model, and obtain the current state corresponding to each of the first users in the vehicle;

[0049] The volume adjustment module is configured to, if it is recognized that the current state is a sleeping state, inquire a second user in the vehicle whether to adjust the multimedia volume.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a vehicle multimedia volume adjustment device, which includes: a memory, a processor, and a vehicle multimedia volume adjustment program stored in the memory and executable on the processor, wherein the vehicle multimedia volume adjustment program is configured to implement the steps of the vehicle multimedia volume adjustment method described above.

[0051] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a vehicle multimedia volume adjustment program is stored. When the vehicle multimedia volume adjustment program is executed by a processor, the steps of the vehicle multimedia volume adjustment method described above are implemented. Beneficial effects

[0052] This application collects multi-angle image data of the vehicle's interior when it detects that the multimedia volume inside the vehicle is greater than a preset volume; constructs a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data; identifies the state of the first user in the vehicle based on the three-dimensional human body model to obtain the current state corresponding to the first user in the vehicle; and if the current state is identified as sleeping, inquires the second user in the vehicle whether to adjust the multimedia volume. Compared to traditional vehicle multimedia volume adjustment methods, the above-mentioned method of the application uses the three-dimensional human body model corresponding to the first user in the vehicle constructed based on multi-angle image data to identify the state of the first user in the vehicle. Therefore, it can automatically identify whether the first user in the vehicle has entered a sleeping state without the driver's attention, and can adjust the vehicle multimedia volume in a timely manner when the first user in the vehicle falls asleep. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] FIG1 is a schematic diagram of the structure of a vehicle multimedia volume adjustment device in a hardware operating environment according to an embodiment of the present application;

[0054] FIG2 is a flow chart of a first embodiment of a method for adjusting vehicle multimedia volume according to the present application;

[0055] FIG3 is a flow chart of a second embodiment of a vehicle multimedia volume adjustment method according to the present application;

[0056] FIG4 is a flow chart of a third embodiment of a vehicle multimedia volume adjustment method according to the present application;

[0057] FIG5 is a structural block diagram of the first embodiment of the vehicle multimedia volume adjustment device of the present application.

[0058] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. Modes for Carrying Out the Invention

[0059] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0060] Refer to Figure 1, which is a schematic diagram of the structure of a vehicle multimedia volume adjustment device in the hardware operating environment involved in an embodiment of the present application.

[0061] As shown in Figure 1, the vehicle multimedia volume adjustment device may include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and memory 1005. Communication bus 1002 is used to facilitate communication between these components. User interface 1003 may include a display and an input unit, such as a keyboard. User interface 1003 may also include a standard wired interface or a wireless interface. Network interface 1004 may include a standard wired interface or a wireless interface, such as a wireless fidelity (Wi-Fi) interface. Memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. Memory 1005 may also be a storage device independent of processor 1001.

[0062] Those skilled in the art will appreciate that the structure shown in FIG1 does not limit the vehicle multimedia volume adjustment device, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0063] As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a vehicle multimedia volume adjustment program.

[0064] In the vehicle multimedia volume adjustment device shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the vehicle multimedia volume adjustment device of the present application can be set in the vehicle multimedia volume adjustment device. The vehicle multimedia volume adjustment device calls the vehicle multimedia volume adjustment program stored in the memory 1005 through the processor 1001 and executes the vehicle multimedia volume adjustment method provided in the embodiment of the present application.

[0065] An embodiment of the present application provides a method for adjusting the volume of vehicle multimedia. Referring to FIG. 2 , FIG. 2 is a flow chart of a first embodiment of the method for adjusting the volume of vehicle multimedia.

[0066] In this embodiment, the vehicle multimedia volume adjustment method includes the following steps:

[0067] Step S1: When it is detected that the multimedia volume inside the vehicle is greater than a preset volume, multi-angle image data of the interior of the vehicle is collected.

[0068] It should be noted that the execution entity of the method of this embodiment can be a terminal device with model building, data processing, and program execution functions, such as a smartphone or computer, or an electronic device with the same or similar functions, such as the aforementioned vehicle multimedia volume adjustment device. This embodiment and the following embodiments will be described below using a vehicle multimedia volume adjustment device (hereinafter referred to as the adjustment device) as an example.

[0069] It is understandable that the above-mentioned preset volume can be automatically defined by the user, such as 30 decibels, 40 decibels, 50 decibels, etc., and this embodiment does not limit this.

[0070] It should be understood that the multi-angle image data may be image data including portrait features of the first user in the car at different angles.

[0071] In a specific implementation, the multi-angle image data of the interior of the vehicle can be collected by a camera component inside the vehicle.

[0072] Step S2: constructing a three-dimensional human body model corresponding to the first user in the car based on the multi-angle image data.

[0073] It should be understood that the first user in the car can be any user in the car, such as the main driver, the co-driver, the rear passenger, etc., and this embodiment does not limit this.

[0074] It should be noted that the above-mentioned three-dimensional human body model can be a human body model with realistic appearance and movements constructed using computer technology and digital simulation technology. This model presents the structure and function of the human body in three-dimensional form, so that the human body's morphology and structure can be understood more intuitively.

[0075] In a specific implementation, three-dimensional information can be extracted from the multi-angle image data through image processing technology, and the three-dimensional model of the human body can be constructed based on the three-dimensional information.

[0076] Step S3: performing status recognition on the first user in the vehicle according to the three-dimensional human body model to obtain the current status corresponding to each of the first users in the vehicle.

[0077] It should be noted that the current states corresponding to the above-mentioned first users in the car may include sleeping states (such as light sleeping states, moderate sleeping states and deep sleeping states, etc.) and non-sleeping states (such as head movement states, hand movement states and leg movement states, etc.), or other states that can reflect the real-time characteristics of the first user in the car. This embodiment does not limit this.

[0078] In a specific implementation, the current activity status of the first user in the car can be determined based on the above-mentioned three-dimensional human body model, such as whether the first user in the car is leaning on the seat, whether his eyes are closed, whether his chest rises and falls smoothly, etc. Then, the state of the first user in the car can be identified based on the above-mentioned current activity status, so as to obtain the current state corresponding to the first user in the car.

[0079] Step S4: If it is identified that the current state is the sleeping state, the second user in the car is asked whether to adjust the multimedia volume.

[0080] It should be understood that the aforementioned second user in the vehicle can also be any user in the vehicle, but there are different priorities: the user with first priority can be the vehicle owner, the primary driver, etc., and the user with second priority can be the user in the vehicle who is awake (the user in the vehicle who is awake can be identified by scanning all occupants using the vehicle's camera assembly). The ownership of priority can also be customized by the vehicle owner, which is not detailed here.

[0081] In a specific implementation, a voice inquiry can be made to the second user in the car through a speaker whether to adjust the multimedia volume, or a text inquiry can be made to the second user in the car through a HUD (Head-Up Display) whether to adjust the multimedia volume. This embodiment does not impose any restrictions on this.

[0082] This embodiment collects multi-angle image data of the vehicle interior when it detects that the multimedia volume inside the vehicle is greater than a preset volume; constructs a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data; identifies the state of the first user in the vehicle based on the three-dimensional human body model to obtain the current state corresponding to the first user in the vehicle; and, if the current state is identified as sleeping, inquires the second user in the vehicle whether to adjust the multimedia volume. Compared to traditional vehicle multimedia volume adjustment methods, this embodiment automatically identifies the first user in the vehicle by using the three-dimensional human body model corresponding to the first user constructed from multi-angle image data. This allows the vehicle multimedia volume to be adjusted promptly when the first user falls asleep, without the driver's attention.

[0083] Refer to FIG3 , which is a flow chart of a second embodiment of a vehicle multimedia volume adjustment method according to the present application.

[0084] Based on the first embodiment, in this embodiment, in order to prevent irrelevant image data from affecting the efficiency of constructing the three-dimensional human body model, step S2 may include:

[0085] Step S21: performing data preprocessing on the multi-angle image data to obtain preprocessed image data.

[0086] In a specific implementation, the multi-angle image data can be preprocessed in the following ways: resizing, adjusting the images to the same size to ensure consistency of the input data; standardization, standardizing the image pixel values ​​to have zero mean and unit variance; and filtering, smoothing the image or highlighting specific features within the image through methods such as Gaussian filtering and median filtering. Of course, the above data preprocessing methods are for illustrative purposes only and are not intended to be limiting. Other methods for preprocessing multi-angle image data are also applicable to this embodiment and are not described in detail here.

[0087] Step S22: performing feature extraction on the preprocessed image data to obtain body features of the first user in the vehicle, and constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the body features.

[0088] In a specific implementation, a spatial rectangular coordinate system can be pre-constructed. Then, in this spatial rectangular coordinate system, the same feature points in the human features are aligned and the different feature points are arranged in relative positions, thereby constructing a three-dimensional human body model corresponding to the first user in the vehicle. More specifically, the definition of the same feature points can be explained by the following example: assuming that multi-angle image data A and multi-angle image data B correspond to the front face image and the left face image of the first user in the vehicle, respectively, but in reality, the facial feature points in the front face image and the facial feature points in the left face image often overlap. These overlapping points are the same feature points, and the non-overlapping points are the different feature points.

[0089] In this embodiment, in order to construct a more accurate three-dimensional human body model, step S22 may include:

[0090] Step S221: performing time-series segmentation on the human body features to obtain time-series segments corresponding to the human body features.

[0091] Step S222: classifying the time sequence segments according to the time sequence information corresponding to each of the time sequence segments, and arranging the classified time sequence segments in time sequence to obtain time-sharing human body models, wherein the time-sharing human body models are human body models corresponding to the first user in the car in different time periods.

[0092] Step S223: constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the time-sharing human body model.

[0093] In a specific implementation, the sliding window method can be used to perform temporal segmentation of human features, thereby obtaining the corresponding temporal segments of the human features. The first step is to determine the window length and the moving step size. The window length determines the frequency domain resolution; the longer the window, the higher the frequency domain resolution. The moving step size determines the temporal resolution; the smaller the step size, the higher the temporal resolution. The second step is to divide the above human features into multiple short-term windows based on the window length and the moving step size. The segments corresponding to these short-term windows are the temporal segments corresponding to the above human features.

[0094] Based on the first embodiment, in this embodiment, in order to improve the timeliness of multimedia volume adjustment, step S40 may include:

[0095] Step S41: When it is identified that the current state is the sleeping state, the second user in the vehicle is asked via the headrest speaker and / or the vehicle screen whether to issue a multimedia volume adjustment instruction.

[0096] Step S42: If yes, gradually adjust the multimedia volume according to the multimedia volume adjustment instruction.

[0097] This embodiment performs data preprocessing on the multi-angle image data to obtain preprocessed image data; performs feature extraction on the preprocessed image data to obtain human features of the first user in the car; performs time-series segmentation on the human features to obtain time segments corresponding to the human features; classifies the time segments according to the time information corresponding to each of the time segments, and arranges the classified time segments in chronological order to obtain time-sharing human body models, wherein the time-sharing human body models are human body models corresponding to the first user in the car in different time periods; constructs a three-dimensional human body model corresponding to the first user in the car based on the time-sharing human body model; when it is recognized that the current state is a sleeping state, inquires the second user in the car through the headrest speaker and / or the car screen whether to issue a multimedia volume adjustment instruction; if so, gradually adjusts the multimedia volume according to the multimedia volume adjustment instruction. Compared with the traditional vehicle multimedia volume adjustment method, the above method of this embodiment classifies and arranges the time segments obtained after time-series segmentation of human body features based on the sliding window method, thereby obtaining a time-sharing human body model (the human body model corresponding to the first user in the car in different time periods), and then can construct a more accurate three-dimensional human body model based on the time-sharing human body model.

[0098] Refer to FIG4 , which is a flow chart of a third embodiment of a vehicle multimedia volume adjustment method according to the present application.

[0099] Based on the above embodiments, in this embodiment, in order to obtain a more lightweight three-dimensional human body model and thus improve the efficiency of feature extraction, step S3 may include:

[0100] Step S31: simplifying the human body three-dimensional model to obtain a simplified human body three-dimensional model.

[0101] In a specific implementation, the above-mentioned three-dimensional human body model can be simplified by using dimensionality reduction technology, model clipping, model quantization, etc., which is not limited in this embodiment.

[0102] Step S32: extracting key features from the simplified three-dimensional human body model, and performing status recognition on the first user in the vehicle based on the key features to obtain the current status corresponding to each of the first users in the vehicle.

[0103] It should be noted that the above key features may include eye features, head features, back features, hip features and leg features.

[0104] In this embodiment, in order to more accurately determine the current status of the first user in the vehicle, step S32 may include:

[0105] Step S321: performing an initial state recognition on the first user in the vehicle based on the eye features to obtain a first state prediction value.

[0106] It is understandable that the above-mentioned eye features may include iris features, sclera features, eye movement features, eyelid features, eye corner features, etc., and this embodiment does not limit this.

[0107] Step S322: performing secondary state recognition on the first user in the vehicle based on the head feature and the back feature to obtain a second state prediction value.

[0108] It is understandable that the head features may include facial features, head pressure features, etc., and the back features may include back pressure features, etc., which are not limited in this embodiment.

[0109] Step S323: performing three state recognitions on the first user in the vehicle based on the hip features and leg features to obtain a third state prediction value.

[0110] It is understandable that the above-mentioned hip characteristics may include hip pressure characteristics, etc., and the above-mentioned leg characteristics may include leg pressure characteristics, etc., and this embodiment does not limit this.

[0111] Step S324: Determine the current state corresponding to the first user in the vehicle through the first state prediction value, the second state prediction value and the third state prediction value.

[0112] It's understandable that relying solely on the aforementioned eye, head, back, hip, and leg features alone cannot accurately determine the current status of the first user in the vehicle. For example, if eye features are used to determine that a passenger's eyes are closed for a certain period of time, the passenger may be listening to music with their eyes closed. Therefore, if the passenger is determined to be asleep and the multimedia volume is adjusted, the passenger's riding experience will be affected. Similarly, determining whether a passenger is asleep based on other individual features cannot guarantee accuracy and is not discussed further here.

[0113] In a specific implementation, the above-mentioned first state prediction value, the above-mentioned second state prediction value and the above-mentioned third state prediction value can be combined to determine the current state of the eyes, head, back, buttocks and legs of the first user in the car, thereby determining whether the first user in the car is in a sleeping state or a non-sleeping state.

[0114] In this embodiment, step S324 may include:

[0115] Step S3241: assign a first weight, a second weight, and a third weight to the first state prediction value, the second state prediction value, and the third state prediction value, respectively.

[0116] Step S3242: Add the first product of the first state prediction value and the first weight, the second product of the second state prediction value and the second weight, and the third product of the third state prediction value and the third weight to obtain a weighted state prediction value.

[0117] Step S3243: If the weighted state prediction value is less than a preset threshold, it is determined that the current state corresponding to the first user in the vehicle is a non-sleep state.

[0118] Step S3244: If the weighted state prediction value is greater than or equal to a preset threshold, it is determined that the current state corresponding to the first user in the vehicle is a sleeping state.

[0119] This embodiment simplifies the human body three-dimensional model to obtain a simplified human body three-dimensional model; extracts key features from the simplified human body three-dimensional model, wherein the key features include eye features, head features, back features, hip features and leg features; performs a first state recognition on the first user in the car based on the eye features to obtain a first state prediction value; performs a second state recognition on the first user in the car based on the head features and the back features to obtain a second state prediction value; performs a third state recognition on the first user in the car based on the hip features and the leg features to obtain a third state prediction value; and respectively The first state prediction value, the second state prediction value, and the third state prediction value are assigned a first weight, a second weight, and a third weight; a first product corresponding to the first state prediction value and the first weight, a second product corresponding to the second state prediction value and the second weight, and a third product corresponding to the third state prediction value and the third weight are added to obtain a weighted state prediction value; if the weighted state prediction value is less than a preset threshold, the current state corresponding to the first user in the vehicle is determined to be a non-sleeping state; if the weighted state prediction value is greater than or equal to the preset threshold, the current state corresponding to the first user in the vehicle is determined to be a sleeping state. Compared to traditional vehicle multimedia volume adjustment methods, the above method of this embodiment performs a total of three state recognitions on the first user in the vehicle based on the first user's eye features, head features, back features, hip features, and leg features, thereby determining the current state of the first user in the vehicle based on the obtained first state prediction value, second state prediction value, and the third state prediction value, thereby improving the recognition accuracy of the state recognition of the first user in the vehicle.

[0120] In addition, an embodiment of the present application further proposes a storage medium, on which a vehicle multimedia volume adjustment program is stored. When the vehicle multimedia volume adjustment program is executed by a processor, the steps of the vehicle multimedia volume adjustment method described above are implemented.

[0121] Refer to FIG. 5 , which is a structural block diagram of a first embodiment of a vehicle multimedia volume adjustment device according to the present application.

[0122] As shown in FIG5 , the vehicle multimedia volume adjustment device proposed in the embodiment of the present application includes:

[0123] The data acquisition module 501 is configured to acquire multi-angle image data of the interior of the vehicle when it is detected that the multimedia volume inside the vehicle is greater than a preset volume;

[0124] A model building module 502 is configured to build a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data;

[0125] A state recognition module 503 is configured to recognize the state of the first user in the vehicle according to the three-dimensional human body model, and obtain the current state corresponding to each of the first users in the vehicle;

[0126] The volume adjustment module 504 is configured to, if it is identified that the current state is the sleeping state, inquire the second user in the vehicle whether to adjust the multimedia volume.

[0127] This embodiment collects multi-angle image data of the vehicle interior when it detects that the multimedia volume inside the vehicle is greater than a preset volume; constructs a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data; identifies the state of the first user in the vehicle based on the three-dimensional human body model to obtain the current state corresponding to the first user in the vehicle; and, if the current state is identified as sleeping, inquires the second user in the vehicle whether to adjust the multimedia volume. Compared to traditional vehicle multimedia volume adjustment methods, this embodiment automatically identifies the first user in the vehicle by using the three-dimensional human body model corresponding to the first user constructed from multi-angle image data. This allows the vehicle multimedia volume to be adjusted promptly when the first user falls asleep, without the driver's attention.

[0128] Based on the first embodiment of the vehicle multimedia volume adjustment device described above, a second embodiment of the vehicle multimedia volume adjustment device described above is proposed.

[0129] In this embodiment, the model construction module 502 is also used to perform data preprocessing on the multi-angle image data to obtain preprocessed image data; perform feature extraction on the preprocessed image data to obtain the human body features of the first user in the car, and construct a three-dimensional human body model corresponding to the first user in the car based on the human body features.

[0130] In this embodiment, the model construction module 502 is also used to perform time-series segmentation on the human body features to obtain time-series segments corresponding to the human body features; classify the time-series segments according to the time-series information corresponding to each of the time-series segments, and arrange the classified time-series segments in chronological order to obtain a time-sharing human body model, which is a human body model corresponding to the first user in the car in different time periods; and construct a three-dimensional human body model corresponding to the first user in the car based on the time-sharing human body model.

[0131] In this embodiment, the model building module 502 is further configured to adjust the images to images of the same size; and to normalize the pixel values ​​of the images of the same size to obtain normalized images.

[0132] The standardized image is smoothed or specific features in the standardized image are highlighted by Gaussian filtering or median filtering to obtain preprocessed image data.

[0133] In this embodiment, the model construction module 502 is also used to construct a spatial rectangular coordinate system; in the spatial rectangular coordinate system, the same feature points in the human body features are aligned and the different feature points are arranged in relative positions to obtain a three-dimensional human body model corresponding to the first user in the car.

[0134] In this embodiment, the state recognition module 503 is also used to simplify the human body three-dimensional model to obtain a simplified human body three-dimensional model; extract key features from the simplified human body three-dimensional model, and perform state recognition on the first user in the car based on the key features to obtain the current state corresponding to the first user in the car.

[0135] In this embodiment, the key features include eye features, head features, back features, hip features and leg features. The state recognition module 503 is also used to perform an initial state recognition of the first user in the car based on the eye features to obtain a first state prediction value; perform a secondary state recognition of the first user in the car based on the head features and the back features to obtain a second state prediction value; perform a tertiary state recognition of the first user in the car based on the hip features and the leg features to obtain a third state prediction value; and determine the current state corresponding to the first user in the car through the first state prediction value, the second state prediction value and the third state prediction value.

[0136] In this embodiment, the state identification module 503 is also used to assign a first weight, a second weight and a third weight to the first state prediction value, the second state prediction value and the third state prediction value, respectively; add the first product corresponding to the first state prediction value and the first weight, the second product corresponding to the second state prediction value and the second weight, and the third product corresponding to the third state prediction value and the third weight to obtain a weighted state prediction value; if the weighted state prediction value is less than a preset threshold, it is judged that the current state corresponding to the first user in the car is a non-sleep state; if the weighted state prediction value is greater than or equal to the preset threshold, it is judged that the current state corresponding to the first user in the car is a sleep state.

[0137] In this embodiment, the state recognition module 503 is further configured to simplify the three-dimensional human body model by using any one of dimensionality reduction technology, model cropping, and model quantization to obtain a simplified three-dimensional human body model.

[0138] In this embodiment, the state recognition module 503 is also used to, when it is recognized that the current state is a sleeping state, inquire through the headrest speaker and / or the car screen whether the second user in the car has issued a multimedia volume adjustment instruction; if so, the multimedia volume is gradually adjusted according to the multimedia volume adjustment instruction.

[0139] In this embodiment, the multi-angle image data is image data of the portrait features of the first user in the car at different angles.

[0140] In this embodiment, the current states corresponding to the first users in the vehicle include a sleeping state and a non-sleeping state.

[0141] Other embodiments or specific implementations of the vehicle multimedia volume adjustment device of the present application can refer to the above-mentioned method embodiments and will not be repeated here.

[0142] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0143] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the existing technology, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0145] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for adjusting the volume of vehicle multimedia, wherein: The method comprises the following steps: When it is detected that the multimedia volume inside the vehicle is greater than a preset volume, collecting multi-angle image data of the interior of the vehicle; constructing a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data; Performing status recognition on the first user in the vehicle according to the three-dimensional human body model to obtain current statuses corresponding to the first user in the vehicle; If it is identified that the current state is a sleeping state, the second user in the car is asked whether to adjust the multimedia volume.

2. The vehicle multimedia volume adjustment method according to claim 1, wherein: The step of constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the multi-angle image data includes: performing data preprocessing on the multi-angle image data to obtain preprocessed image data; Feature extraction is performed on the preprocessed image data to obtain human body features of the first user in the car, and a three-dimensional human body model corresponding to the first user in the car is constructed based on the human body features.

3. The vehicle multimedia volume adjustment method according to claim 2, wherein: The step of constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the human body features includes: Performing time series segmentation on the human body features to obtain time series segments corresponding to the human body features; Classifying the time sequence segments according to the time sequence information corresponding to each of the time sequence segments, and arranging the classified time sequence segments in time sequence to obtain time-sharing human body models, wherein the time-sharing human body models are human body models corresponding to the first user in the car in different time periods; A three-dimensional human body model corresponding to the first user in the vehicle is constructed based on the time-sharing human body model.

4. The vehicle multimedia volume adjustment method according to claim 2, wherein: The step of performing data preprocessing on the multi-angle image data to obtain preprocessed image data comprises: resizing the images to images of the same size; Normalizing the pixel values ​​of the images of the same size to obtain a standardized image; The standardized image is smoothed or specific features in the standardized image are highlighted by Gaussian filtering or median filtering to obtain preprocessed image data.

5. The vehicle multimedia volume adjustment method according to claim 2, wherein: The step of extracting features from the preprocessed image data to obtain body features of the first user in the vehicle, and constructing a three-dimensional human body model corresponding to the first user in the vehicle based on the body features includes: Construct a spatial rectangular coordinate system; In the spatial rectangular coordinate system, the same feature points in the human body features are aligned and the different feature points are arranged in relative positions to obtain a three-dimensional human body model corresponding to the first user in the car.

6. The vehicle multimedia volume adjustment method according to claim 1, wherein: The step of identifying the state of the first user in the vehicle according to the three-dimensional human body model to obtain the current state corresponding to each of the first users in the vehicle includes: Simplifying the three-dimensional human body model to obtain a simplified three-dimensional human body model; Key features are extracted from the simplified three-dimensional human body model, and the state of the first user in the vehicle is identified based on the key features to obtain the current state corresponding to the first user in the vehicle.

7. The vehicle multimedia volume adjustment method according to claim 6, wherein: The key features include eye features, head features, back features, buttocks features, and leg features. The step of identifying the state of the first user in the vehicle based on the key features to obtain the current state corresponding to each of the first users in the vehicle includes: Performing an initial state recognition on the first user in the vehicle based on the eye feature to obtain a first state prediction value; Performing secondary state recognition on the first user in the vehicle based on the head feature and the back feature to obtain a second state prediction value; Performing three state recognitions on the first user in the vehicle based on the hip features and the leg features to obtain a third state prediction value; The current states corresponding to the first user in the vehicle are determined respectively through the first state prediction value, the second state prediction value and the third state prediction value.

8. The vehicle multimedia volume adjustment method according to claim 7, wherein: The step of determining the current states corresponding to the first user in the vehicle respectively according to the first state prediction value, the second state prediction value, and the third state prediction value includes: Assigning a first weight, a second weight, and a third weight to the first state prediction value, the second state prediction value, and the third state prediction value, respectively; Adding a first product of the first state prediction value and the first weight, a second product of the second state prediction value and the second weight, and a third product of the third state prediction value and the third weight to obtain a weighted state prediction value; If the weighted state prediction value is less than a preset threshold, determining that the current state corresponding to the first user in the vehicle is a non-sleeping state; If the weighted state prediction value is greater than or equal to a preset threshold, it is determined that the current state corresponding to the first user in the vehicle is a sleeping state.

9. The vehicle multimedia volume adjustment method according to claim 6, wherein: The step of simplifying the human body three-dimensional model to obtain a simplified human body three-dimensional model includes: The human body three-dimensional model is simplified by any one of dimensionality reduction technology, model clipping and model quantization to obtain a simplified human body three-dimensional model.

10. The vehicle multimedia volume adjustment method according to any one of claims 1 to 9, wherein: If the current state is identified as a sleeping state, the step of asking the second user in the vehicle whether to adjust the multimedia volume includes: When the current state is identified as a sleeping state, inquiring a second user in the vehicle via the headrest speaker and / or the vehicle screen whether to issue a multimedia volume adjustment instruction; If so, the multimedia volume is gradually adjusted according to the multimedia volume adjustment instruction.

11. The vehicle multimedia volume adjustment method according to claim 1, wherein: The multi-angle image data is image data of the portrait features of the first user in the car at different angles.

12. The vehicle multimedia volume adjustment method according to claim 1, wherein: The current states corresponding to the first users in the vehicle include a sleeping state and a non-sleeping state.

13. A vehicle multimedia volume adjustment device, wherein: The vehicle multimedia volume adjustment device comprises: a data acquisition module, configured to acquire multi-angle image data of the interior of the vehicle when detecting that the multimedia volume inside the vehicle is greater than a preset volume; A model building module, configured to build a three-dimensional human body model corresponding to a first user in the vehicle based on the multi-angle image data; a state recognition module, configured to recognize the state of the first user in the vehicle according to the three-dimensional human body model, and obtain the current state corresponding to each of the first users in the vehicle; The volume adjustment module is configured to, if it is recognized that the current state is a sleeping state, inquire a second user in the vehicle whether to adjust the multimedia volume.

14. A vehicle multimedia volume adjustment device, wherein: The device includes: a memory, a processor, and a vehicle multimedia volume adjustment program stored in the memory and executable on the processor, wherein the vehicle multimedia volume adjustment program is configured to implement the steps of the vehicle multimedia volume adjustment method according to any one of claims 1 to 12.

15. A storage medium, wherein: The storage medium stores a vehicle multimedia volume adjustment program, which, when executed by a processor, implements the steps of the vehicle multimedia volume adjustment method according to any one of claims 1 to 12.

Images