Automobile volume adjusting method, electronic equipment and storage medium

Abstract

The invention provides an automobile volume adjusting method, electronic equipment and a storage medium, and the method comprises the steps: obtaining training data, the training data comprises a plurality of user images marked with user states, and the user states comprise a no-disturb state and a non-no-disturb state; training the neural network model by using the training data to obtain a user state recognition model; acquiring a target image of a target user on the vehicle; identifying the target image by using a user state identification model, and determining the user state of the target user; and if the target user is in the non-disturbing state, adjusting the volume of a playing device at a position corresponding to the target user on the vehicle. The method can improve the efficiency and accuracy of identifying the passenger state, adjust the volume of the passenger position in time, and improve the riding experience of the passenger.

Description

Technical Field

[0001] This application relates to the field of vehicles, and particularly to a method for adjusting the volume of an automobile, an electronic device, and a storage medium. Background Art

[0002] Most vehicles are equipped with in-vehicle audio systems, and users can use the in-vehicle audio systems to play music, audiobooks, or navigation voices, etc. To avoid disturbing passengers, it is sometimes necessary to adjust the volume of the in-vehicle audio system. Currently, generally, the driver adjusts the volume of the in-vehicle audio system. Adjusting the volume of the in-vehicle audio system by the driver may affect the driving safety of the driver and may also fail to adjust the volume in a timely manner, affecting the riding experience. Summary of the Invention

[0003] Embodiments of this application disclose a method for adjusting the volume of an automobile, an electronic device, and a storage medium, which can solve the technical problem of poor riding experience of the target user.

[0004] This application provides a method for adjusting the volume of an automobile. The method includes: obtaining training data, where the training data includes multiple user images labeled with user states, and the user states include a do-not-disturb state and a non-do-not-disturb state; using the training data to train a neural network model to obtain a user state recognition model; obtaining a target image of a target user on the vehicle; using the user state recognition model to recognize the target image to determine the user state of the target user; if the target user is in the do-not-disturb state, adjusting the volume of the playback device at the corresponding position of the target user on the vehicle.

[0005] In some optional embodiments, the method further includes: obtaining user images without labeled user states as test data; inputting the test data into the user state recognition model, and calculating the accuracy rate of recognizing the user state according to the test results output by the user state recognition model; calculating the misjudgment rate of the user state recognition model based on the accuracy rate of recognizing the user state; if the misjudgment rate is greater than a first preset value, adding new training data and using the new training data to retrain the user state recognition model.

[0006] In some optional embodiments, the using the user state recognition model to recognize the target image to determine the user state of the target user includes: extracting feature data of multiple dimensions of the target image; inputting the feature data of multiple dimensions into the user state recognition model, and according to the output result of the user state recognition model, determining the probability value that the target user is in the do-not-disturb state; if the probability value is greater than or equal to a second preset value, determining that the target user is in the do-not-disturb state; if the probability value is less than the second preset value, determining that the target user is in the non-do-not-disturb state.

[0007] In some alternative embodiments, inputting the feature data of the multiple dimensions into the user status recognition model and determining the probability value that the target user is in the do-not-disturb status according to the output result of the user status recognition model includes: performing weighted summation on the probability values corresponding to the feature data of the multiple dimensions to obtain the probability value that the target user is in the do-not-disturb status.

[0008] In some alternative embodiments, if the target user is in the do-not-disturb status, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle includes: within a first preset time, if the number of times the target user is continuously recognized as being in the do-not-disturb status reaches a first preset number and the target user is located at a non-driver position, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle.

[0009] In some alternative embodiments, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle includes: reducing or turning off the volume of the playback device at the position corresponding to the target user.

[0010] In some alternative embodiments, after reducing or turning off the volume of the playback device at the position corresponding to the target user, the method further includes: within a second preset time, if the number of times the target user is continuously recognized as not being in the do-not-disturb status reaches a second preset number, increasing or turning on the volume of the playback device at the position corresponding to the target user.

[0011] In some alternative embodiments, the training data further includes user images labeled as being in a fatigued state or a distracted state, and the method further includes: if the target user is in a fatigued state or a distracted state and the target user is located at the driver position, increasing the volume of the playback device at the position corresponding to the target user.

[0012] This application also provides an electronic device, which includes a processor and a memory. When the processor executes a computer program stored in the memory, the automobile volume adjustment method as described above is implemented.

[0013] This application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the automobile volume adjustment method as described above is implemented.

[0014] Compared with the prior art, the car volume adjustment method provided by this application obtains training data, which includes multiple user images marked with user status. The user status includes a do-not-disturb status and a non-do-not-disturb status. The training data is used to train a neural network model to obtain a user status recognition model. The target image of the target user on the vehicle is obtained, and the user status recognition model is used to recognize the target image to determine the user status of the target user. If the target user is in the do-not-disturb status, the volume of the playback device at the corresponding position of the target user on the vehicle is adjusted. The car volume adjustment method provided by this application can improve the accuracy of recognizing the user status of the target user, effectively adjust the playback device at the position of the target user, and enhance the riding experience of the target user. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a schematic structural diagram of an electronic device provided by an embodiment of this application.

[0016] Figure 2 is a flowchart of the car volume adjustment method provided by an embodiment of this application.

[0017] Figure 3 is a schematic diagram of the car position identification provided by an embodiment of this application.

[0018] Figure 4 is a flowchart of the volume adjustment of the playback device provided by an embodiment of this application.

[0019] Figure 5 is a flowchart of the volume adjustment of the playback device provided by another embodiment of this application.

[0020] Figure 6 is a flowchart of the retraining of the user status recognition model provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] For ease of understanding, some explanations of concepts related to the embodiments of this application are given by way of example for reference.

[0022] It should be noted that "at least one" in this application means one or more, and "multiple" means two or more than two. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0023] In order to better understand the car volume adjustment method, electronic device and storage medium provided in the embodiments of the present application, the application scenario of the car volume adjustment method of the present application is first described below.

[0024] Figure 1 : is a schematic diagram of an application scenario of the car volume adjustment method provided in an embodiment of the present application. The car volume adjustment method provided in an embodiment of the present application is applied to an electronic device 1, and the electronic device 1 can be arranged on a car, and the electronic device 1 is connected to the vehicle-mounted control system in the car to control the electronic device 1 to identify the user status of the target user and adjust the car volume accordingly. The electronic device 1 includes a memory 11, at least one processor 12, a shooting device 13 and a plurality of playback devices 14 electrically connected to each other. In other embodiments, the electronic device can be an independent terminal device (for example, a mobile phone, a computer, etc.), connected to a shooting device 13 (for example, a camera) and a playback device 14 (for example, a car audio, etc.) inside the car. The shooting device 13 can be one or more cameras installed inside the car, and the playback device 14 can be multiple and correspond to different seats respectively. The electronic device may further include or be connected to an external display device, and the display device can be used to play videos, etc.

[0025] In addition, in other embodiments, the electronic device 1 may be an on-board device in a car and may be equipped with an on-board control system.

[0026] Those skilled in the art will understand that the Figure 1 It is only an example of the electronic device 1 and does not constitute a limitation of the electronic device 1. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.

[0027] Figure 2 is a flow chart of a method for adjusting the volume of a car provided in an embodiment of the present application. The method for adjusting the volume of a car is applied to an electronic device (e.g. Figure 1 According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0028] Step S201 : acquiring training data, wherein the training data includes a plurality of user images marked with user status, wherein the user status includes a do not disturb status and a non-do not disturb status.

[0029] The embodiment of the present application is to identify the user status of a target user in a car. The user image can be obtained by taking a picture with a camera, which can be a camera built into the car, and the user image in the training data can also be obtained by obtaining an existing face recognition data set or behavior recognition data set.

[0030] Exemplarily, the user images captured by the photographing device are subjected to user status annotation, and the user images are annotated as the do-not-disturb status and the non-do-not-disturb status. The do-not-disturb status may be that the passenger is on the phone, listening to music with headphones, closing eyes and sleeping, etc., and the non-do-not-disturb status may be that the passenger is sitting on the seat without other actions.

[0031] Step S202: Use the training data to train the neural network model to obtain a user status recognition model.

[0032] Using the training data to train the neural network model means inputting the training data into the neural network model for training to obtain a user status recognition model.

[0033] In an embodiment, the method for adjusting the vehicle volume further includes: obtaining user images with unannotated user status as test data; inputting the test data into the user status recognition model, and calculating the accuracy rate of recognizing the user status according to the test results output by the user status recognition model; calculating the misjudgment rate of the user status recognition model based on the accuracy rate of recognizing the user status; if the misjudgment rate is greater than a first preset value, adding new training data, and using the new training data to retrain the user status recognition model.

[0034] Obtaining user images with unannotated user status as test data, and the user images with unannotated user status may be user images captured by the in-vehicle camera device. Input a preset number of test data into the user status recognition model. The test data is not subjected to status annotation before being input into the user status recognition model. Calculate the accuracy rate of recognizing the user status according to the test results output by the user status recognition model. For example: the number of input test data is 10, and the test results output by the user status recognition model include the annotation of the user status. The 10 input user images are annotated and divided into the do-not-disturb status and the non-do-not-disturb status, and calculate the accuracy rate of the test results annotated by the user status recognition model. The accuracy rate can be calculated after being recognized by other recognition models, or can be obtained by manually judging the test results output by the user status recognition model. Calculate the misjudgment rate of the user status recognition model based on the accuracy rate obtained by the above method.

[0035] The misjudgment rate includes the false positive rate (FPR) and the false negative rate (FNR). Input the test data into the user status recognition model for testing, and calculate the false positive rate and the false negative rate according to the accuracy rate obtained from the output results of the user status recognition model.

[0036] The false positive rate refers to the positive prediction error rate, which is the proportion of all negative classes predicted as positive. For example, when obtaining a target image of a target user, the proportion of the target user in the target image who is not on the phone but is misjudged as on the phone, that is, the proportion of the non-Do Not Disturb state being judged as the Do Not Disturb state.

[0037] The pseudo negative rate refers to the negative class prediction error rate, which is the proportion of all positive classes predicted as negative classes. For example, when obtaining a target image of a target user, the proportion of the target user in the target image who is talking on the phone but is misjudged as not talking on the phone, that is, the proportion of the Do Not Disturb state being judged as the non-Do Not Disturb state.

[0038] Exemplarily, the preset indicator may be 1%. If any ratio of the false positive rate and the false negative rate is greater than 1%, it is determined that the user state recognition model does not accurately recognize the user state of the target user.

[0039] In one embodiment of the present application, data that causes inaccurate recognition is marked, new training data is obtained, and the user state recognition model is retrained using the new training data.

[0040] In one example, the recognition accuracy of the user state recognition model may be affected by factors such as race, skin color, light, and weather. In order to improve the accuracy of the user state recognition model, the recognition results of the user state recognition model and the data input to the user state recognition model are transmitted back to the cloud backend, where the data is analyzed and the data causing inaccurate recognition is extracted and marked. Based on the marked data, new training data is added and the user state recognition model is retrained to improve the recognition accuracy of the user state recognition model.

[0041] For example, for the same target user, different results are recognized for the same behavior under different lighting conditions. It is determined that the behavior is inaccurate due to lighting, and new training data under different lighting conditions are added. The user state recognition model is retrained using the training data under different lighting conditions to obtain the retrained user state recognition model, thereby improving the efficiency of user state recognition.

[0042] Step S203, obtaining a target image of a target user in the vehicle.

[0043] During the driving process, the car can capture target images of multiple target users through a camera, and the camera can be a camera built into the car. The purpose of acquiring the target images is to identify the user status of the target user, so as to timely adjust the volume of the playback device at the location of the target user according to the user status.

[0044] Identify the opening or closing of any car door. When the opening or closing of the car door is recognized, an image inside the car is captured by an electronic device inside the car, and the electronic device can be a shooting device inside the car. Using the captured image inside the car, determine the number of people inside the car. When the number of people inside the car is recognized as 1, it is determined that there are no passengers in the car, and the target image of the driver is obtained. When the driver selects to turn on a playback device such as music or an audiobook, the sound output of only the playback device on the driver's seat can be turned on, reducing the operation of other playback devices and increasing the lifespan of the playback devices. When the number of people inside the car is recognized as greater than 1, it is determined that there are passengers in the car, and the target image of the driver and the target images of the passengers are obtained through the electronic device, and multiple target images of multiple target users can be obtained simultaneously.

[0045] Step S204, use the user status recognition model to recognize the target image and determine the user status of the target user.

[0046] Taking the single target image of a single target user obtained as an example, taking the single target user obtained as a passenger as an example. Obtain the target image captured by the shooting device of the target user, and use the trained user status recognition model to recognize the target image to obtain the user status of the target user.

[0047] In one embodiment, the using the user status recognition model to recognize the target image and determine the user status of the target user includes: extracting feature data of multiple dimensions of the target image; inputting the feature data of multiple dimensions into the user status recognition model, and according to the output result of the user status recognition model, determining the probability value that the target user is in the do-not-disturb state; if the probability value is greater than or equal to a second preset value, determining that the target user is in the do-not-disturb state; if the probability value is less than the second preset value, determining that the target user is in the non-do-not-disturb state.

[0048] In one embodiment, the determining the probability value that the target user is in the do-not-disturb state according to the output result of the user status recognition model includes: performing weighted summation on the probability values corresponding to the feature data of multiple dimensions to obtain the probability value that the target user is in the do-not-disturb state.

[0049] In this embodiment, obtain the target image captured by the shooting device, obtain the feature data of multiple dimensions in the target image, input the feature data of different dimensions into the user status recognition model, recognize the user status of the target user, output the recognition results of the feature data of each dimension, calculate the probability value of the do-not-disturb state corresponding to the feature data of each dimension, and perform weighted summation on the probability values corresponding to the feature data of multiple dimensions to obtain the probability value that the target user is in the do-not-disturb state.

[0050] Extract the feature data of multiple dimensions of the target image. Corresponding feature extraction algorithms can be used for feature extraction to form the feature data of each dimension. For example, for the feature data of the expression dimension, an expression feature extraction algorithm can be used for feature extraction. The expression feature extraction algorithm can be a principal component analysis (PCA) feature extraction algorithm, an independent component correlation algorithm (ICA) feature extraction algorithm, etc. For the feature data of the behavior dimension, a behavior recognition algorithm can be used to extract the features of the behavior dimension.

[0051] Compare the calculated probability value with a second preset value. If the probability value is greater than or equal to the second preset value, determine that the target user is in the do-not-disturb state. If the probability value is less than the second preset value, determine that the target user is in the non-do-not-disturb state. For example, set the second preset value to 0.95. If the calculated probability value is 0.9, then the target user is in the non-do-not-disturb state.

[0052] In this embodiment, by extracting the feature data of different dimensions, the different dimensions can be the expression data or behavior data of the target image, etc. Calculate the probability values of the feature data of different dimensions, and identify the state of the target user by obtaining the feature data of different dimensions, which improves the accuracy of identifying the user state of the target user and further improves the riding experience of the target user.

[0053] Step S205, if the target user is in the do-not-disturb state, adjust the volume of the playback device at the position corresponding to the target user on the vehicle.

[0054] It can be understood that during the driving of the vehicle, when the target user gets on the vehicle, they are in the non-do-not-disturb state. During the long driving process, based on the recognition results of the feature data of different dimensions, if it is determined that the target user is in the do-not-disturb state, adjust the volume of the playback device at the position where the target user is located.

[0055] In one implementation, the step of "if the target user is in the do-not-disturb state, adjust the volume of the playback device at the position corresponding to the target user on the vehicle" includes: within a first preset time, if the number of times the target user is continuously recognized as being in the do-not-disturb state reaches a first preset number and the target user is located in a non-driver position, adjust the volume of the playback device at the position corresponding to the target user on the vehicle.

[0056] In one embodiment, adjusting the volume of the playback device at the corresponding position of the target user on the vehicle includes: reducing or turning off the volume of the playback device at the corresponding position of the target user.

[0057] Figure 3 It is a schematic diagram of vehicle position identification provided by an embodiment of the present application. As Figure 3 shown, seat identification is performed on the seats of the vehicle, and identification codes are set on each seat of the vehicle. The position where the driver is located is identified as identification code 31, the co-pilot position is identified as identification code 32, and the rear seats of the vehicle are identified as identification code 33.

[0058] For target users in non-driver positions, such as the co-pilot position and the rear seats, in order to improve the accuracy of identification and avoid short-term user behaviors of the target user, a first preset time is set. If within the first preset time, the number of times the target user is continuously identified as in the do-not-disturb state reaches a first preset number, and the target user is a passenger, the volume of the playback device at the corresponding position of the passenger is adjusted.

[0059] When a target user in the vehicle is recognized, based on the pressure generated by the target user on the seat, the position where the target user is located is determined, and the identification code corresponding to the position of the target user is obtained. Combining the user state of the recognized target user and the identification code of the position where the target user is located, the volume of the playback device at the position corresponding to the identification code is adjusted.

[0060] Exemplarily, within 5 minutes, the number of times the target user is recognized as in the do-not-disturb state reaches 50 times, the volume of the playback device at the corresponding position of the target user is adjusted, and the volume of the playback device at the corresponding position of the target user is reduced or turned off to improve the riding experience of the target user.

[0061] In one embodiment, after reducing or turning off the volume of the playback device at the corresponding position of the target user, the method further includes: within a second preset time, if the number of times the target user is continuously recognized as not in the do-not-disturb state reaches a second preset number, increasing or turning on the volume of the playback device at the corresponding position of the target user.

[0062] During the driving of the vehicle, it has been determined that the target user in the vehicle is in the do-not-disturb state. For example, the target user is a passenger and the passenger is sleeping. After recognizing that the target user is in the do-not-disturb state, when the number of times the target user is continuously recognized as not in the do-not-disturb state reaches the second preset number, it is determined that the target user is not in the do-not-disturb state. For example, the passenger wakes up from sleep.

[0063] After it has been determined that the target user is in the Do Not Disturb state and then it is recognized that the target user is in a non-Do Not Disturb state, control the playback device at the location of the target user to increase the volume of the playback device at that location or turn on the volume of the playback device at that location.

[0064] If the target user is a passenger, the location where the passenger is located may be the co-pilot or the rear row. When it is recognized that the passenger is in the Do Not Disturb state and is located in the rear row seat, the volume output at the rear row location can be reduced or turned off, while maintaining the volume output of the playback devices at the driver's and co-pilot's playback device locations. When it is recognized that the passenger is in the Do Not Disturb state and is located in the co-pilot, the volume output at the co-pilot location can be reduced or turned off, while maintaining the volume output of the playback device at the driver's location, improving the passenger's riding experience while not affecting the volume output at the driver's location and ensuring that voice broadcasts such as those of Gaode Map can operate normally.

[0065] Figure 4 It is a flowchart of the volume adjustment of the playback device provided by an embodiment of the present application. As Figure 4 shown, in actual applications, the volume adjustment of the playback device may include the following steps:

[0066] Step S401, recognize that the vehicle door has been opened and then closed.

[0067] The vehicle's built-in sensing device can be used to recognize whether the vehicle door is opened and closed.

[0068] Step S402, detect the number of people in the vehicle.

[0069] The vehicle's built-in camera can be used to recognize the number of people in the vehicle.

[0070] Step S403, determine whether there are passengers in the vehicle according to the number of people in the vehicle.

[0071] The driver's position is the number 1. If it is recognized that the number is 2 or greater than 2, then it is determined that there are passengers.

[0072] Or, if it is recognized that there are people in other positions except the driver's position, then it is determined that there are passengers.

[0073] Step S404, when it is recognized that there are passengers in the vehicle, use the camera in the vehicle for monitoring and obtain the target image of the passengers. If no passengers are recognized, the operation of monitoring the passenger status is not performed.

[0074] Step S405, determine whether the passengers are asleep.

[0075] The status data of passengers can be monitored by a passenger monitoring system (OMS, Occupancy Monitoring System) and a driver monitoring system (Driver Monitoring System, DMS) installed on the vehicle, and the status data of passengers can be identified using a user status recognition model. For example, after the user status recognition model identifies the target image, it determines that the user status of the target user is the Do Not Disturb status, that is, it determines that the passenger is in a sleeping state, and uses the OMS and DMS to obtain the sleeping state of the passenger.

[0076] Step S406, if the passenger is in a sleeping state, upload the current passenger position where the passenger is located and the sleeping state to the vehicle-mounted system.

[0077] Step S407, determine whether the passenger position is in the rear passenger position.

[0078] Step S408, if the passenger position is in the rear passenger position, turn off or reduce the volume of the playback device at the rear passenger position, and keep the volume of the playback device at the front passenger position or turn on the playback device at the front passenger position.

[0079] Step S409, if the passenger position is in the front passenger seat position, turn off or reduce the volume of the playback device at the front passenger seat position, and keep the volume of the playback device at the driver's position or turn on the playback device at the driver's position.

[0080] Step S410, continuously monitor the vehicle.

[0081] Step S411, monitor whether the passengers in the vehicle wake up.

[0082] Step S412, when it is monitored that the passenger wakes up, adjust the playback device back to the default value, and when it is monitored that the passenger has been in a sleeping state, return to execute Step S410.

[0083] In another embodiment of the present application, when it is not monitored that the passenger is in a non-Do Not Disturb state, the volume of the playback device at the central position of the vehicle can be adjusted so that everyone in the vehicle can hear the music. When it is monitored that the vehicle is turned off, the monitoring of the passengers in the vehicle can be stopped to reduce the monitoring cost.

[0084] Figure 5 It is a flowchart of the volume adjustment of the playback device provided by another embodiment of the present application. As Figure 5 shown, the volume adjustment of the playback device may include the following steps:

[0085] Step S501, recognize that the vehicle door is opened and then closed.

[0086] Step S502, detect the number of people in the vehicle.

[0087] Step S503: Determine whether there are passengers in the vehicle according to the number of people in the vehicle.

[0088] Step S504: When it is recognized that there are passengers in the vehicle, use the camera in the vehicle to monitor and obtain the target images of the passengers. If no passengers are recognized, do not perform the monitoring of the passengers' states.

[0089] Step S505: Determine whether the passengers are asleep.

[0090] Step S506: If the passengers are not in the sleeping state, adjust the volume of the playback device at the central position of the vehicle so that everyone in the vehicle can hear the music.

[0091] Step S507: If the passengers are in the sleeping state, upload the current passenger positions of the passengers and the sleeping state to the vehicle-mounted system.

[0092] Step S508: Determine whether the passenger position is in the rear passenger position.

[0093] Step S509: If the passenger position is in the rear passenger position, turn off or reduce the volume of the playback device at the rear passenger position, and keep the volume of the playback device at the front passenger position or turn on the playback device at the front passenger position.

[0094] Step S510: If the passenger position is in the front passenger seat position, turn off or reduce the volume of the playback device at the front passenger seat position, and keep the volume of the playback device at the driver's position or turn on the playback device at the driver's position.

[0095] Step S511: Monitor whether the vehicle is turned off.

[0096] When it is monitored that the vehicle is not turned off, return to execute Step S504. If it is monitored that the vehicle is turned off, continue to execute Step S512.

[0097] Step S512: When it is monitored that the vehicle is turned off, stop monitoring the states of the passengers. In an embodiment, the training data further includes user images labeled as in a fatigued state or a distracted state. The method further includes:

[0098] If the target user is in a fatigued state or a distracted state and the target user is located at the driver's position, increase the volume of the playback device at the corresponding position of the target user.

[0099] In order to improve the safety of the driver during driving, during the training of the user state recognition model, the training data further includes user images labeled as in a fatigued state or a distracted state. Adding such training data is to identify the state of the driver to improve driving safety.

[0100] When it is recognized that the state of the driver is a fatigued state or a distracted state, increase the volume of the playback device at the position where the driver is located to remind the driver and avoid fatigue driving and distracted driving.

[0101] In the embodiment of the present application, by training a user state recognition model, after obtaining the trained user state recognition model, the state of the target user is recognized. The target user includes passengers and drivers. By extracting feature data of multiple dimensions from the target image of the target user, calculating the probability values of all dimension features, and obtaining the probability value of the Do Not Disturb state, the accuracy of recognizing the user state is improved. The embodiment of the present application not only includes the recognition of passengers, but also includes the recognition of drivers, which not only ensures the safe driving of the driver, but also ensures the riding experience of the passengers.

[0102] In a specific embodiment, a passenger monitoring system and a driver monitoring system installed on the vehicle can be used to monitor the states of passengers and drivers. During the actual driving process of the vehicle, the user state recognition model is used to obtain the recognized passenger data and driver data. The recognized passenger data and driver data can be collected in real time through the passenger monitoring system and the driver monitoring system. The driving duration of the vehicle, the current driving time of the vehicle, the light condition, the driving speed of the vehicle, the weather, the recognition result (success / failure) obtained by the user state recognition model, and the current state of the driver (dangerous driving / safe driving) can also be collected in real time through the passenger monitoring system and the driver monitoring system.

[0103] When the collected data reaches a certain amount, the above-mentioned collected relevant data and recognition results are transmitted back to the cloud background for analysis. According to the analysis results, the data that causes inaccurate recognition is obtained, and this type of data is collected again to retrain the user state recognition model to improve the accuracy of the user recognition model.

[0104] Figure 6 It is a flowchart for retraining the user state recognition model provided by the embodiment of the present application. As Figure 6 shown, the retraining of the user state recognition model includes the following steps:

[0105] Step S601, obtain the target image of the target user on the vehicle.

[0106] Step S602, recognize the target user in the target image, and determine whether the target user is in the Do Not Disturb state. If the target user is in the Do Not Disturb state, execute step S604.

[0107] The Do Not Disturb state can be that the passenger is on the phone, eyes closed and sleeping, wearing headphones, etc.

[0108] Step S603, if the target user is not in the Do Not Disturb state, determine whether a human face can be recognized from the target image. If no human face can be recognized, return to execute Step S601. If a human face can be recognized, execute Step S604.

[0109] Step S604, collect data of the target user and relevant data of the vehicle.

[0110] The data of the target user may include the status data of the target user (data corresponding to the non-Do Not Disturb state and the Do Not Disturb state). The relevant data of the vehicle can be collected in real time through the passenger monitoring system and the driver monitoring system, including the driving duration of the vehicle, the current driving time of the vehicle, the light condition, the driving speed of the vehicle, and the weather.

[0111] Step S605, when the data of the target user and the relevant data of the vehicle collected reach a preset threshold, transmit the data of the target user and the relevant data of the vehicle to the cloud.

[0112] The preset threshold can be a pre-set data volume. When the data of the target user and the relevant data of the vehicle collected reach a certain data volume, the data of the target user and the relevant data of the vehicle are transmitted to the cloud.

[0113] Step S606, analyze the data transmitted to the cloud, obtain the data with inaccurate recognition, and mark the data with inaccurate recognition.

[0114] Step S607, add new training data.

[0115] The training data is new data corresponding to the data with inaccurate recognition. For example, if the data with inaccurate recognition is caused by the light condition, then add data under different light conditions as the new training data.

[0116] Step S608, retrain the user status recognition model using the new training data.

[0117] Step S609, after successfully testing the retrained user status recognition model, update the user status recognition model.

[0118] The current user status recognition model can be corrected by using the Over-the-Air (OTA) technology.

[0119] Please continue to refer to Figure 1 , in this embodiment, the memory 11 may be the internal memory of the electronic device 1, that is, the memory built into the electronic device 1. In other embodiments, the memory 11 may also be the external memory of the electronic device 1, that is, the memory externally connected to the electronic device 1.

[0120] In some embodiments, the memory 11 is used to store program codes and various data, and to achieve high-speed and automatic access to programs or data during the operation of the electronic device 1.

[0121] The memory 11 may include a random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one disk storage device, a flash memory device, or other volatile solid state storage devices.

[0122] In one embodiment, the processor 12 may be a Central Processing Unit (CPU), and may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any other conventional processor, etc.

[0123] If the program codes and various data in the memory 11 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, all or part of the processes in the above-described embodiment methods of the present application, such as the method for adjusting the volume of an automobile, may also be completed by a computer program instructing relevant hardware. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments may be implemented. Among them, the computer program includes computer program codes, and the computer program codes may be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program codes, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a Read-Only Memory (ROM), etc.

[0124] It can be understood that the above-described module division is a logical function division, and there may be other division methods in actual implementation. In addition, in each embodiment of the present application, each functional module can be integrated in the same processing unit, or each module can exist physically alone, or two or more modules can be integrated in the same unit. The above-integrated modules can be implemented in the form of hardware or in the form of a combination of hardware and software functional modules.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. An automobile volume adjustment method, characterized in that, The method includes: Obtaining training data, where the training data includes multiple user images labeled with user states, and the user states include a do-not-disturb state and a non-do-not-disturb state; Using the training data to train a neural network model to obtain a user state recognition model; Obtaining a target image of a target user on the vehicle; Using the user state recognition model to recognize the target image to determine the user state of the target user; If the target user is in the do-not-disturb state, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle.

2. The car volume adjustment method according to claim 1, characterized in that The method further includes: Obtaining user images without labeled user states as test data; Inputting the test data into the user state recognition model, and calculating the accuracy rate of recognizing the user state according to the test results output by the user state recognition model; Calculating the misjudgment rate of the user state recognition model based on the accuracy rate of recognizing the user state; If the misjudgment rate is greater than a first preset value, adding new training data and using the new training data to retrain the user state recognition model.

3. The method for adjusting the volume of an automobile according to claim 1, wherein The using the user state recognition model to recognize the target image to determine the user state of the target user includes: Extracting feature data of multiple dimensions of the target image; Inputting the feature data of multiple dimensions into the user state recognition model, and according to the output result of the user state recognition model, determining the probability value that the target user is in the do-not-disturb state; If the probability value is greater than or equal to a second preset value, determining that the target user is in the do-not-disturb state; If the probability value is less than the second preset value, determining that the target user is in the non-do-not-disturb state.

4. The automotive volume adjustment method according to claim 3, wherein The inputting the feature data of multiple dimensions into the user state recognition model, and according to the output result of the user state recognition model, determining the probability value that the target user is in the do-not-disturb state includes: Performing weighted summation on the probability values corresponding to the feature data of multiple dimensions to obtain the probability value that the target user is in the do-not-disturb state.

5. The method for adjusting the volume of an automobile according to claim 1, wherein The if the target user is in the do-not-disturb state, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle includes: Within a first preset time, if the number of times the target user is continuously recognized as being in the do-not-disturb state reaches a first preset number of times and the target user is in a non-driver position, adjusting the volume of the playback device at the position corresponding to the target user on the vehicle.

6. The method for adjusting the volume of an automobile according to claim 5, wherein Adjusting the volume of the playback device at the position corresponding to the target user on the vehicle includes: Reducing or turning off the volume of the playback device at the position corresponding to the target user.

7. The method for adjusting the volume of an automobile according to claim 6, wherein After reducing or turning off the volume of the playback device at the position corresponding to the target user, the method further includes: Within a second preset time, if the number of times the target user is continuously recognized as being in the non-do-not-disturb state reaches a second preset number of times, increasing or turning on the volume of the playback device at the position corresponding to the target user.

8. The method for adjusting the volume of an automobile according to claim 1, characterized in that, The training data further includes user images labeled as being in a fatigued state or a distracted state, and the method further includes: If the target user is in a fatigued state or a distracted state and the target user is located in the driver's position, increase the volume of the playback device at the corresponding position of the target user.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, and the processor is configured to execute a computer program stored in the memory to implement the vehicle volume adjustment method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the vehicle volume adjustment method according to any one of claims 1 to 8 is implemented.

Images