Vehicle-mounted intelligent audio-video adaptive playing method, device, equipment and medium

Abstract

The invention relates to a vehicle-mounted intelligent audio-video adaptive playing method, device, equipment and medium, and the method comprises the steps: carrying out the multi-level playing demand feature extraction of obtained vehicle operation state, cabin environment state and user behavior state data, generating a user dynamic playing demand feature, a vehicle operation adaptive feature and a cabin environment adaptive feature; and generating a network based on a preset playing strategy, carrying out matching processing on the characteristics to obtain an adaptive playing strategy, adjusting a playing parameter set of the vehicle-mounted audio and video playing equipment according to the adaptive playing strategy, and executing playing control, thereby comprehensively sensing vehicle-mounted environment changes, intelligently analyzing user requirements, automatically adjusting the playing strategy, and improving the playing efficiency. The audio-video playing effect is optimized, the user experience is improved, and the driving safety is ensured.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle engineering technology, and in particular to a method, device, equipment, and medium for in-vehicle intelligent audio and video adaptation playback. Background Art

[0002] In-vehicle audio and video systems have become an essential component for enhancing the driving experience and passenger comfort. However, traditional in-vehicle audio and video playback methods often lack adaptability to changes in the vehicle's operating environment and are unable to intelligently adjust based on the vehicle's operating status, the cabin environment, and user behavior. For example, if the audio and video system continues to play at a high volume during high-speed driving or emergency braking, it may distract the driver and increase driving risks. Similarly, if the audio and video system fails to automatically adjust its playback strategy in a noisy cabin or when user behavior changes (such as answering a phone call), the user's viewing or listening experience will also be affected.

[0003] At present, although some in-vehicle audio and video systems have begun to try to introduce environmental perception technology, most of these systems only make simple adjustments to a single environmental factor, lack the ability to comprehensively analyze and process multiple environmental factors, and are unable to accurately adapt to users' dynamic needs. Summary of the Invention

[0004] The purpose of the present invention is to provide a method, device, equipment and medium for in-vehicle intelligent audio and video adaptation playback, aiming to accurately adapt according to the dynamic needs of users and enhance the user's experience of using in-vehicle intelligent audio and video.

[0005] To achieve the above objectives, a first aspect of the present disclosure provides a method for in-vehicle intelligent audio and video adaptive playback, the method comprising:

[0006] Perform multi-level user playback demand feature extraction operations on the acquired user behavior status data to generate user dynamic playback demand features;

[0007] Perform multi-level environment playback requirement feature extraction operations on the obtained cockpit environment status data to generate cockpit environment adaptation features;

[0008] Perform multi-level vehicle playback demand feature extraction operations on the acquired vehicle operation status data to generate vehicle operation adaptation features;

[0009] Based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics, a plurality of playback constraint characteristics are obtained by performing a pairwise cross-feature cross-attention operation. Based on the primary constraint reward of a discrete action and the secondary constraint reward of a continuous action, the multiple playback constraint feature constraints are constrained to generate a playback constraint condition set including a primary playback constraint condition and a plurality of secondary playback constraint conditions.

[0010] According to the main playback constraint and the secondary playback constraint in the playback constraint set, an adaptive playback strategy for the current playback scene is generated, and the playback parameter set of the in-vehicle audio and video playback device is adjusted according to the adaptive playback strategy, and the playback control operation is performed based on the adjusted playback parameter set.

[0011] In a possible implementation, adjusting the playback parameter set of the in-vehicle audio and video playback device according to the adapted playback strategy, and performing the playback control operation based on the adjusted playback parameter set, includes:

[0012] parsing the playback content priority rule in the adapted playback strategy, and filtering a target playback content list from a media resource library according to the playback content priority rule;

[0013] Analyze the playback volume adjustment rules in the adaptive playback strategy and dynamically adjust the initial volume parameters of the target playback content according to the safety factor of the vehicle driving scenario;

[0014] Analyze the screen brightness adjustment rules in the adaptive playback strategy and adjust the display brightness parameters according to the cabin lighting scene characteristics and the user's physiological state characteristics;

[0015] Analyze the noise reduction intensity control rules in the adaptive playback strategy and adjust the active noise reduction intensity parameters according to the noise interference characteristics and the user's physiological fatigue level;

[0016] A playback control instruction is generated based on the target playback content list, the initial volume parameter, the display brightness parameter, and the active noise reduction intensity parameter, and the playback control instruction is sent to the in-vehicle audio and video playback device to perform a real-time playback operation.

[0017] In a possible implementation, parsing the playback content priority rule in the adapted playback strategy and filtering the target playback content list from the media resource library according to the playback content priority rule includes:

[0018] Obtain the media type preference weight and content theme preference label from the user's historical playback preference features;

[0019] Obtain real-time semantic keywords and content screening conditions from the user's current playback intention features;

[0020] extracting a first candidate content set from a media resource library according to the media type preference weight and the content theme preference tag;

[0021] performing content matching calculation on the first candidate content set according to the real-time semantic keywords and the content screening condition to obtain a second candidate content set;

[0022] The second candidate content set is sorted based on the playback content priority rule to generate a target playback content list.

[0023] In one possible implementation, parsing the playback volume adjustment rule in the adaptive playback strategy and dynamically adjusting the initial volume parameters of the target playback content according to the safety factor of the vehicle driving scenario includes:

[0024] Obtain the safety level threshold range corresponding to the vehicle driving scenario safety factor;

[0025] When it is detected that the safety factor of the vehicle driving scene is lower than the first safety threshold, the volume is automatically increased, and the volume increase amplitude is calculated according to the safety factor difference;

[0026] When it is detected that the safety factor of the vehicle driving scene is higher than the second safety threshold, the volume hold operation is triggered and the current volume parameter is maintained unchanged;

[0027] When it is detected that the safety factor of the vehicle driving scene is between the first safety threshold and the second safety threshold, a volume gradual adjustment operation is triggered, and the volume adjustment step size is dynamically adjusted according to the safety factor change rate;

[0028] An adjusted initial volume parameter is generated based on the volume boost amplitude, the volume adjustment step, and the current volume parameter.

[0029] In a possible implementation, performing a multi-level user playback demand feature extraction operation on the acquired user behavior status data to generate user dynamic playback demand features includes:

[0030] Performing time series behavior pattern analysis on the user operation record data in the user behavior status data to obtain the user's historical playback preference characteristics;

[0031] Performing real-time semantic parsing on the user interaction response data in the user behavior status data to extract features of the user's current playback intention;

[0032] Performing physiological state fluctuation recognition processing on the user physiological monitoring data in the user behavior state data to generate a user physiological state feature;

[0033] The user's historical playback preference features, the user's current playback intention features, and the user's physiological state features are subjected to feature fusion processing to generate the user's dynamic playback demand features.

[0034] In a possible implementation, performing a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate a vehicle operation adaptation feature further includes:

[0035] Performing driving scene classification processing on the vehicle speed change data in the vehicle running state data to obtain vehicle driving scene type characteristics;

[0036] performing vibration frequency analysis on vehicle vibration data in the vehicle operating status data to extract cabin vibration interference characteristics;

[0037] performing path semantic parsing on the navigation path data in the vehicle operation status data to generate navigation path association features;

[0038] The vehicle driving scene type feature, the cabin vibration interference feature and the navigation path association feature are dynamically associated with each other to generate the vehicle operation adaptation feature.

[0039] In a possible implementation, performing a multi-level environment playback requirement feature extraction operation on the acquired cabin environment status data to generate a cabin environment adaptation feature further includes:

[0040] Performing lighting scene classification processing on the light intensity data in the state monitoring data set to obtain cabin lighting scene features;

[0041] Performing noise spectrum analysis on the environmental noise data in the state monitoring data set to extract noise interference characteristics;

[0042] performing gas component identification processing on the air quality data in the state monitoring data set to generate cabin air state characteristics;

[0043] The cabin lighting scene features, the noise interference features, and the cabin air state features are subjected to multi-dimensional fusion processing to generate the cabin environment adaptation features.

[0044] According to a second aspect of the present disclosure, there is provided an in-vehicle intelligent audio and video adaptation and playback device, the device comprising:

[0045] A first generating module is configured to perform a multi-level user playback demand feature extraction operation on the acquired user behavior state data to generate a user dynamic playback demand feature;

[0046] A second generating module is configured to perform a multi-level environment playback requirement feature extraction operation on the acquired cockpit environment state data to generate a cockpit environment adaptation feature;

[0047] A third generating module is configured to perform a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate a vehicle operation adaptation feature;

[0048] a determination and generation module configured to obtain a plurality of playback constraint features by performing a pairwise cross-feature cross-attention operation based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics, and generate a playback constraint condition set including a primary playback constraint condition and a plurality of secondary playback constraint conditions for the plurality of playback constraint feature constraint rewards based on a primary constraint reward for discrete actions and a secondary constraint reward for continuous actions;

[0049] The playback control module is configured to generate an adaptive playback strategy for the current playback scene based on the main playback constraint and the secondary playback constraint in the playback constraint set, adjust the playback parameter set of the in-vehicle audio and video playback device according to the adaptive playback strategy, and perform playback control operations based on the adjusted playback parameter set.

[0050] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0051] a memory having a computer program stored thereon;

[0052] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods in the first aspect.

[0053] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0054] The present invention provides a method, device, equipment, and medium for in-vehicle intelligent video and audio adaptation playback. Compared with the prior art, it has the following advantages:

[0055] By obtaining a set of status monitoring data of the in-vehicle environment, including vehicle operation status data, cabin environment status data and user behavior status data, and performing multi-level playback demand feature extraction operations, it is possible to comprehensively and accurately capture the user's dynamic playback needs in different scenarios, as well as the adaptation characteristics of the vehicle and cabin environment. Based on the preset playback strategy generation network, this method can intelligently match these features to obtain the adaptive playback strategy for the current playback scenario, ensuring that the audio and video playback meets both user needs and adapts to the vehicle and cabin environment. Furthermore, by adjusting the playback parameter set of the in-vehicle audio and video playback device according to the adaptive playback strategy and executing playback control operations, this method can optimize the audio and video playback effect in real time and dynamically, enhance the user's viewing or listening experience, while ensuring driving safety, and significantly improving the intelligence level and user experience of the in-vehicle audio and video system.

[0056] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0058] Figure 1 This is a flow chart of a method for in-vehicle intelligent audio and video adaptation playback according to an embodiment of the specification.

[0059] Figure 2 This is a block diagram of an in-vehicle intelligent audio and video adaptation and playback device according to an embodiment of the specification.

[0060] Figure 3 This is a block diagram of a vehicle-mounted intelligent audio and video adaptation and playback device according to an embodiment of the specification. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] The following describes the specific embodiments of the present disclosure in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure.

[0063] The present invention provides a method for in-vehicle intelligent audio and video adaptation playback. Figure 1 This is a flow chart illustrating a method for in-vehicle intelligent video and audio adaptation playback according to one embodiment. Specifically, the method includes:

[0064] In step S11, a multi-level user playback demand feature extraction operation is performed on the acquired user behavior status data to generate a user dynamic playback demand feature;

[0065] Among them, user behavior status data refers to various behavioral data generated by users when using the in-vehicle audio and video system, such as users' operations on the playback content (such as pause, fast forward, switch songs, etc.), operation frequency, stay time, collection preferences, etc. These data reflect the user's behavioral habits and demand tendencies when using the audio and video system.

[0066] The multi-level user playback demand feature extraction operation can be to analyze and process the user behavior status data through different levels (such as shallow feature extraction layer, deep feature extraction layer, etc.), and explore the user's playback demand characteristics at different levels, such as the user's preference for music type (shallow features) and playback habits in different scenarios (deep features).

[0067] The user's dynamic playback demand characteristics can be generated based on the user's behavior status data after multi-level feature extraction, and can reflect the user's current demand characteristics for in-car audio and video playback in real time. It will be dynamically updated as the user's behavior changes.

[0068] In the embodiment of the present disclosure, the acquired user behavior status data is preprocessed, including data cleaning (removing noise data, outliers, etc.), data normalization (unifying data of different ranges into a specific interval), and other operations for subsequent feature extraction. Then, a multi-layer neural network structure is used for feature extraction. The shallow network may adopt a convolutional neural network (CNN) or a recurrent neural network (RNN), etc., to extract local features and temporal features of user behavior data, such as the user's click frequency on different types of music, operating habits in different time periods, etc. The deep network may adopt a deep neural network (DNN) or an attention mechanism network to further fuse and abstract the shallow features, and dig out the user's deeper playback demand features, such as the user's playback preferences under different emotional states. Finally, the features extracted by the multi-layer network are integrated to generate the user's dynamic playback demand features.

[0069] For example, user behavior status data is collected through in-vehicle devices. The user's operations on the in-vehicle audio and video system are recorded by the operation recording device, with operation types set to c1_1 and c1_2. The voice and touch information of the user's interaction with the system is collected by the interactive response device, with voice commands set to c2_1 and c2_2, and touch commands set to c2_3 and c2_4. The physiological monitoring device on the seat collects the user's physiological data, with heart rate set to c3_h and blood pressure set to c3_b, forming the user behavior status data set [c1_1, c1_2, c2_1, c2_2, c2_3, c2_4, c3_h, c3_b, ...].

[0070] In step S12, a multi-level environment playback requirement feature extraction operation is performed on the acquired cabin environment status data to generate a cabin environment adaptation feature;

[0071] Among them, the cabin environment status data is used to describe the internal environmental conditions of the vehicle cabin, including but not limited to cabin temperature, humidity, light intensity, noise level, seat adjustment status, etc. These environmental factors will affect the user's experience of audio and video playback in the cabin.

[0072] The multi-level cabin environment playback demand feature extraction operation can be based on the cabin environment status data, using a multi-level analysis method to extract features related to audio and video playback needs, such as user preferences for music volume and sound quality at different temperatures, and screen brightness requirements under different light intensities.

[0073] The cabin environment adaptation features can be obtained through multi-level feature extraction. They are used to describe the adaptation relationship between the cabin environment and the audio and video playback requirements. They reflect the adaptation requirements of the cabin environment status for audio and video playback.

[0074] In the disclosed embodiment, the cabin environment status data is first preprocessed, such as standardizing data such as temperature and humidity. Then, a multi-level feature extraction method is used. The shallow feature extraction layer can adopt a rule-based method, for example, according to the correspondence between common cabin environments and audio and video playback requirements, some simple features can be extracted, such as users may prefer a lower volume at high temperatures. The deep feature extraction layer can use machine learning models, such as decision trees, support vector machines, etc., to conduct a more in-depth analysis of shallow features, considering the impact of the combination of multiple environmental factors on audio and video playback requirements, such as the user's comprehensive requirements for screen brightness and sound quality under different lighting and noise levels. Finally, the features extracted from each layer are fused to generate cabin environment adaptation features.

[0075] For example, the light sensor in the cabin collects light intensity data, set as b1; the noise sensor collects noise data at different frequencies, set as f1 and f2 respectively, and the corresponding noise data are b2_f1 and b2_f2; the air quality sensor collects air composition data, such as oxygen-related data set as b3_o and carbon dioxide-related data set as b3_c, forming the cabin environment status data set [b1, b2_f1, b2_f2, b3_o, b3_c, ...].

[0076] In step S13, a multi-level vehicle playback demand feature extraction operation is performed on the acquired vehicle operation status data to generate a vehicle operation adaptation feature;

[0077] Among them, vehicle operation status data refers to various data related to the vehicle's driving process, such as vehicle speed, acceleration, steering angle, driving mode (such as sports mode, economic mode, etc.), vehicle fault information, etc. The vehicle operation status will affect the user's experience and needs for audio and video playback during driving.

[0078] The multi-level vehicle operation and playback demand feature extraction operation can be a multi-level analysis and processing of vehicle operation status data to extract features related to audio and video playback needs, such as the user's preference for music rhythm and volume when driving at high speed, and the special demand for audio and video playback when the vehicle breaks down.

[0079] The vehicle operation adaptation feature is generated after multi-level feature extraction. It is used to describe the adaptation relationship between the vehicle operation status and the audio and video playback requirements. It reflects the adaptation requirements of the vehicle operation status for audio and video playback.

[0080] In the disclosed embodiment, the vehicle operation status data is preprocessed, such as smoothing the vehicle speed, acceleration and other data to remove the influence of instantaneous fluctuations. Using a multi-level feature extraction structure, the shallow network can extract some basic correlation features between the vehicle operation status and the audio and video playback needs. For example, when driving at high speed, the user may want a faster music rhythm. The deep network can use deep learning models, such as long short-term memory networks (LSTM), to consider the impact of the temporal changes in the vehicle operation status on the audio and video playback needs, such as the changes in the user's demand for volume and sound effects during the continuous acceleration of the vehicle. Finally, the features extracted at each layer are integrated to generate the vehicle operation adaptation features.

[0081] For example, when a vehicle is running, various sensors begin collecting data. Regarding the vehicle's operating status data, sensors installed near the engine collect power-related data, designated a1; sensors located on the chassis collect vibration-related data, designated a2. These data constitute the vehicle's operating status data set, which has the form [a1, a2, ...].

[0082] In step S14, a plurality of playback constraint features are obtained by performing a pairwise cross-feature cross-attention operation based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics. Based on the primary constraint reward of discrete actions and the secondary constraint reward of continuous actions, a playback constraint condition set including a primary playback constraint condition and a plurality of secondary playback constraints is generated for the plurality of playback constraint feature constraint rewards.

[0083] Among them, cross-feature cross-attention operation is used to perform cross-attention operation on the user's dynamic playback demand features, cabin environment adaptation features and vehicle operation adaptation features in pairs. By calculating the attention weights between features, the parts that influence each other between different features are highlighted, thereby obtaining multiple playback constraint features.

[0084] The primary constraint reward for discrete actions is used to clarify and differentiate behavioral options (such as play, pause, and change songs) within the reinforcement learning framework. The primary constraint reward is a reward value given to discrete actions that meet the primary playback constraints (such as playback actions explicitly requested by the user), guiding the system to make decisions that meet the user's primary needs.

[0085] Secondary constraint rewards for continuous actions are actions that continuously change within a certain range (such as continuous volume adjustment or brightness adjustment). Secondary constraint rewards are given to continuous actions that meet secondary playback constraints (such as cabin environment adaptation and vehicle operation adaptation). They are used to further optimize playback parameters and improve the playback experience.

[0086] The constraint reward is based on the main constraint reward of discrete actions and the secondary constraint reward of continuous actions. It evaluates the multiple playback constraint features obtained through cross-feature cross-attention operation to determine the influence of each playback constraint feature on the final playback strategy.

[0087] In the disclosed embodiments, user dynamic playback demand features, cabin environment adaptation features, and vehicle operation adaptation features are combined in pairs. For example, the user dynamic playback demand feature is combined with the cabin environment adaptation feature, the user dynamic playback demand feature is combined with the vehicle operation adaptation feature, and the cabin environment adaptation feature is combined with the vehicle operation adaptation feature. For each pair of combined features, a cross-attention mechanism is used for calculation.

[0088] Specifically, for the feature vector X corresponding to the user's dynamic playback demand feature and the feature vector Y corresponding to the cabin environment adaptation feature, we first calculate the attention weight of each element in X with respect to each element in Y. Then, we perform a weighted sum of the elements in Y based on the attention weights to obtain the attention representation of X with respect to Y. Similarly, we calculate the attention representation of Y with respect to X. These two attention representations are fused to obtain the cross-attention feature of this combined feature pair, which is the first playback constraint feature.

[0089] Similarly, for the feature vector X corresponding to the user's dynamic playback demand feature and the feature vector Z corresponding to the vehicle operation adaptation feature, we first calculate the attention weight of each element in X with respect to each element in Z. Then, we perform a weighted sum of the elements in Z based on the attention weights to obtain the attention representation of X with respect to Z. Similarly, we calculate the attention representation of Z with respect to X. These two attention representations are fused to obtain the cross-attention feature of this combined feature pair, which is the second playback constraint feature.

[0090] Similarly, for the feature vector Y corresponding to the cabin environment adaptation feature and the feature vector Z corresponding to the vehicle operation adaptation feature, we first calculate the attention weight of each element in Y with respect to each element in Z. Then, we perform a weighted sum of the elements in Z based on the attention weights to obtain the attention representation of Y with respect to Z. Similarly, we calculate the attention representation of Z with respect to Y. These two attention representations are fused to obtain the cross-attention feature of this combined feature pair, i.e., the third playback constraint feature. In this way, multiple playback constraint features can be obtained.

[0091] In the embodiment of the present disclosure, based on the framework of reinforcement learning, the main constraint rewards for discrete actions and the sub-constraint rewards for continuous actions are defined. For discrete actions, corresponding reward values ​​are given according to whether the main playback constraint conditions (such as playback actions explicitly requested by the user) are met. For example, a positive reward is given when the main constraint conditions are met, and a negative reward is given when they are not met. For continuous actions, corresponding reward values ​​are given according to whether the sub-constraint conditions for playback (such as cabin environment adaptation, vehicle operation adaptation, etc.) are met. The size of the reward value can be quantified according to the degree of compliance with the sub-constraint conditions. Then, based on these reward values, constraint rewards are given to multiple playback constraint features, that is, the weight of each playback constraint feature in subsequent decisions is adjusted according to the size of the reward value, and a playback constraint condition set including the main playback constraint conditions and multiple playback sub-constraint conditions is generated.

[0092] In step S15, an adaptive playback strategy for the current playback scene is generated based on the main playback constraint and the secondary playback constraint in the playback constraint set, and the playback parameter set of the in-vehicle audio and video playback device is adjusted according to the adaptive playback strategy, and the playback control operation is performed based on the adjusted playback parameter set.

[0093] Among them, the adaptive playback strategy is used to guide the decision-making plan of the vehicle-mounted audio and video playback device for playback in the current playback scenario. It stipulates the specific value range and adjustment method of the playback parameter set. The playback parameter set can include various adjustable parameters of the vehicle-mounted audio and video playback device, such as volume, brightness, contrast, playback mode (such as sequential playback, random playback, single loop, etc.), playback content, etc. The adjustment of these parameters will affect the user's audio and video playback experience. The playback control operation is to perform actual control operations on the vehicle-mounted audio and video playback device based on the adjusted playback parameter set, such as adjusting the volume, changing the screen brightness, switching playback content, etc., to achieve an audio and video playback effect that meets user needs and the current scenario.

[0094] In the embodiment of the present disclosure, an optimization algorithm (such as a genetic algorithm, a particle swarm optimization algorithm, etc.) or a rule-based decision-making method is used to generate an adaptive playback strategy for the current playback scenario based on the playback main constraint and the playback sub-constraint in the playback constraint set. For example, if the playback main constraint requires playing a song specified by the user, and the playback sub-constraint requires lowering the volume when driving at high speeds, then the generated adaptive playback strategy may be to dynamically adjust the volume according to the vehicle speed while playing the user-specified song. Then, the playback parameter set of the in-vehicle audio and video playback device is adjusted according to the adaptive playback strategy, such as setting the specific values ​​of parameters such as volume, brightness, and playback mode. Finally, based on the adjusted playback parameter set, playback control operations are performed through the control interface of the in-vehicle audio and video playback device, such as adjusting the parameter settings of the hardware device, to achieve an audio and video playback effect that meets user needs and the current scenario.

[0095] The above-mentioned technical solution obtains a set of status monitoring data of the vehicle environment, including vehicle operation status data, cabin environment status data and user behavior status data, and performs multi-level playback demand feature extraction operations. It can comprehensively and accurately capture the user's dynamic playback needs in different scenarios, as well as the adaptation characteristics of the vehicle and cabin environment. Based on the preset playback strategy generation network, this method can intelligently match these features to obtain the adaptive playback strategy for the current playback scenario, ensuring that the audio and video playback meets both user needs and adapts to the vehicle and cabin environment. Furthermore, by adjusting the playback parameter set of the in-vehicle audio and video playback device according to the adaptive playback strategy and executing playback control operations, this method can optimize the audio and video playback effect in real time and dynamically, enhance the user's viewing or listening experience, while ensuring driving safety, and significantly improving the intelligence level and user experience of the in-vehicle audio and video system.

[0096] In a possible implementation, in step S15, adjusting the playback parameter set of the vehicle-mounted audio and video playback device according to the adapted playback strategy, and performing the playback control operation based on the adjusted playback parameter set, includes:

[0097] In step S151, the playback content priority rule in the adapted playback strategy is parsed, and a target playback content list is filtered from a media resource library according to the playback content priority rule;

[0098] In the disclosed embodiment, the playback content priority rules in the adaptive playback strategy are stored in a specific data structure (such as a tree structure, a linked list structure, or a database table structure). First, the rule data is read, and the priority order of different types of playback content is parsed by traversal or query. For example, the rule may stipulate that in a navigation scenario, the navigation prompt tone has the highest priority, followed by emergency news broadcasts, and then the user's favorite music, etc. During the parsing process, a priority value can be assigned to each playback content type, and the smaller the value, the higher the priority.

[0099] Furthermore, based on the playback content priority rules obtained through analysis, you can filter from the media resource library. The media resource library is a database containing various types of media files (such as audio, video) and their related metadata (such as file name, type, duration, user rating, etc.). You can query the media resource library in order of priority, and add the playback content that meets the current scenario and user needs to the target playback content list. For example, if you are currently in a navigation scenario, you can first filter out the navigation prompt audio file, and then filter out emergency news broadcasts and user favorites based on the user's historical preferences and the current time (such as whether it is a news period), and finally generate a target playback content list arranged by priority.

[0100] For example, first analyze the content priority rules within the adaptive playback strategy. Assume that these rules specify a priority order for different music genres and themes. From the media resource library, based on the media type preference weights (e.g., w1*k1) in the user's historical playback preference features and the content theme preference tag (t1), a first set of candidate content that meets these preferences is extracted, such as a collection of songs, audio programs, and other resources. Then, based on the real-time semantic keywords (k1) in the user's current playback intent features and the content filtering criteria (c1), a content matching degree is calculated for this first set of candidate content. For example, the similarity between the title and description of each candidate content and the keyword k1 is compared, as well as whether the content filtering criteria c1 is met. Matching can be determined by setting a specific matching algorithm, such as calculating the frequency and position of the keyword k1 in the description text of each candidate content. Assume that the matching degree result is denoted by m, a multi-dimensional numerical set derived from a specific matching algorithm, with each dimension representing a different aspect of the matching degree, such as semantic matching or thematic matching. After calculation, a second candidate content set is obtained. The content in this set is further screened based on the first candidate content set according to the degree of matching. Finally, the second candidate content set is sorted based on the playback content priority rule. For example, according to the priority order of different music types, themes, etc. specified in the rules, the content in the set is arranged to generate a target playback content list. Assume that the target playback content list is represented by L, which is a list containing multiple media resource identifiers, each identifier corresponding to a specific playback content in the media resource library, and these identifiers are arranged in order of priority.

[0101] In step S152, the playback volume adjustment rules in the adapted playback strategy are parsed, and the initial volume parameters of the target playback content are dynamically adjusted according to the safety factor of the vehicle driving scenario;

[0102] In the disclosed embodiment, the playback volume adjustment rules may be formulated based on factors such as the safety factor of the vehicle driving scenario and user preferences. The rules can be read to parse the volume adjustment range and method corresponding to the safety factors of different driving scenarios. For example, the rules may stipulate that in urban congestion scenarios (lower safety factor), the initial volume should be set at a lower level to avoid interfering with the driver's perception of the surrounding environment; while in highway scenarios (higher safety factor), the initial volume can be appropriately increased. At the same time, the rules may also take into account the user's historical volume preferences and make adjustments within the allowed range.

[0103] Furthermore, the initial volume parameters of the target playback content are dynamically adjusted based on the driving scene safety factor obtained in real time by the vehicle and the volume adjustment rules obtained through analysis. The driving scene safety factor can be calculated using data provided by vehicle sensors (such as speed sensors and acceleration sensors) and the navigation system. For example, when it is detected that the vehicle is in a city congestion scene, the initial volume can be lowered to a preset low level according to the rules; when the vehicle enters a highway scene, the volume can be gradually increased to an appropriate level. During the adjustment process, the user's manual volume adjustment history can be taken into account to avoid sudden volume changes causing discomfort to the user.

[0104] For example, the playback volume adjustment rule is extracted from the adaptive playback strategy. The rule is related to the vehicle driving scene safety factor. Assume that the vehicle driving scene safety factor is represented by s, which is a multi-dimensional numerical set that includes factors that measure the safety level of the driving scene in different aspects. For example, s may include numerical values ​​corresponding to factors such as vehicle speed stability and road congestion. The initial volume parameter is set to v0, which is also a multi-dimensional parameter set. For example, it may include volume settings for different audio channels. Get the safety level threshold range corresponding to the vehicle driving scene safety factor s and set it to [s_min, s_max]. This is also a multi-dimensional range set. Each dimension corresponds to the safety level threshold of a different factor in s.

[0105] When it is detected that a dimension value in the vehicle driving scene safety factor s is lower than the first safety threshold s_low (s_low is a set of values ​​of the same dimension as s, and each dimension corresponds to the first safety threshold of different factors in s), the automatic volume increase operation is triggered. At this time, the volume increase amplitude is calculated based on the safety factor difference. Assuming that the safety factor difference is △s, △s is a set of differences between the dimension values ​​in s lower than s_low and the dimension values ​​corresponding to s_low. Through a specific calculation method, for example, according to a preset proportional relationship, the proportional coefficient is set to k1 (k1 is a multi-dimensional coefficient set, corresponding to the dimension of △s, and is used to calculate the volume increase amplitude under different factors), the volume increase amplitude is a = △s × k1, a is a multi-dimensional volume increase amplitude set, and each dimension corresponds to the increase amplitude of different audio channels or volume adjustment aspects.

[0106] When it is detected that a dimension value in the vehicle driving scene safety factor s is higher than the second safety threshold s_high (s_high is a set of numerical values ​​with the same dimension as s, and each dimension corresponds to the second safety threshold of different factors in s), the volume hold operation is triggered and the current volume parameter v0 is maintained unchanged.

[0107] When it is detected that a dimension value in the vehicle driving scene safety factor s is between the first safety threshold s_low and the second safety threshold s_high, the volume gradual adjustment operation is triggered. First, the safety factor change rate is calculated. Assuming that within a time interval △t, the safety factor changes from s1 to s2 (s1 and s2 are both numerical sets with the same dimension as s), the safety factor change rate is set to r = (s2-s1) / △t, r is a multi-dimensional change rate set, and each dimension corresponds to the change rate of different factors in s. Then, the volume adjustment step size is dynamically adjusted according to the safety factor change rate r. Assume that the relationship between the volume adjustment step size and the safety factor change rate is represented by a function f, that is, the step size b = f(r), b is a multi-dimensional volume adjustment step size set, and each dimension corresponds to the step size of different audio channels or volume adjustment aspects.

[0108] Finally, the adjusted initial volume parameter v is generated based on the volume boost a, the volume adjustment step b, and the current volume parameter v0. If the volume is automatically increased, v = v0 + a (the "+" here represents an addition operation in each corresponding dimension to ensure dimensional consistency). If the volume is gradually adjusted, v0 is gradually adjusted according to the volume adjustment step b. For example, after n adjustments, v = v0 + n × b (again, the calculations here are performed on each dimension to ensure dimensional consistency).

[0109] In step S153, the screen brightness adjustment rule in the adapted playback strategy is parsed, and the display brightness parameters are adjusted according to the cabin lighting scene characteristics and the user's physiological state characteristics;

[0110] In the disclosed embodiments, screen brightness adjustment rules typically comprehensively consider the characteristics of the cabin lighting scenario and the user's physiological state. These rules can be read to parse the screen brightness adjustment range and method corresponding to different lighting scenarios and user physiological states. For example, the rules may specify that in bright daylight scenarios, screen brightness should be increased to a higher level to ensure the user can clearly see the screen content; while in low-light scenarios at night, screen brightness should be reduced to a lower level to avoid irritation to the user's eyes. Furthermore, the rules may also take into account the user's physiological state, such as whether the user is fatigued, and appropriately reduce the screen brightness to reduce eye strain in such cases.

[0111] Furthermore, based on the lighting scene characteristics (such as light intensity and color temperature) obtained by the cabin light sensor and the user's physiological state characteristics (such as heart rate and blink rate) obtained by physiological monitoring equipment (such as heart rate sensors and eye trackers), combined with the analyzed screen brightness adjustment rules, the display brightness parameters are adjusted. For example, when the cabin light intensity is detected to be high and the user is awake, the screen brightness can be increased to an appropriate level according to the rules; when the light intensity is detected to be low and the user is fatigued, the screen brightness can be reduced. During the adjustment process, a smooth transition can be used to avoid sudden changes in screen brightness that may cause visual shock to the user.

[0112] For example, screen brightness adjustment rules are obtained from the adaptive playback strategy. The cabin lighting scene feature is set to l, which is a multi-dimensional numerical set that may include information on dimensions such as light intensity and light uniformity. The user's physiological state feature is set to p, which is also a multi-dimensional numerical set that includes dimensions such as the user's fatigue level and excitement level. The display brightness parameter is initially set to d0, which is a multi-dimensional parameter set that may include brightness settings for different display areas.

[0113] Light intensity classification is performed based on the light intensity dimension value in the cabin lighting scene feature l, set to l_bright. Assume that different light intensity ranges are set, such as [l_low, l_mid] for normal lighting and [l_mid, l_high] for high lighting. (These ranges are multidimensional, corresponding to the dimensions of l; only the light intensity dimension is used as an example here.) When l_bright is within different ranges, the display brightness parameters are adjusted based on the fatigue level dimension value in the user's physiological state feature p, set to p_fatigue.

[0114] For example, when l_bright is in the bright light range and p_fatigue is above a certain threshold p_threshold (p_threshold is a numerical set of the same dimension as p, and only the fatigue dimension is used as an example here), according to the screen brightness adjustment rules, the display brightness needs to be reduced. It is assumed that the degree of reduction is related to the degree to which l_bright exceeds l_mid and the degree to which p_fatigue exceeds p_threshold. The reduction is calculated by setting the proportional coefficient related to l_bright to k2 and the proportional coefficient related to p_fatigue to k3 according to a preset proportional relationship (k2 and k3 are both multi-dimensional coefficient sets corresponding to the dimensions of l and p). The reduction is △d1 = (l_bright - l_mid) × k2 + (p_fatigue - p_threshold) × k3, where △d1 is a multi-dimensional brightness reduction set, and each dimension corresponds to a different display area or brightness adjustment aspect. At this time, the adjusted display brightness parameter d1 = d0 - Δd1 (the "-" here represents a subtraction operation on each corresponding dimension to ensure dimensional uniformity).

[0115] When l_bright is within the normal lighting range and p_fatigue is within a different range, the screen brightness adjustment rules are also used for corresponding adjustments. For example, if p_fatigue falls below another threshold, p_low_threshold, the display brightness may need to be appropriately increased. Assuming the increase is calculated similarly, based on a preset proportional relationship, with the relevant proportional coefficients k4 and k5, the increase is △d2 = (p_low_threshold - p_fatigue) × k4 + other relevant factors × k5 (where "other relevant factors" may be other dimensional information in l or p, determined according to the rules). △d2 is a multi-dimensional set of brightness increase amounts. The adjusted display brightness parameter d2 = d0 + △d2 (calculated for each dimension to ensure dimensional consistency). In this way, the display brightness parameters are adjusted according to the screen brightness adjustment rules based on the different cabin lighting scene characteristics l and the user physiological state characteristics p.

[0116] In step S154, the noise reduction intensity control rule in the adaptive playback strategy is parsed, and the active noise reduction intensity parameter is adjusted according to the noise interference characteristics and the user's physiological fatigue level;

[0117] In the disclosed embodiment, the noise reduction intensity control rules are formulated based on the noise interference characteristics and the user's physiological fatigue level. The rules can be read to parse the active noise reduction intensity adjustment range and method corresponding to different noise interference levels and user physiological fatigue levels. For example, the rules may stipulate that when the noise interference is large and the user's physiological fatigue level is high, the active noise reduction intensity should be set to a higher level to provide a quieter listening environment; while when the noise interference is small and the user's physiological fatigue level is low, the active noise reduction intensity can be appropriately reduced to save energy.

[0118] Furthermore, the active noise reduction intensity parameters are adjusted based on the noise interference characteristics (such as noise frequency and amplitude) obtained by the ambient noise sensor and the user's physiological fatigue level obtained by physiological monitoring equipment, combined with the noise reduction intensity control rules obtained through analysis. For example, when a high-frequency noise interference is detected in the cabin and the user has a high heart rate and a low blink rate (indicating possible fatigue), the active noise reduction intensity can be increased to a higher level according to the rules. When the noise interference decreases and the user's physiological fatigue level decreases, the active noise reduction intensity can be appropriately reduced. During the adjustment process, the noise reduction effect can be monitored in real time, and fine-tuning can be made based on actual conditions to ensure the optimal noise reduction effect.

[0119] For example, the noise reduction intensity control rules are extracted from the adaptive playback strategy. The noise interference feature is set to n, which is a multi-dimensional numerical set, such as noise intensity and noise distribution at different frequencies. The user's physiological fatigue level is set to f, which is a multi-dimensional numerical set. For example, the fatigue level may be derived from different physiological indicators, and each dimension represents the contribution of different indicators to the fatigue level. The active noise reduction intensity parameter is initially set to r0, which is a multi-dimensional parameter set, such as the noise reduction intensity setting for different frequency bands.

[0120] For the noise interference feature n, analyze the noise intensity dimension at different frequencies, set to n_intensity_f (f represents different frequencies, and n_intensity_f is a multi-dimensional set of values, each dimension corresponding to the noise intensity at a frequency). Based on the noise spectrum matching process, the value of a dimension in the user's physiological fatigue level f, set to f_dim, is used to adjust the active noise reduction intensity parameter.

[0121] Assume that when n_intensity_f exceeds a certain threshold n_threshold_f at certain frequencies (n_threshold_f is a set of values ​​with the same dimensions as n_intensity_f, with each dimension corresponding to a noise intensity threshold at a frequency), and f_dim reaches a certain level, ANC strength needs to be increased. The increased strength is calculated as follows: based on a preset relationship, let the coefficient associated with n_intensity_f be k6, and the coefficient associated with f_dim be k7 (both k6 and k7 are multidimensional coefficient sets corresponding to the dimensions of n and f). The increased strength is △r1 = (n_intensity_f - n_threshold_f) × k6 + f_dim × k7, where △r1 is a multidimensional set of ANC strength increase amplitudes, each dimension corresponding to a different frequency band or noise reduction adjustment amplitude. The adjusted ANC strength parameter r1 is now r0 + △r1 (calculated for each dimension to ensure dimensional consistency).

[0122] When n_intensity_f is lower than another threshold n_low_threshold_f at certain frequencies and f_dim is in different ranges, according to the noise reduction intensity control rules, the active noise reduction intensity may need to be reduced or kept unchanged. For example, if n_intensity_f is lower than n_low_threshold_f and f_dim is also low, the calculation method of the reduced intensity is similar. According to the preset coefficients k8 and k9, the reduced intensity is △r2 = (n_low_threshold_f-n_intensity_f) × k8 + other relevant factors × k9 ("other relevant factors" may be other dimensional information in n or f, determined according to the rules). △r2 is a multi-dimensional active noise reduction intensity reduction amplitude set. At this time, the adjusted active noise reduction intensity parameter r2 = r0-△r2 (calculated correspondingly for each dimension to ensure dimensional consistency). In this way, the active noise reduction intensity parameter is adjusted according to the noise interference feature n and the user's physiological fatigue level f according to the noise reduction intensity control rules.

[0123] In step S155, a playback control instruction is generated based on the target playback content list, the initial volume parameter, the display brightness parameter and the active noise reduction intensity parameter, and the playback control instruction is sent to the in-vehicle audio and video playback device to perform a real-time playback operation.

[0124] In the disclosed embodiment, the target playback content list, initial volume parameter, display brightness parameter, and active noise reduction intensity parameter are integrated to generate a playback control instruction. The playback control instruction is usually encapsulated in a specific data format (such as JSON, XML), containing the specific values ​​of each parameter and the corresponding control command. For example, the instruction will clearly specify the target content file name to be played, the initial volume level, the display brightness value, and the active noise reduction intensity level.

[0125] Furthermore, the generated playback control instructions are sent to the in-vehicle audio and video playback device through the in-vehicle network (such as CAN bus, Ethernet). After receiving the instructions, the in-vehicle audio and video playback device will parse the instruction content and adjust its own playback status according to the parameters in the instruction. For example, the playback device will select the file to be played according to the target playback content list, set the volume according to the initial volume parameter, adjust the screen brightness according to the display brightness parameter, and start or adjust the active noise reduction function according to the active noise reduction intensity parameter, thereby realizing real-time playback operation.

[0126] For example, the obtained target playback content list L, the adjusted initial volume parameter v, the adjusted display brightness parameter d (d may be d1, d2, etc. adjusted according to different conditions) and the adjusted active noise reduction intensity parameter r (r may be r1, r2, etc. adjusted according to different conditions) are integrated. When generating the playback control instruction, it is encoded in a format that can be recognized by the in-vehicle audio and video playback device. For example, for the target playback content list L, each media resource identifier in the list is converted into an instruction format that can be recognized by the device, and the converted format is set to L_code. For the initial volume parameter v, the volume value of each dimension is converted according to the volume adjustment instruction format of the device, and the converted format is set to v_code. Similarly, the display brightness parameter d is converted to d_code, and the active noise reduction intensity parameter r is converted to r_code.

[0127] Then, L_code, v_code, d_code and r_code are combined into a playback control instruction C in a certain order. For example, C = [L_code, v_code, d_code, r_code] (the combination here is only an example, and the actual combination is determined according to the device communication protocol). Finally, the playback control instruction C is sent to the vehicle-mounted audio and video playback device through the communication link between the vehicle-mounted system and the vehicle-mounted audio and video playback device. After the device receives the instruction C, it parses the various parts of the instruction, selects the corresponding target playback content according to L_code, adjusts the volume according to v_code, sets the display brightness according to d_code, and adjusts the active noise reduction intensity according to r_code, thereby performing real-time playback operations and providing users with an audio and video playback experience adapted to the current vehicle environment.

[0128] In a possible implementation, in step S151, parsing the playback content priority rule in the adapted playback strategy and filtering the target playback content list from the media resource library according to the playback content priority rule includes:

[0129] In step S1511, the media type preference weight and content theme preference tag in the user's historical playback preference characteristics are obtained;

[0130] In the disclosed embodiment, the user's operating behavior on the in-vehicle audio and video playback device is recorded over a long period of time, such as the frequency, duration, and completeness of the user's playback of different types of media (such as music, audiobooks, videos, etc.). Through statistical analysis of these historical data, a machine learning algorithm (such as a decision tree, a neural network, etc.) is used to calculate the user's preference weight for each media type. For example, if the user frequently plays music and each playback duration is long, the preference weight for music will be relatively high; if the user rarely plays videos, the preference weight for videos will be low.

[0131] Furthermore, natural language processing techniques (such as text mining, topic models, etc.) are used to perform topic analysis on the media content played by the user. For audio content, it can be first converted into text (such as through speech recognition technology), and then the text can be subjected to topic extraction; for video content, subtitles, audio text, and image features in the video can be extracted, and its theme can be determined through comprehensive analysis. Common topic tags may include news, entertainment, education, sports, etc. The number and frequency of users playing content with different themes can be counted to determine the user's content topic preference tags. For example, if a user often plays content related to sports, then "sports" will be marked as the user's preferred topic tag.

[0132] In step S1512, real-time semantic keywords and content screening conditions in the user's current playback intention feature are obtained;

[0133] In the disclosed embodiment, real-time semantic keywords of the user's current playback intention are obtained through a variety of methods. One method is to use voice recognition technology to monitor the user's voice commands in the cockpit in real time. For example, the user says "I want to listen to XXX's song", and voice recognition can convert the voice into text, and extract keywords such as "XXX" and "song" through natural language processing technology. Another method is to analyze the user's current operation behavior, such as the user's search input and click operations on the media resource library interface. For example, if the user enters "science fiction movie" in the search box, keywords such as "science fiction" and "movie" can be extracted.

[0134] Furthermore, content filtering criteria may not only be derived from the user's voice commands and operating behaviors, but also from the vehicle's current state and environmental information. For example, if the vehicle is on a long journey, "suitable for long-distance driving" could be an implicit filtering criterion; if the cabin is dimly lit, "suitable for nighttime viewing" could be used as a content filtering criterion.

[0135] In step S1513, a first candidate content set is extracted from a media resource library according to the media type preference weight and the content theme preference tag;

[0136] In the embodiment of the present disclosure, all media contents in the media resource library are traversed. For each media content, its media type and content theme information are obtained. Based on the obtained user media type preference weight and content theme preference label, the content in the media resource library is preliminarily screened. For each media content, its matching degree with the user preference can be calculated. The matching degree can be calculated using a weighted summation method, for example, the matching degree between the media type of the media content and the user media type preference weight is multiplied by the corresponding weight, and the matching degree between the content theme and the user content theme preference label is multiplied by the corresponding weight. If the matching degree exceeds a certain threshold, the media content will be added to the first candidate content set. For example, if the user's preference weight for music is high, and the current media content is music and the theme meets the user's preference label, then the media content is more likely to be added to the first candidate content set.

[0137] In step S1514, content matching degree is calculated for the first candidate content set according to the real-time semantic keywords and the content screening condition to obtain a second candidate content set;

[0138] In the disclosed embodiment, for each media content in the first candidate content set, the content matching degree is calculated based on the real-time semantic keywords and content screening conditions obtained in step S1512. For real-time semantic keywords, a text similarity algorithm (such as cosine similarity, Jaccard similarity, etc.) is used to calculate the similarity between the text information such as the title, description, and tags of the media content and the keywords. For example, if the title of the media content contains keywords input by the user, the similarity will be higher. For content screening conditions, the media content can be judged according to the specific requirements of the conditions. For example, if the screening condition is "duration is less than 60 minutes", it can be checked whether the duration of the media content meets the condition.

[0139] Furthermore, based on the calculated content matching results, media content with a higher matching degree is retained in the first candidate content set to form a second candidate content set. The matching degree threshold can be set based on actual conditions. For example, it can be set to the average matching degree of all media content plus a certain deviation value to ensure that the selected content not only meets the user's current intent but also has a certain degree of diversity.

[0140] In step S1515 , the second candidate content set is sorted based on the playback content priority rule to generate a target playback content list.

[0141] In the disclosed embodiment, the playback content priority rules in the adapted playback strategy are read. These rules may be formulated based on various factors, such as the timeliness of the content (e.g., news has a higher timeliness), the user's urgent needs (e.g., navigation prompts have the highest priority), the popularity of the content (e.g., content with higher user ratings has a higher priority), etc. Based on these rules, a priority value is assigned to each media content in the second candidate content set.

[0142] Furthermore, the second candidate content set is sorted according to the assigned priority values, with media content with higher priority values ​​being ranked higher. Then, based on actual requirements (such as a playlist length limit), a certain number of media content can be selected from the sorted set to generate a target playlist. For example, if the playlist length limit is 10, the 10 media content with the highest priority will be selected as the target playlist.

[0143] In one possible implementation, in step S152, parsing the playback volume adjustment rules in the adapted playback strategy and dynamically adjusting the initial volume parameters of the target playback content according to the vehicle driving scenario safety factor includes:

[0144] In step S1521, a safety level threshold range corresponding to the vehicle driving scenario safety factor is obtained;

[0145] In the disclosed embodiment, the safety level threshold range corresponding to the vehicle driving scenario safety factor is determined based on a comprehensive assessment of the safety risks in different driving scenarios. These assessment factors include vehicle speed, road type (such as highways, urban roads, rural roads, etc.), traffic flow, weather conditions (such as rainy days, foggy days, sunny days, etc.), and the vehicle's surrounding environment (such as whether there are pedestrians, obstacles, etc.). For example, when driving at high speed on a highway with heavy traffic, the safety risk is relatively high, and the corresponding safety level threshold range will be low; while when driving at low speed on an urban road with light traffic, the safety risk is relatively low, and the corresponding safety level threshold range will be high.

[0146] By collecting and analyzing a large amount of actual driving data, combined with expert experience and simulation experiments, we can assign corresponding safety factor threshold ranges to different safety levels. For example, safety levels can be divided into low, medium, and high, each corresponding to a different safety factor threshold range. The safety factor threshold range for low safety level might be 0 to 0.3, for medium safety level 0.3 to 0.7, and for high safety level 0.7 to 1.0.

[0147] In step S1522, when it is detected that the safety factor of the vehicle driving scene is lower than the first safety threshold, the volume automatic increase operation is triggered, and the volume increase amplitude is calculated according to the safety factor difference;

[0148] In the disclosed embodiment, a vehicle driving scenario safety factor is obtained in real time. This safety factor is calculated using data collected by various sensors (such as a speed sensor, an accelerometer, a radar sensor, and a camera). When the safety factor is detected to be lower than a preset first safety threshold (e.g., 0.3), it can be determined that the current driving scenario presents a high safety risk and that measures need to be taken to improve the driver's awareness of the surrounding environment, thereby triggering an automatic volume increase.

[0149] In the embodiment of the present disclosure, the volume boost is calculated based on the safety factor difference. The safety factor difference refers to the difference between the first safety threshold and the current safety factor. For example, if the first safety threshold is 0.3 and the current safety factor is 0.1, the safety factor difference is 0.2. A mapping relationship between the volume boost and the safety factor difference can be pre-set, and the mapping relationship can be obtained through experiments and data analysis. Generally speaking, the larger the safety factor difference, the larger the volume boost. For example, when the safety factor difference is in the range of 0 to 0.1, the volume boost is 5 decibels; when the safety factor difference is in the range of 0.1 to 0.2, the volume boost is 10 decibels, and so on. Based on the calculated volume boost, the initial volume parameters of the target playback content can be automatically increased while ensuring that it does not cause excessive interference to the driver.

[0150] In step S1523, when it is detected that the safety factor of the vehicle driving scene is higher than the second safety threshold, the volume hold operation is triggered and the current volume parameter is maintained unchanged;

[0151] In the disclosed embodiment, when the driving scenario safety factor is detected to be higher than a preset second safety threshold (e.g., 0.7), it can be determined that the current driving scenario is relatively safe, the driver's awareness of the surrounding environment is strong, and there is no need to adjust the volume to improve attention. Therefore, the volume hold operation is triggered to maintain the current volume parameter unchanged.

[0152] Furthermore, the volume parameters of the current target content are recorded and maintained during subsequent playback until the safety factor changes and other volume adjustment conditions are met. This ensures that users can enjoy audio and video content at a comfortable volume in safe driving scenarios while avoiding the impact of frequent volume changes on the user experience.

[0153] In step S1524, when it is detected that the vehicle driving scene safety factor is between the first safety threshold and the second safety threshold, a volume gradual adjustment operation is triggered, and the volume adjustment step size is dynamically adjusted according to the safety factor change rate;

[0154] In the disclosed embodiment, when the driving scene safety factor is detected to be between a first safety threshold (e.g., 0.3) and a second safety threshold (e.g., 0.7), the current driving scene can be determined to be at a medium safety level, and the volume needs to be dynamically adjusted based on the change in the safety factor to achieve the goal of not affecting the driver's perception of the surrounding environment while allowing the user to enjoy the audio and video content normally. Therefore, the volume gradual adjustment operation is triggered.

[0155] The volume adjustment step size is dynamically adjusted based on the rate of change of the safety factor. The rate of change of the safety factor refers to the amount of change in the safety factor per unit time. The rate of change of the safety factor can be calculated in real time, and the volume adjustment step size can be determined based on pre-set rules. For example, when the rate of change of the safety factor is slow, it indicates that the safety status of the driving scene changes relatively smoothly, and the volume adjustment step size can be smaller, such as 1-2 decibels per adjustment; when the rate of change of the safety factor is fast, it indicates that the safety status of the driving scene changes more drastically, and the volume adjustment step size can be larger, such as 3-5 decibels per adjustment. Based on the calculated volume adjustment step size, the volume parameters of the target playback content can be gradually adjusted to achieve gradual volume adjustment.

[0156] In step S1525 , an adjusted initial volume parameter is generated based on the volume boost amplitude, the volume adjustment step, and the current volume parameter.

[0157] In the embodiment of the present disclosure, the adjusted initial volume parameter is generated based on the determined volume boost amplitude, volume adjustment step size and current volume parameter. If the current operation is automatic volume boost (step S1522), the current volume parameter can be added to the volume boost amplitude to obtain the adjusted initial volume parameter. For example, if the current volume parameter is 30 decibels and the volume boost amplitude is 10 decibels, the adjusted initial volume parameter is 40 decibels. If the current operation is gradual volume adjustment (step S1524), the volume can be gradually adjusted based on the volume adjustment step size and the current volume parameter. For example, if the current volume parameter is 30 decibels, the volume adjustment step size is 2 decibels, and after one adjustment, the adjusted initial volume parameter is 32 decibels. In the process of generating the initial volume parameter, the upper and lower limits of the volume can be considered to ensure that the adjusted volume parameter is within a reasonable range and does not cause discomfort to the driver and passengers. For example, the upper limit of the volume can be set to 70 decibels and the lower limit can be set to 20 decibels.

[0158] Next, it is assumed that this in-vehicle intelligent audio and video adaptation playback method involves an artificial intelligence model for constructing a playback strategy generation network. The construction and training process of the artificial intelligence model is described below.

[0159] Step S210: Constructing a playback strategy generation network model.

[0160] The model consists of three key modules: a playback scenario matching unit, an environmental interference suppression unit, and a comprehensive decision-making unit. The playback scenario matching unit prioritizes the user's dynamic playback requirements and the vehicle's operational adaptation characteristics; the environmental interference suppression unit focuses on suppressing the interference between the cabin's adaptive characteristics and the user's dynamic playback requirements; and the comprehensive decision-making unit integrates the results of the first two units to generate the final playback strategy.

[0161] In terms of the hierarchical structure of the model, starting from the input layer, the input layer receives the user's dynamic playback demand characteristics, vehicle operation adaptation characteristics and cabin environment adaptation characteristics. These features pass through the preprocessing layer, which normalizes or standardizes the input features to ensure the uniformity of dimensions and dimensional matching between different features. For example, for the different dimensional parameters in the user's dynamic playback demand characteristics, assuming they are u1, u2, u3, etc., through a specific normalization function, such as normalizing u1 to the [0, 1] interval, the normalized parameter is set to u1_norm, and the calculation method is u1_norm = (u1-u1_min) / (u1_max-u1_min) (this is just an example, the actual normalization method is determined according to the characteristics), and the other dimensional parameters are processed in the same way. Similar preprocessing is also performed on the vehicle operation adaptation characteristics and the cabin environment adaptation characteristics.

[0162] The preprocessed features enter the hidden layer, which contains multiple neuron layers. Assume there are three hidden layers: hidden_layer1, hidden_layer2, and hidden_layer3. Each hidden layer is connected to the previous layer via a weight matrix. For example, hidden_layer1 is connected to the preprocessing layer via weight matrix W1, hidden_layer2 is connected to hidden_layer1 via weight matrix W2, and hidden_layer3 is connected to hidden_layer2 via weight matrix W3. Neurons undergo nonlinear transformations using activation functions. Assuming the commonly used ReLU activation function, for neuron i in hidden_layer1, its input is x_i, and its output is y_i = max(0, x_i), where x_i is the sum of the products of the previous layer's output and the corresponding elements of weight matrix W1, i.e., x_i = ∑(j)w1_ij × input_j (where input_j is the jth output of the preprocessing layer and w1_ij is the element in row i and column j of weight matrix W1). The same goes for hidden_layer2 and hidden_layer3.

[0163] Finally, the output of the hidden layer enters the output layer, which, based on the functions of the playback scenario matching unit, the environmental interference suppression unit, and the comprehensive decision-making unit, outputs the corresponding playback strategy-related results. For example, the playback scenario matching unit outputs a numerical representation of the first set of playback constraints, and the environmental interference suppression unit outputs a numerical representation of the second set of playback constraints. The comprehensive decision-making unit further processes these outputs to generate the final numerical representation of the adapted playback strategy. These numerical representations are then converted into the actual playback strategy rules through operations such as decoding.

[0164] Step S220: Prepare training data.

[0165] The training data includes a large number of vehicle environment status monitoring data set samples and corresponding known adaptive playback strategies. For each vehicle environment status monitoring data set sample, it also includes vehicle operation status data, cabin environment status data and user behavior status data. These data samples are collected in different actual vehicle scenarios to ensure the diversity and authenticity of the data. For example, data collection is carried out under different driving conditions (urban roads, highways, rural roads, etc.), different cabin environment conditions (different light intensity, noise level, air quality, etc.) and different user behavior patterns (different playback preferences, physiological states, interactive behaviors, etc.).

[0166] For each data sample, a multi-level playback demand feature extraction operation is performed according to the method described above to generate corresponding user dynamic playback demand features, vehicle operation adaptation features, and cabin environment adaptation features. At the same time, the adaptive playback strategy actually adopted in the vehicle environment is recorded and converted into a numerical representation that matches the output of the playback strategy generation network. For example, for the playback content priority rule, the priority order of different content is converted into a numerical vector, and each element in the vector represents the priority value of a content; similar numerical processing is also performed for the playback volume adjustment rules, screen brightness adjustment rules, etc. In this way, each training data sample consists of an extracted feature set and the corresponding numerical representation of the adaptive playback strategy.

[0167] Step S230: training the playback strategy to generate a network model.

[0168] Set the training parameters, including the learning rate η, the number of iterations N, and the loss function L. The loss function L is used to measure the difference between the playback strategy predicted by the model and the actual adaptive playback strategy. For example, the mean square error loss function can be used. For each training data sample, let the playback strategy value predicted by the model be denoted as y_pred and the actual adaptive playback strategy value be denoted as y_true. The loss function value is L = ∑(i)(y_pred_i-y_true_i) 2 (The summation here is performed on all dimensions i, determined according to the dimensions represented by the actual playback strategy values).

[0169] In each iteration, a batch of data samples is randomly selected from the training dataset, with the number of samples being B. The user's dynamic playback demand characteristics, vehicle operating state adaptation characteristics, and cabin environment adaptation characteristics of these samples are input into the playback strategy generation network model. The model performs forward propagation calculations based on the current weight matrix to obtain a numerical representation of the predicted playback strategy.

[0170] The loss function is then used to calculate the loss between the predicted and actual results. The backpropagation algorithm is used to calculate the gradient of the loss function with respect to each element of the weight matrix. For example, for the weight matrix W1, the backpropagation algorithm calculates the gradient matrix ▽W1. This calculation involves backpropagating the error from the output layer to the input layer, and using the chain rule to calculate the degree of influence of each weight on the loss function.

[0171] Based on the calculated gradient, the weight matrix is ​​updated at the learning rate η. For example, for weight matrix W1, the update formula is W1 = W1 - η × ▽ W1. Similarly, updates are performed for W2 and W3. This process is repeated for N training iterations until the loss function converges to a smaller range, indicating that the model has achieved good performance and can accurately generate an adaptive playback strategy based on the input vehicle environment characteristics.

[0172] Through this training process, the playback strategy generation network model can learn the complex relationship between different in-vehicle environment characteristics and adaptive playback strategies. Once trained, the model can be applied to actual in-vehicle intelligent audio and video adaptive playback systems.

[0173] In a possible implementation, in step S11, performing a multi-level user playback demand feature extraction operation on the acquired user behavior state data to generate user dynamic playback demand features includes:

[0174] In step S111, the user operation record data in the user behavior state data is subjected to time series behavior pattern analysis to obtain the user's historical playback preference characteristics;

[0175] User operation record data refers to the data generated by various operations performed by users on the in-vehicle audio and video system while using the system. For example, the time, frequency, and operation object (specific song, video, etc.) of operations such as play, pause, fast forward, rewind, switch songs, favorite, and like are used to reflect users' past operating habits and preferences.

[0176] Temporal behavior pattern analysis involves analyzing user operation records in chronological order to uncover patterns and patterns in user operations over different time periods. For example, this could include analyzing what type of music users prefer to play during specific time periods (such as commuting), or analyzing how their audio and video content choices differ between weekdays and weekends.

[0177] Among them, the user's historical playback preference characteristics are obtained by analyzing and processing the time-series behavior pattern of the user's operation record data. They are used to describe the user's preference characteristics for audio and video content in the past period of time, such as the user's favorite music types (pop, rock, classical, etc.), frequently watched video types (movies, TV series, variety shows, etc.), favorite singers or actors, etc.

[0178] In the disclosed embodiment, the user operation record data is pre-processed, including data cleaning (removing duplicate, erroneous or incomplete data records), data sorting (sorting by operation time) and other operations. Then, time series analysis methods, such as sliding window analysis and Markov models, are used to mine the user operation record data. Sliding window analysis can divide the user operation record data according to a certain time window, and count the number of user operations on different types of audio and video content in each window, so as to discover the changes in user preferences in different time periods. The Markov model can be used to analyze the transition probability between user operations, such as the probability of a user switching from playing one popular song to playing another popular song, and the probability of switching to other types of songs, so as to infer the user's playback preference pattern. Through these analysis methods, the user's historical playback preference characteristics are obtained.

[0179] For example, based on the user's operation records of the in-car audio and video system over a period of time, assume that the operation records during this period are [clicking song A is recorded as s1, switching to random play mode is recorded as r1, clicking song A again is recorded as s2, switching to local music play mode is recorded as l1, clicking song B is recorded as s3, and clicking song A again is recorded as s4]. Analyzing these operations in chronological order, it is found that song A is clicked many times and often switches to local music play mode. This derives the user's historical playback preference characteristics, sets the preference weight for the music genre to which song A belongs as w1, and sets the preference for local music play mode as w2, forming the user's historical playback preference characteristics set [w1, w2, ...].

[0180] In step S112, real-time semantic parsing is performed on the user interaction response data in the user behavior status data to extract the user's current playback intention features;

[0181] User interaction response data is generated when users interact with the in-vehicle audio and video system, including voice commands, text input, touch screen operation feedback, etc. This data reflects the user's current needs and intentions for audio and video playback.

[0182] In the embodiment of the present disclosure, when performing real-time semantic parsing of user interaction response data, it is first necessary to build a semantic parsing model. The model can adopt a rule-based method or a machine learning-based method. The rule-based method is to parse the instructions or text input by the user according to pre-defined grammatical rules and semantic rules. For example, some keywords and corresponding semantic meanings are defined, and when these keywords appear in the user input, the user's intention is determined according to the rules. The machine learning-based method can use a natural language processing (NLP) model, such as a recurrent neural network (RNN), a long short-term memory network (LSTM) or a Transformer model, to train a large amount of user interaction data, so that the model can automatically learn the semantics and intentions of the user input. In actual applications, the user interaction response data is input into the semantic parsing model, and the model will output the user's current playback intention features, such as the type of song the user wants to play, the singer and other specific information.

[0183] For example, a user interacts with the in-vehicle system, such as saying "Play upbeat music." The in-vehicle voice interaction module receives the speech, converts it into text, analyzes the text, and extracts the key information "upbeat music." It determines that the user's current playback intent includes a desire for upbeat music, set to i1. If the user touches the screen and selects the "Classic Songs" category, the interaction is analyzed to extract a playback intent feature for classic songs, set to i2, forming the user's current playback intent feature set [i1, i2, ...].

[0184] In step S113, the physiological state fluctuation recognition processing is performed on the user physiological monitoring data in the user behavior state data to generate the user physiological state characteristics;

[0185] Among them, user physiological monitoring data is data related to the user's physiological state obtained through vehicle-mounted equipment or other wearable devices, such as heart rate, blood pressure, body temperature, skin galvanic response, etc. This data can reflect the user's emotional state, fatigue level and other physiological changes during driving.

[0186] In the embodiment of the present disclosure, when performing physiological state fluctuation identification processing on the user's physiological monitoring data, it is first necessary to pre-process the physiological monitoring data, such as filtering processing (removing noise interference), normalization processing (unifying data of different dimensions into a specific interval), etc. Then, feature extraction methods are used, such as time domain feature extraction (calculating the average value, standard deviation, maximum value, minimum value, etc. of data such as heart rate and blood pressure), frequency domain feature extraction (analyzing the frequency components of physiological signals through methods such as Fourier transform), etc., to extract the features of the physiological monitoring data. Then, classification or regression algorithms, such as support vector machines (SVM), decision trees, neural networks, etc., are used to analyze and identify the extracted features. For example, by training a classification model, different physiological states (such as excitement, calmness, fatigue, etc.) are used as category labels, and the model is allowed to learn the characteristic distribution of physiological monitoring data under different physiological states, thereby realizing the identification of the user's physiological state. Finally, the user's physiological state features are generated.

[0187] For example, assume that a user's heart rate values ​​are h1, h2, and h3, and their blood pressure values ​​are p1, p2, and p3 over a period of time. Set the normal heart rate range to h_min to h_max, and the normal blood pressure range to p_min to p_max. Observe changes in heart rate and blood pressure. If the heart rate changes from h1 to h3, or the blood pressure changes from p1 to p3, outside the normal fluctuation range, determine that the user's physiological state has changed, such as excitement. Set the excitement level to e1, and generate the user's physiological state feature set [e1, ...].

[0188] In step S114, the user's historical playback preference features, the user's current playback intention features, and the user's physiological state features are subjected to feature fusion processing to generate the user's dynamic playback demand features.

[0189] In the disclosed embodiment, feature fusion processing can adopt a variety of methods, such as weighted fusion, splicing fusion, attention mechanism fusion, etc. Weighted fusion is to assign a weight to each feature according to the importance of each feature, and then multiply each feature by the corresponding weight and add them together to obtain the fused feature. For example, assuming that the importance weights of the user's historical playback preference feature, the user's current playback intention feature, and the user's physiological state feature are w1, w2, and w3 respectively, the fused feature F can be expressed as F = w1×F1+w2×F2+w3×F3, where F1, F2, and F3 are the three original features respectively. Splicing fusion is to directly splice the three features together to form a longer feature vector. Attention mechanism fusion is to highlight the influence of important features by calculating the attention weight between each feature, and then fuse each feature according to the attention weight. Through these feature fusion methods, the user's historical playback preference feature, the user's current playback intention feature, and the user's physiological state feature are integrated to generate the user's dynamic playback demand feature.

[0190] For example, the user's historical playback preference feature set [w1, w2, ...], the user's current playback intention feature set [i1, i2, ...], and the user's physiological state feature set [e1, ...] are fused. Weights are assigned to different features: in the user's historical playback preference feature, the weight of w1 is set to k1, and the weight of w2 is set to k2; in the user's current playback intention feature, the weight of i1 is set to k3, and the weight of i2 is set to k4; and in the user's physiological state feature, the weight of e1 is set to k5. Through weighted calculation, for example, the fused value of w1 is w1×k1, the fused value of i1 is i1×k3, and the fused value of e1 is e1×k5. These weighted values ​​are concatenated to form the user's dynamic playback demand feature set [w1×k1, i1×k3, e1×k5, ...].

[0191] In a possible implementation, in step S13, performing a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate a vehicle operation adaptation feature further includes:

[0192] In step S131, the vehicle speed change data in the vehicle running state data is processed into driving scene types to obtain vehicle driving scene type features;

[0193] Vehicle speed change data records the changes in vehicle speed over time during driving, including increases, decreases, and speed fluctuations. This data can be used to analyze the vehicle's driving status and scenarios. Vehicle driving scenario type features describe the characteristics of the vehicle's current driving scenario.

[0194] In the disclosed embodiment, the vehicle speed change data is preprocessed, including data filtering (removing noise interference), data smoothing (eliminating instantaneous fluctuations) and other operations to improve the accuracy and stability of the data. Then, a clustering algorithm (such as the K-Means clustering algorithm) or a rule-based method is used to divide the driving scene. Taking the K-Means clustering algorithm as an example, the vehicle speed change data is used as a feature vector, and the data points are divided into different clusters through iterative calculation, and each cluster represents a driving scene. In the division process, the characteristics of different scenes can be defined according to statistics such as the mean value, variance, maximum value, and minimum value of the vehicle speed. For example, a cluster with a low mean speed and a large variance is divided into an urban congestion scene, and a cluster with a high mean speed and a small variance is divided into a highway scene, thereby obtaining the vehicle driving scene type characteristics.

[0195] For example, assume the vehicle speed data is [v1, v2, v3, v4, v5]. The speed range and driving state are set to define driving scenarios. For example, a vehicle with a speed between v_low and v_mid and frequent starts and stops is considered a city congestion scenario, while a vehicle with a speed between v_mid and v_high and stable driving is considered a highway scenario. The current scenario is determined based on the actual vehicle speed and driving state. If it meets the conditions for a city congestion scenario, the scenario identifier is set to sc1, forming the vehicle driving scenario type feature set [sc1, ...].

[0196] In step S132, a vibration frequency analysis process is performed on the vehicle vibration data in the vehicle operation status data to extract cabin vibration interference characteristics;

[0197] The cabin vibration interference feature describes the interference of cabin vibration on audio and video playback. For example, vibrations of certain frequencies may cause noise in audio equipment or affect screen display effects.

[0198] In the disclosed embodiment, the vehicle vibration data is pre-processed, such as pre-emphasis processing (enhancing high-frequency components) before Fourier transform, to improve the spectral characteristics of the signal. Then, a spectrum analysis method such as fast Fourier transform (FFT) is used to perform vibration frequency analysis on the vibration data. FFT can convert the vibration signal in the time domain into a spectrum diagram in the frequency domain, and determine the main frequency distribution of the vibration signal by analyzing the amplitudes of different frequency components in the spectrum diagram. Then, based on the preset interference frequency threshold, the vibration frequency range that may interfere with audio and video playback is extracted to generate a cabin vibration interference feature. For example, if it is found that the vibration amplitude in a certain frequency range exceeds the threshold, and the frequency range is close to the sensitive frequency of the audio equipment or screen, then the frequency range is used as part of the cabin vibration interference feature.

[0199] For example, when driving on an uneven road, the vibration sensor collects data, such as the vibration intensity z1 at frequency f1 and the vibration intensity z2 at frequency f2. The vibration intensities at different frequencies are analyzed and a vibration intensity threshold z_threshold is set. If z1 is greater than z_threshold, the vibration at frequency f1 is considered to be interfering with the cabin. The interference level is set to dz1, forming the cabin vibration interference feature set [dz1, dz2, ...].

[0200] In step S133, performing path semantic parsing processing on the navigation path data in the vehicle operation status data to generate navigation path association features;

[0201] The navigation path association feature is used to describe the characteristics of the association between the navigation path and the audio and video playback requirements. For example, when passing by a tourist attraction, the user may want to play related tourist introduction audio.

[0202] In the embodiment of the present disclosure, the navigation path data is preprocessed, such as performing word segmentation and part-of-speech tagging on the path text, so as to perform subsequent semantic analysis. Then, natural language processing (NLP) technology is used to perform path semantic parsing. A rule-based method can be used, such as defining some keywords and corresponding semantic rules. When these keywords appear in the path text, the semantic information of the path is determined according to the rules. For example, when keywords such as "tourist attractions" and "museums" appear in the path text, it is inferred that the user may be in a travel scenario and wants to play relevant travel introduction audio. A machine learning-based method can also be used, such as using a recurrent neural network (RNN) or a Transformer model to train a large amount of navigation path data and corresponding semantic labels, so that the model can automatically learn the mapping relationship between the path and the semantics. In actual applications, the navigation path data is input into the trained semantic parsing model, and the model outputs the navigation path association features.

[0203] For example, navigation route data includes road segment information and destination information. For example, if the route is "from a certain place to a shopping mall," the route is parsed to extract relevant information and play the corresponding music. If the destination is a shopping mall, the destination identifier is dest1. It is inferred that people near the shopping mall may prefer relaxing music, and the music preference level is m1. This forms the navigation route-related feature set [dest1, m1, ...].

[0204] In step S134, the vehicle driving scene type feature, the cabin vibration interference feature, and the navigation path association feature are dynamically associated to generate the vehicle operation adaptation feature.

[0205] In the disclosed embodiment, dynamic association processing can use weighted fusion, conditional association and other methods to assign different weights to the vehicle driving scene type features, cabin vibration interference features and navigation path association features, and then multiply each feature by the corresponding weight and add them together to obtain the fused features. According to different driving scene types, the cabin vibration interference features and navigation path association features are targetedly associated. For example, in a city congestion scenario, the cabin vibration interference feature may have a greater impact on the playback experience, so the weight of the cabin vibration interference feature can be increased; in a highway scenario, the navigation path association feature may be more important, such as recommending suitable music types based on the expected driving time and road conditions. In this case, the weight of the navigation path association feature can be increased. Through these dynamic association processing methods, the three features are comprehensively analyzed to generate vehicle operation adaptation features.

[0206] For example, the vehicle driving scene type feature set is [sc1, ...], the cabin vibration interference feature set is [dz1, dz2, ...], and the navigation path association feature set is [dest1, m1, ...]. During dynamic association processing, the weight of each feature is considered: the driving scene feature weight is set to q1, the vibration interference feature weight is set to q2, and the navigation path association feature weight is set to q3. For example, for the music preference feature, the new value is (dest1×q3+m1×q3+sc1×q1+dz1×q2+dz2×q2) / (q1+q2+q3). The processed values ​​are concatenated to form the vehicle operation adaptation feature set [(dest1×q3+m1×q3+sc1×q1+dz1×q2+dz2×q2) / (q1+q2+q3), ...].

[0207] In a possible implementation, in step S12, performing a multi-level environment playback requirement feature extraction operation on the acquired cabin environment state data to generate a cabin environment adaptation feature further includes:

[0208] In step S121, the light intensity data in the state monitoring data set is subjected to illumination scene classification processing to obtain cabin illumination scene features;

[0209] In the disclosed embodiment, the light intensity data is pre-processed, including data filtering (removing noise interference, such as abnormal values ​​caused by instantaneous fluctuations of the sensor), data normalization (unifying light intensity data of different ranges into a specific interval for easy subsequent processing), and other operations. Then, a classification algorithm (such as a decision tree classification algorithm) or a threshold-based method is used to classify the lighting scene. Taking the decision tree classification algorithm as an example, based on historical data and empirical knowledge, the light intensity range, change trend and other characteristics corresponding to different lighting scenes are determined as the basis for node division of the decision tree. For example, when the light intensity is greater than a higher threshold and the change is relatively stable, it is divided into a strong light scene during the day; when the light intensity is less than a lower threshold, it is divided into a weak light scene at night. By continuously training and optimizing the decision tree model, it can accurately classify new light intensity data and obtain the characteristics of the cabin lighting scene.

[0210] For example, we can classify lighting scenes based on light intensity data and set a light intensity range, such as g_low to g_mid for normal light scenes and g_mid to g_high for strong light scenes. If the current light intensity is g1, we determine the scene it belongs to. If it is within the strong light scene range, we set the scene identifier to ls1, forming the cabin lighting scene feature set [ls1, ...].

[0211] In step S122, noise spectrum analysis is performed on the environmental noise data in the state monitoring data set to extract noise interference features;

[0212] In the embodiment of the present disclosure, the environmental noise data is pre-processed, such as pre-emphasis processing (enhancing high-frequency components and improving spectral characteristics) to better perform spectral analysis. Then, a spectral analysis method such as fast Fourier transform (FFT) is used to perform noise spectrum analysis on the environmental noise data. FFT can convert the noise signal in the time domain into a spectrum diagram in the frequency domain, and determine the main frequency distribution of the noise signal by analyzing the amplitudes of different frequency components in the spectrum diagram. Then, according to the preset interference frequency threshold and amplitude threshold, the noise frequency range and corresponding amplitude information that may interfere with audio and video playback are extracted to generate noise interference features. For example, if it is found that the noise amplitude in a certain frequency range exceeds the threshold, and the frequency range is close to the key frequency of the audio played by the audio equipment, the frequency range and amplitude information are used as noise interference features.

[0213] For example, a noise sensor collects noise data, such as the noise intensity at frequency f3 is n1, and the noise intensity at frequency f4 is n2. The noise intensities at different frequencies are analyzed, and a noise intensity threshold n_threshold is set. If n1 is greater than n_threshold, the noise interference level at that frequency is set to dn1, forming a noise interference feature set [dn1, dn2, ...].

[0214] In step S123, gas component identification processing is performed on the air quality data in the state monitoring data set to generate cabin air state characteristics;

[0215] In the disclosed embodiment, the air quality data is pre-processed, such as data cleaning (removing invalid or erroneous data), data calibration (ensuring the accuracy of sensor data), and other operations. Then, gas sensor technology and pattern recognition algorithms are used to perform gas component identification processing. Different gas sensors have different response characteristics to specific gas components, and the combination of multiple sensors can achieve the detection of multiple gas components. During the identification process, the electrical signal output by the sensor is converted into a gas concentration value, and then a pattern recognition algorithm (such as a neural network algorithm) is used to classify and identify the gas concentration data. For example, a neural network model is trained, and the concentration data of different gas components is used as input, and the corresponding air quality status (such as fresh, general, polluted, etc.) is used as output, so that the model can automatically learn the mapping relationship between gas components and air quality status. In actual applications, the air quality data is input into the trained model, and the model will output the cabin air status characteristics.

[0216] For example, an air quality sensor detects air composition, such as oxygen concentration o1 and carbon dioxide concentration c1. The cabin air state is determined based on the gas composition and concentration. A normal oxygen concentration range of o_min to o_max and a normal carbon dioxide concentration range of c_min to c_max are set. If o1 is outside the normal oxygen concentration range, the air state is identified as as1, forming the cabin air state feature set [as1, ...].

[0217] In step S124 , the cabin lighting scene features, the noise interference features, and the cabin air state features are subjected to multi-dimensional fusion processing to generate the cabin environment adaptation features.

[0218] In this disclosed embodiment, different features are combined and evaluated based on predefined rules. For example, when the cabin lighting scene is dim at night and there is significant noise interference, soothing music with a high frequency content may be prioritized to reduce the impact of noise interference on the auditory experience. Through these multi-dimensional fusion processing methods, the three features are comprehensively analyzed to generate cabin environment adaptation features.

[0219] For example, the cabin lighting scene feature set [ls1, ...], the noise interference feature set [dn1, dn2, ...], and the cabin air state feature set [as1, ...] are fused. Each feature is assigned a weight: the lighting scene feature ls1 is weighted as w_ls, the noise interference feature dn1 is weighted as w_dn, and the air state feature as1 is weighted as w_as. Through weighted calculation, for example, the fused value of ls1 is ls1*w_ls, the fused value of dn1 is dn1*w_dn, and the fused value of as1 is as1*w_as. These weighted values ​​are concatenated to form the cabin environment adaptation feature set [ls1*w_ls, dn1*w_dn, as1*w_as, ...].

[0220] The embodiment of the present disclosure also provides a vehicle-mounted intelligent audio and video adaptation and playback device, which is applied to a bean curd sheet drying and baking oven. Figure 2 As shown, the device includes:

[0221] The present disclosure also provides a vehicle-mounted intelligent audio and video adaptation playback device, see Figure 2 As shown, the device includes:

[0222] The first generating module 210 is configured to perform a multi-level user playback demand feature extraction operation on the acquired user behavior state data to generate a user dynamic playback demand feature;

[0223] The second generating module 220 is configured to perform a multi-level environment playback requirement feature extraction operation on the acquired cockpit environment state data to generate a cockpit environment adaptation feature;

[0224] The third generating module 230 is configured to perform a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate a vehicle operation adaptation feature;

[0225] The determination and generation module 240 is configured to obtain multiple playback constraint features by performing a pairwise cross-feature cross-attention operation based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics, and generate a playback constraint condition set including a primary playback constraint condition and multiple secondary playback constraint conditions based on the primary constraint reward of a discrete action and the secondary constraint reward of a continuous action for the multiple playback constraint feature constraint rewards;

[0226] The playback control module 250 is configured to generate an adaptive playback strategy for the current playback scene based on the main playback constraint and the secondary playback constraint in the playback constraint set, adjust the playback parameter set of the in-vehicle audio and video playback device according to the adaptive playback strategy, and perform playback control operations based on the adjusted playback parameter set.

[0227] In a possible implementation, the playback control module 250 is configured to:

[0228] parsing the playback content priority rule in the adapted playback strategy, and filtering a target playback content list from a media resource library according to the playback content priority rule;

[0229] Analyze the playback volume adjustment rules in the adaptive playback strategy and dynamically adjust the initial volume parameters of the target playback content according to the safety factor of the vehicle driving scenario;

[0230] Analyze the screen brightness adjustment rules in the adaptive playback strategy and adjust the display brightness parameters according to the cabin lighting scene characteristics and the user's physiological state characteristics;

[0231] Analyze the noise reduction intensity control rules in the adaptive playback strategy and adjust the active noise reduction intensity parameters according to the noise interference characteristics and the user's physiological fatigue level;

[0232] A playback control instruction is generated based on the target playback content list, the initial volume parameter, the display brightness parameter, and the active noise reduction intensity parameter, and the playback control instruction is sent to the in-vehicle audio and video playback device to perform a real-time playback operation.

[0233] In a possible implementation, the playback control module 250 is configured to:

[0234] Obtain the media type preference weight and content theme preference label from the user's historical playback preference features;

[0235] Obtain real-time semantic keywords and content screening conditions from the user's current playback intention features;

[0236] extracting a first candidate content set from a media resource library according to the media type preference weight and the content theme preference tag;

[0237] performing content matching calculation on the first candidate content set according to the real-time semantic keywords and the content screening condition to obtain a second candidate content set;

[0238] The second candidate content set is sorted based on the playback content priority rule to generate a target playback content list.

[0239] In a possible implementation, the playback control module 250 is configured to:

[0240] Obtain the safety level threshold range corresponding to the vehicle driving scenario safety factor;

[0241] When it is detected that the safety factor of the vehicle driving scene is lower than the first safety threshold, the volume is automatically increased, and the volume increase amplitude is calculated according to the safety factor difference;

[0242] When it is detected that the safety factor of the vehicle driving scene is higher than the second safety threshold, the volume hold operation is triggered and the current volume parameter is maintained unchanged;

[0243] When it is detected that the safety factor of the vehicle driving scene is between the first safety threshold and the second safety threshold, a volume gradual adjustment operation is triggered, and the volume adjustment step size is dynamically adjusted according to the safety factor change rate;

[0244] An adjusted initial volume parameter is generated based on the volume boost amplitude, the volume adjustment step, and the current volume parameter.

[0245] In a possible implementation, the first generating module 210 is configured to:

[0246] Performing time series behavior pattern analysis on the user operation record data in the user behavior status data to obtain the user's historical playback preference characteristics;

[0247] Performing real-time semantic parsing on the user interaction response data in the user behavior status data to extract features of the user's current playback intention;

[0248] Performing physiological state fluctuation recognition processing on the user physiological monitoring data in the user behavior state data to generate a user physiological state feature;

[0249] The user's historical playback preference features, the user's current playback intention features, and the user's physiological state features are subjected to feature fusion processing to generate the user's dynamic playback demand features.

[0250] In a possible implementation, the third generating module 230 is configured to:

[0251] Performing driving scene classification processing on the vehicle speed change data in the vehicle running state data to obtain vehicle driving scene type characteristics;

[0252] performing vibration frequency analysis on vehicle vibration data in the vehicle operating status data to extract cabin vibration interference characteristics;

[0253] performing path semantic parsing on the navigation path data in the vehicle operation status data to generate navigation path association features;

[0254] The vehicle driving scene type feature, the cabin vibration interference feature and the navigation path association feature are dynamically associated with each other to generate the vehicle operation adaptation feature.

[0255] In a possible implementation, the second generating module 220 is configured to:

[0256] Performing lighting scene classification processing on the light intensity data in the state monitoring data set to obtain cabin lighting scene features;

[0257] Performing noise spectrum analysis on the environmental noise data in the state monitoring data set to extract noise interference characteristics;

[0258] performing gas component identification processing on the air quality data in the state monitoring data set to generate cabin air state characteristics;

[0259] The cabin lighting scene features, the noise interference features, and the cabin air state features are subjected to multi-dimensional fusion processing to generate the cabin environment adaptation features.

[0260] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in any one of the aforementioned embodiments when the program is executed by a processor.

[0261] The present disclosure also provides an electronic device, including:

[0262] a memory having a computer program stored thereon;

[0263] A processor is used to execute the computer program in the memory to implement the steps of the method in any one of the aforementioned embodiments.

[0264] Figure 3 The in-vehicle intelligent audio and video adaptation and playback device 100 shown includes: a processor 1001 and a memory 1003. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the in-vehicle intelligent audio and video adaptation and playback device 100 may further include a communication component 1004, which can be used for data interaction between the device 100 and other devices, such as data transmission and / or data reception. It should be noted that in actual scheduling, the communication component 1004 is not limited to one, and the structure of the in-vehicle intelligent audio and video adaptation and playback device 100 does not constitute a limitation on the embodiments of the present application.

[0265] The processor 1001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 1001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0266] The bus 1002 may include a path for transmitting information between the above components. The bus 1002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 1002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0267] The memory 1003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store program code and can be read by a computer, without limitation here.

[0268] The memory 1003 is used to store program codes for executing the embodiments of the present disclosure, and the execution is controlled by the processor 1001. The processor 1001 is used to execute the program codes stored in the memory 1003 to implement the steps shown in the embodiment of the above-mentioned vehicle-mounted intelligent audio and video adaptive playback method.

[0269] The embodiment of the present disclosure also provides a computer-readable storage medium, which stores program code. When the program code is executed by a processor, it can implement the steps and corresponding contents of the aforementioned embodiment of the in-vehicle intelligent audio and video adaptation playback method.

[0270] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, various changes, modifications, replacements and variations can be made to these embodiments, and these changes, modifications, replacements and variations all fall within the scope of protection of the present disclosure.

[0271] It should also be noted that the various specific technical features described in the above specific embodiments may be combined in any suitable manner, unless there is any contradiction, and these combinations shall also be considered as the contents disclosed in this disclosure. To avoid unnecessary repetition, this disclosure will not further describe various possible combinations. The technical scope of this application is not limited to the contents of the specification and must be determined based on the scope of the claims.

Claims

1. A vehicle-mounted intelligent audio and video adaptation and playback method, characterized in that: The method comprises: Perform multi-level user playback demand feature extraction operations on the acquired user behavior status data to generate user dynamic playback demand features; Perform multi-level environment playback requirement feature extraction operations on the obtained cockpit environment status data to generate cockpit environment adaptation features; Perform multi-level vehicle playback demand feature extraction operations on the acquired vehicle operation status data to generate vehicle operation adaptation features; Based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics, a plurality of playback constraint characteristics are obtained by performing a pairwise cross-feature cross-attention operation. Based on the primary constraint reward of a discrete action and the secondary constraint reward of a continuous action, a playback constraint condition set including a primary playback constraint condition and a plurality of secondary playback constraint conditions is generated for the plurality of playback constraint characteristic constraint rewards; According to the main playback constraint and the secondary playback constraint in the playback constraint set, an adaptive playback strategy for the current playback scene is generated, and the playback parameter set of the in-vehicle audio and video playback device is adjusted according to the adaptive playback strategy, and the playback control operation is performed based on the adjusted playback parameter set.

2. The in-vehicle intelligent audio and video adaptation playback method according to claim 1, characterized in that: The adjusting the playback parameter set of the vehicle-mounted audio and video playback device according to the adapted playback strategy, and performing the playback control operation based on the adjusted playback parameter set, includes: parsing the playback content priority rule in the adapted playback strategy, and filtering a target playback content list from a media resource library according to the playback content priority rule; Analyze the playback volume adjustment rules in the adaptive playback strategy and dynamically adjust the initial volume parameters of the target playback content according to the safety factor of the vehicle driving scenario; Analyze the screen brightness adjustment rules in the adaptive playback strategy and adjust the display brightness parameters according to the cabin lighting scene characteristics and the user's physiological state characteristics; Analyze the noise reduction intensity control rules in the adaptive playback strategy and adjust the active noise reduction intensity parameters according to the noise interference characteristics and the user's physiological fatigue level; A playback control instruction is generated based on the target playback content list, the initial volume parameter, the display brightness parameter, and the active noise reduction intensity parameter, and the playback control instruction is sent to the in-vehicle audio and video playback device to perform a real-time playback operation.

3. The vehicle-mounted intelligent audio and video adaptation and playback method according to claim 2, characterized in that: The step of parsing the playback content priority rule in the adapted playback strategy and filtering the target playback content list from the media resource library according to the playback content priority rule includes: Obtain the media type preference weight and content theme preference label from the user's historical playback preference features; Obtain real-time semantic keywords and content screening conditions from the user's current playback intention features; extracting a first candidate content set from a media resource library according to the media type preference weight and the content theme preference tag; performing content matching calculation on the first candidate content set according to the real-time semantic keywords and the content screening condition to obtain a second candidate content set; The second candidate content set is sorted based on the playback content priority rule to generate a target playback content list.

4. The vehicle-mounted intelligent audio and video adaptation and playback method according to claim 2, characterized in that: The step of analyzing the playback volume adjustment rules in the adaptive playback strategy and dynamically adjusting the initial volume parameters of the target playback content according to the safety factor of the vehicle driving scenario includes: Obtain the safety level threshold range corresponding to the vehicle driving scenario safety factor; When it is detected that the safety factor of the vehicle driving scene is lower than the first safety threshold, the volume is automatically increased, and the volume increase amplitude is calculated according to the safety factor difference; When it is detected that the safety factor of the vehicle driving scene is higher than the second safety threshold, the volume hold operation is triggered and the current volume parameter is maintained unchanged; When it is detected that the safety factor of the vehicle driving scene is between the first safety threshold and the second safety threshold, a volume gradual adjustment operation is triggered, and the volume adjustment step size is dynamically adjusted according to the safety factor change rate; An adjusted initial volume parameter is generated based on the volume boost amplitude, the volume adjustment step, and the current volume parameter.

5. The vehicle-mounted intelligent audio and video adaptation and playback method according to any one of claims 1 to 4, characterized in that: The multi-level user playback demand feature extraction operation is performed on the acquired user behavior status data to generate user dynamic playback demand features, including: Performing time series behavior pattern analysis on the user operation record data in the user behavior status data to obtain the user's historical playback preference characteristics; Performing real-time semantic parsing on the user interaction response data in the user behavior status data to extract features of the user's current playback intention; Performing physiological state fluctuation recognition processing on the user physiological monitoring data in the user behavior state data to generate a user physiological state feature; The user's historical playback preference features, the user's current playback intention features, and the user's physiological state features are subjected to feature fusion processing to generate the user's dynamic playback demand features.

6. The vehicle-mounted intelligent audio and video adaptation and playback method according to any one of claims 1 to 4, characterized in that: The step of performing a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate vehicle operation adaptation features further includes: Performing driving scene classification processing on the vehicle speed change data in the vehicle running state data to obtain vehicle driving scene type characteristics; performing vibration frequency analysis on vehicle vibration data in the vehicle operating status data to extract cabin vibration interference characteristics; performing path semantic parsing on the navigation path data in the vehicle operation status data to generate navigation path association features; The vehicle driving scene type feature, the cabin vibration interference feature and the navigation path association feature are dynamically associated with each other to generate the vehicle operation adaptation feature.

7. The vehicle-mounted intelligent audio and video adaptation and playback method according to any one of claims 1 to 4, characterized in that: The step of performing a multi-level environment playback requirement feature extraction operation on the acquired cabin environment status data to generate a cabin environment adaptation feature further includes: Performing lighting scene classification processing on the light intensity data in the state monitoring data set to obtain cabin lighting scene features; Performing noise spectrum analysis on the environmental noise data in the state monitoring data set to extract noise interference characteristics; performing gas component identification processing on the air quality data in the state monitoring data set to generate cabin air state characteristics; The cabin lighting scene features, the noise interference features, and the cabin air state features are subjected to multi-dimensional fusion processing to generate the cabin environment adaptation features.

8. An in-vehicle intelligent audio and video adaptation and playback device, characterized in that: The device comprises: The first generating module is configured to perform a multi-level user playback demand feature extraction operation on the acquired user behavior state data to generate a user dynamic playback demand feature; A second generating module is configured to perform a multi-level environment playback requirement feature extraction operation on the acquired cockpit environment state data to generate a cockpit environment adaptation feature; A third generating module is configured to perform a multi-level vehicle playback demand feature extraction operation on the acquired vehicle operation status data to generate a vehicle operation adaptation feature; a determination and generation module configured to obtain a plurality of playback constraint features by performing a pairwise cross-feature cross-attention operation based on the user's dynamic playback demand characteristics, the cabin environment adaptation characteristics, and the vehicle operation adaptation characteristics, and generate a playback constraint condition set including a primary playback constraint condition and a plurality of secondary playback constraint conditions for the plurality of playback constraint feature constraint rewards based on a primary constraint reward for discrete actions and a secondary constraint reward for continuous actions; The playback control module is configured to generate an adaptive playback strategy for the current playback scene based on the main playback constraint and the secondary playback constraint in the playback constraint set, adjust the playback parameter set of the in-vehicle audio and video playback device according to the adaptive playback strategy, and perform playback control operations based on the adjusted playback parameter set.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images