Integrated intelligent audio-video playing system

Abstract

The invention relates to the technical field of audio-video intelligent optimization, in particular to an integrated intelligent audio-video playing system, which comprises the following steps: collecting environmental data to obtain historical audio-video environmental data; performing prediction according to the historical audio and video environment data to obtain predicted audio and video environment data; acquiring the stored playable audio-video data, and identifying audio-video features of the playable audio-video data; according to the playing duration of the playable audio-video data and the predicted audio-video environment data, identifying to obtain playing environment data; identifying the playing environment data to obtain playing environment characteristics; performing identification according to the audio and video features and the playing environment features to obtain an audio and video recommendation score; recommending according to the audio and video recommendation score, and determining to-be-played audio and video data; and obtaining audio-video features and playing environment data of the to-be-played audio-video data, and performing audio-video playing optimization according to the playing environment data and the audio-video features. According to the invention, intelligent recommendation and optimization of audio and video are realized through the audio and video data and the ship motion data.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent audio and video optimization, and in particular to an integrated intelligent audio and video playback system. Background Art

[0002] With the rapid development of multimedia technology, the application demand for intelligent audio and video playback systems in mobile carriers is growing. However, in dynamic environments, especially in scenes affected by complex external conditions such as surface ships, the audio and video playback effect is affected by many factors such as ship movement and environmental changes.

[0003] Existing systems mostly rely on static environmental parameters to adjust audio and video, and lack the ability to predict dynamic environmental changes; they ignore the deep correlation between environmental parameters and audio and video content characteristics, and are unable to adapt to the dynamic changes of audio and video content and environmental conditions; in addition, during the navigation of the ship, the roll, pitch and heave motions caused by the navigation of the ship and the waves have significant time-varying characteristics. Traditional methods cannot predict the shaking trend in advance, resulting in a lag in the adjustment of picture stability.

[0004] To this end, an integrated intelligent audio and video playback system is proposed. Summary of the invention

[0005] The object of the present invention is to provide an integrated intelligent audio and video playback system, including collecting environmental data to obtain historical audio and video environment data; making predictions based on the historical audio and video environment data to obtain predicted audio and video environment data; acquiring stored playable audio and video data and identifying audio and video features of the playable audio and video data; identifying playback environment data based on the playback duration of the playable audio and video data and the predicted audio and video environment data; identifying the playback environment data to obtain playback environment features; obtaining an audio and video recommendation score based on the audio and video features and the playback environment features; making recommendations based on the audio and video recommendation score and determining the audio and video data to be played; acquiring the audio and video features and the playback environment data of the audio and video data to be played, and optimizing the audio and video playback based on the playback environment data and the audio and video features; and realizing intelligent recommendation and optimization of audio and video through audio and video data and ship motion data.

[0006] To achieve the above object, the present invention provides the following technical solutions: An integrated intelligent audio and video playback system, comprising: The audio-visual environment recognition module collects environment data to obtain historical audio-visual environment data; performs prediction based on the historical audio-visual environment data to obtain predicted audio-visual environment data; the predicted audio-visual environment data includes predicted brightness data, predicted sound data, and predicted shaking data; The audio and video playback recommendation module obtains the stored playable audio and video data and identifies the audio and video features of the playable audio and video data; the audio and video features include audio and video brightness features, audio and video sound features, and audio and video shaking features; identifies the playback environment data according to the playback time of the playable audio and video data and the predicted audio and video environment data; identifies the playback environment data to obtain the playback environment features; and identifies the audio and video recommendation score according to the audio and video features and the playback environment features; The audio and video playback optimization module makes recommendations based on the audio and video recommendation scores and determines the audio and video data to be played; obtains the audio and video features and playback environment data of the audio and video data to be played, and optimizes the audio and video playback based on the playback environment data and audio and video features.

[0007] The process of obtaining predicted audio-visual environment data according to the historical audio-visual environment data is as follows: Acquire navigation track data of a ship on the water surface, wherein the navigation track data includes navigation coordinate points, navigation speed and navigation time points; collect historical environment data of the area where the navigation track data passes as historical audio-visual environment data; A prediction is performed based on the historical audio-visual environment data, the navigation speed and the navigation time point to obtain the predicted environment data when the ship passes the navigation coordinate point as the predicted audio-visual environment data.

[0008] The playable audio and video data and the playback environment data are identified through the audio and video matching recognition model; the audio and video matching recognition model includes: a data feature recognition layer, a first matching recognition layer, a second matching recognition layer, a third matching recognition layer and a recommendation score calculation layer; The data feature recognition layer obtains audio and video features according to the playable audio and video data, including audio and video brightness features, audio and video sound features, and audio and video shaking features; and obtains playback environment features according to the playback environment data, including environment brightness features, environment sound features, and environment shaking features; The first match recognition layer performs recognition based on the audio and video brightness characteristics and the ambient brightness characteristics to obtain a brightness match score; The second match recognition layer performs recognition based on the audio and video sound features and the environmental sound features to obtain a sound match score; The third match recognition layer performs recognition based on the video and audio shaking characteristics and the environment shaking characteristics to obtain a stable match score; The recommendation score calculation layer calculates and obtains the audio and video recommendation score based on the brightness match score, the sound match score and the stability match score.

[0009] The formula for calculating the audio and video recommendation score based on the brightness match score, sound match score and stability match score is: ; ; in, Indicates the video recommendation rating; Indicates the brightness fit score; Indicates the sound fit score; represents the stable fit score; represents the first fit weight; represents the second fit weight; Denotes the third fit weight.

[0010] The audio and video playback optimization process includes dividing the playback time of the audio and video data to be played to obtain an optimization period; the process of obtaining the optimization period includes: Obtaining audio and video characteristics and playback environment data of the audio and video data to be played; The audio and video features are identified by a clustering algorithm, and cluster analysis is performed according to the audio and video brightness features, audio and video sound features, and audio and video shaking features to obtain the first period division data; The playback environment data is identified by a clustering algorithm, and cluster analysis is performed according to the predicted brightness data, the predicted sound data, and the predicted shaking data to obtain the second period division data; The playback duration is divided according to the first cycle division data and the second cycle division data to obtain an optimized cycle.

[0011] During the optimization cycle, playback optimization is performed based on playback environment data and audio and video characteristics, including brightness optimization, sound optimization, and stability optimization; The brightness optimization includes identifying brightness optimization data according to the predicted brightness data and the brightness characteristics of the audio and video in the playback environment data, and optimizing the brightness of the audio and video according to the brightness optimization data; The sound optimization includes identifying sound optimization data according to the predicted sound data and the sound characteristics of the audio and video in the playback environment data, and optimizing the sound of the audio and video according to the sound optimization data; The stable optimization includes identifying stable optimization data according to the predicted jitter data and the audio and video jitter characteristics in the playback environment data, and optimizing the audio and video picture according to the stable optimization data.

[0012] The acquisition of the stable optimization data includes: Identifying the predicted sloshing data to obtain predicted sloshing data features; the predicted sloshing data includes a roll parameter, a pitch parameter, and a heave parameter of the vessel; Detecting the in-screen motion vector of the audio and video data to be played, identifying the screen shaking and the frequency of change, and obtaining the audio and video shaking characteristics; The predicted sway data features and audio-visual sway features corresponding to the same optimization period are obtained, the sway motion of the ship and the sway motion of the audio-visual picture are superimposed, the motion direction and amplitude that need to be compensated are calculated, and stable optimization data are obtained.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention is based on a deep learning network to construct an audio-visual fit recognition model; the playable audio-visual data and the playback environment data are identified to obtain audio-visual features and playback environment features; a brightness fit score is obtained based on the audio-visual brightness features and the environment brightness features, a sound fit score is obtained based on the audio-visual sound features and the environment sound features, and a stability fit score is obtained based on the audio-visual shaking features and the environment shaking features; an audio-visual recommendation score is obtained by measuring the brightness fit score, the sound fit score and the stability fit score; thereby obtaining the degree of fit between the audio-visual data and the playback environment data.

[0014] 2. The present invention obtains audio-visual features and playback environment data of audio-visual data to be played; identifies the audio-visual features through a clustering algorithm to obtain first period division data; identifies the playback environment data through a clustering algorithm to obtain second period division data; divides the playback time according to the first period division data and the second period division data to obtain an optimization period; thereby ensuring that the audio-visual playback data and the playback environment data tend to be stable within each optimization period.

[0015] 3. The present invention identifies the predicted jitter data to obtain the predicted jitter data characteristics; detects the in-screen motion vector of the audio and video data to be played, identifies the picture jitter and change frequency, and obtains the audio and video jitter characteristics; obtains stable optimization data based on the predicted jitter data characteristics and the audio and video jitter characteristics within the same optimization cycle; thereby accurately compensating and optimizing the audio and video picture according to the ship jitter data and the audio and video picture jitter, thereby improving the picture playback effect of the audio and video in a complex water surface environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural schematic diagram of an integrated intelligent audio and video playback system of the present invention; Figure 2 It is a schematic diagram of the video recommendation process of the present invention; Figure 3 Schematic diagram of the structure of the audio-visual matching recognition model of the present invention; Figure 4 It is a schematic diagram of the audio and video optimization process of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Embodiment 1 An integrated intelligent audio and video playback system, the structure of which is as follows Figure 1 As shown, including: The audio-visual environment recognition module collects environmental data to obtain historical audio-visual environment data; makes predictions based on the historical audio-visual environment data to obtain predicted audio-visual environment data; the predicted audio-visual environment data includes predicted brightness data, predicted sound data and predicted shaking data.

[0019] The process of obtaining predicted audio-visual environment data according to the historical audio-visual environment data is as follows: Acquire navigation track data of a ship on the water surface, wherein the navigation track data includes navigation coordinate points, navigation speed and navigation time points; collect historical environment data of the area where the navigation track data passes as historical audio-visual environment data; A prediction is performed based on the historical audio-visual environment data, the navigation speed and the navigation time point to obtain the predicted environment data when the ship passes the navigation coordinate point as the predicted audio-visual environment data.

[0020] The present invention obtains the navigation track data of the ship, including the navigation coordinate points, navigation speed and navigation time points; collects the historical audio-visual environment data of the area through which the navigation track passes, and makes a prediction based on the historical audio-visual environment data, navigation speed and navigation time points to obtain the predicted environment data when the ship passes the navigation coordinate points; thereby accurately collecting the environmental data of the ship during navigation.

[0021] The audio and video playback recommendation module obtains the stored playable audio and video data and identifies the audio and video features of the playable audio and video data; the audio and video features include audio and video brightness features, audio and video sound features, and audio and video shaking features; according to the playback time of the playable audio and video data and the predicted audio and video environment data, the playback environment data is identified; the playback environment data is identified to obtain the playback environment features; and the audio and video recommendation score is obtained based on the audio and video features and the playback environment features. The process of audio and video recommendation is as follows: Figure 2 shown.

[0022] The playable audio and video data and the playback environment data are identified through the audio and video matching recognition model; the audio and video matching recognition model is constructed based on the deep learning network, and its structure is as follows: Figure 3As shown, it includes: a data feature recognition layer, a first match recognition layer, a second match recognition layer, a third match recognition layer and a recommendation score calculation layer; The data feature recognition layer obtains audio and video features according to the playable audio and video data, including audio and video brightness features, audio and video sound features, and audio and video shaking features; and obtains playback environment features according to the playback environment data, including environment brightness features, environment sound features, and environment shaking features; The first match recognition layer performs recognition based on the audio and video brightness characteristics and the ambient brightness characteristics to obtain a brightness match score; The second match recognition layer performs recognition based on the audio and video sound features and the environmental sound features to obtain a sound match score; The third match recognition layer performs recognition based on the video and audio shaking characteristics and the environment shaking characteristics to obtain a stable match score; The recommendation score calculation layer calculates and obtains the audio and video recommendation score based on the brightness match score, the sound match score and the stability match score.

[0023] The formula for calculating the audio and video recommendation score based on the brightness match score, sound match score and stability match score is: ; ; in, Indicates the video recommendation rating; Indicates the brightness fit score; Indicates the sound fit score; represents the stable fit score; represents the first fit weight; represents the second fit weight; Denotes the third fit weight.

[0024] The present invention is based on a deep learning network to construct an audio-visual fit recognition model; the playable audio-visual data and the playback environment data are identified to obtain audio-visual features and playback environment features; a brightness fit score is obtained based on the audio-visual brightness features and the environment brightness features, a sound fit score is obtained based on the audio-visual sound features and the environment sound features, and a stability fit score is obtained based on the audio-visual shaking features and the environment shaking features; an audio-visual recommendation score is obtained by measuring the brightness fit score, the sound fit score and the stability fit score; thereby obtaining the degree of fit between the audio-visual data and the water surface playback environment data.

[0025] The audio and video playback optimization module makes recommendations based on the audio and video recommendation scores and determines the audio and video data to be played; obtains the audio and video features and playback environment data of the audio and video data to be played, and optimizes the audio and video playback based on the playback environment data and audio and video features. Figure 4 shown.

[0026] The audio and video playback optimization process includes dividing the playback time of the audio and video data to be played to obtain an optimization period; the process of obtaining the optimization period includes: Obtaining audio and video characteristics and playback environment data of the audio and video data to be played; The audio and video features are identified by a clustering algorithm, and cluster analysis is performed according to the audio and video brightness features, audio and video sound features, and audio and video shaking features to obtain the first period division data; The playback environment data is identified by a clustering algorithm, and cluster analysis is performed according to the predicted brightness data, the predicted sound data, and the predicted shaking data to obtain the second period division data; The playback duration is divided according to the first cycle division data and the second cycle division data to obtain an optimized cycle.

[0027] The present invention obtains audio-visual features and playback environment data of audio-visual data to be played; identifies the audio-visual features through a clustering algorithm to obtain first period division data; identifies the playback environment data through a clustering algorithm to obtain second period division data; divides the playback time according to the first period division data and the second period division data to obtain an optimization period; thereby ensuring that the audio-visual playback data and the playback environment data tend to be stable within each optimization period.

[0028] During the optimization cycle, playback optimization is performed based on playback environment data and audio and video characteristics, including brightness optimization, sound optimization, and stability optimization; The brightness optimization includes identifying brightness optimization data according to the predicted brightness data and the brightness characteristics of the audio and video in the playback environment data, and optimizing the brightness of the audio and video according to the brightness optimization data; The sound optimization includes identifying sound optimization data according to the predicted sound data and the sound characteristics of the audio and video in the playback environment data, and optimizing the sound of the audio and video according to the sound optimization data; The stable optimization includes identifying stable optimization data according to the predicted jitter data and the audio and video jitter characteristics in the playback environment data, and optimizing the audio and video picture according to the stable optimization data.

[0029] The present invention identifies brightness optimization data based on predicted brightness data of playback environment data and audio and video brightness characteristics; identifies sound optimization data based on predicted sound data of playback environment data and audio and video sound characteristics; identifies stability optimization data based on predicted shaking data and audio and video shaking characteristics in the playback environment data; optimizes audio and video based on brightness optimization data, sound optimization data and stability optimization data, and effectively improves the playback effect of audio and video in a water surface environment.

[0030] The acquisition of the stable optimization data includes: Identifying the predicted sloshing data to obtain predicted sloshing data features; the predicted sloshing data includes a roll parameter, a pitch parameter, and a heave parameter of the vessel; Detecting the in-screen motion vector of the audio and video data to be played, identifying the screen shaking and the frequency of change, and obtaining the audio and video shaking characteristics; The predicted sway data features and audio-visual sway features corresponding to the same optimization period are obtained, the sway motion of the ship and the sway motion of the audio-visual picture are superimposed, the motion direction and amplitude that need to be compensated are calculated, and stable optimization data are obtained.

[0031] The present invention identifies predicted jitter data to obtain predicted jitter data features; detects the in-screen motion vector of the audio and video data to be played, identifies the screen jitter and change frequency, and obtains the audio and video jitter features; obtains stable optimization data based on the predicted jitter data features and the audio and video jitter features within the same optimization cycle; thereby accurately compensating and optimizing the audio and video picture based on the ship jitter data and the audio and video picture jitter, thereby improving the picture playback effect of the audio and video in a complex water surface environment.

[0032] The present invention collects environmental data to obtain historical audio-visual environment data; makes predictions based on the historical audio-visual environment data to obtain predicted audio-visual environment data; obtains stored playable audio-visual data and identifies audio-visual features of the playable audio-visual data; identifies playback environment data based on the playback duration of the playable audio-visual data and the predicted audio-visual environment data; identifies playback environment data to obtain playback environment features; obtains audio-visual recommendation scores based on the audio-visual features and playback environment features; makes recommendations based on the audio-visual recommendation scores and determines audio-visual data to be played; obtains audio-visual features and playback environment data of the audio-visual data to be played, optimizes audio-visual playback based on the playback environment data and audio-visual features, and realizes intelligent recommendation and optimization of audio-visual in a complex water surface environment.

[0033] Embodiment 2 An integrated intelligent audio and video playback system, the structure of which is as follows Figure 1 As shown, including: The audio-visual environment recognition module collects environmental data to obtain historical audio-visual environment data; makes predictions based on the historical audio-visual environment data to obtain predicted audio-visual environment data; the predicted audio-visual environment data includes predicted brightness data, predicted sound data and predicted shaking data.

[0034] The historical audio-visual environment data includes environment brightness data, environment sound data and environment shaking data; The environmental brightness data is the brightness parameter in the audio and video playback environment; The environmental sound data is the sound parameters in the audio and video playback environment, including the ship's own noise, water surface noise and air noise; specifically, the ship's own noise includes engine noise, gear noise and ship whistle, etc., the water surface noise includes water wave noise and collision noise between the ship and the water wave, and the air noise includes wind noise, seabird noise and other ship noise; The environmental sloshing data reflects the motion state of the vessel itself, including roll parameters, pitch parameters and heave parameters; wherein the roll parameters include the roll angle, the pitch parameters include the pitch angle, and the heave parameters include the heave displacement.

[0035] The roll angle, pitch angle and heave displacement of a ship are important parameters that describe the motion state of a ship during navigation; the roll angle refers to the angle at which the hull tilts left and right, which is mainly caused by side waves or wind, and its amplitude is affected by the ship's speed and structure; the pitch angle refers to the up and down movement of the bow and stern, which is related to the wave frequency and ship speed and affects the navigation performance; the heave displacement refers to the displacement of the ship in the vertical direction, which is caused by changes in waves and ship speed.

[0036] The process of obtaining predicted audio-visual environment data according to the historical audio-visual environment data is as follows: Acquire navigation track data of a ship on the water surface, wherein the navigation track data includes navigation coordinate points, navigation speed and navigation time points; collect historical environment data of the area where the navigation track data passes as historical audio-visual environment data; A prediction is performed based on the historical audio-visual environment data, the navigation speed and the navigation time point to obtain the predicted environment data when the ship passes the navigation coordinate point as the predicted audio-visual environment data.

[0037] In predicting audio-visual environment data, a dual prediction mechanism of LSTM and trajectory correction is adopted; The historical audio-visual environment data, navigation speed and navigation time point are predicted through the LSTM prediction model to obtain the first prediction data; then the error is corrected through the trajectory curvature and environmental fluctuations; the prediction formula is: ; in, Represents predicted audio-visual environment data; Represents the LSTM prediction model; Represents input data, including historical audio-visual environment data, navigation speed and navigation time point; represents the trajectory correction coefficient; represents the trajectory correction function, which is verified based on the collected data; represents the trajectory curvature; It represents the environmental fluctuation factor, reflects the fluctuation trend of the environment, and is determined based on historical data. The track curvature describes the curvature of the navigation path. It is used to quantify the severity of the ship's turning or path change in dynamic environment prediction, and directly affects the prediction accuracy of environmental data.

[0038] The present invention obtains the navigation track data of the ship, including the navigation coordinate points, navigation speed and navigation time points; collects the historical audio-visual environment data of the area through which the navigation track passes, and makes a prediction based on the historical audio-visual environment data, navigation speed and navigation time points to obtain the predicted environment data when the ship passes the navigation coordinate points; thereby accurately collecting the environmental data of the ship during navigation.

[0039] The audio and video playback recommendation module obtains the stored playable audio and video data and identifies the audio and video features of the playable audio and video data; the audio and video features include audio and video brightness features, audio and video sound features and audio and video shaking features; according to the playback time of the playable audio and video data and the predicted audio and video environment data, the playback environment data is identified; the playback environment data is identified to obtain the playback environment features; and the audio and video recommendation score is obtained based on the audio and video features and the playback environment features.

[0040] The playable audio and video data and the playback environment data are identified through the audio and video matching recognition model; the audio and video matching recognition model is constructed based on the deep learning network, and includes: data feature recognition layer, first matching recognition layer, second matching recognition layer, third matching recognition layer and recommendation score calculation layer; The data feature recognition layer obtains audio and video features according to the playable audio and video data, including audio and video brightness features, audio and video sound features, and audio and video shaking features; and obtains playback environment features according to the playback environment data, including environment brightness features, environment sound features, and environment shaking features; The first matching recognition layer performs recognition based on the audio and video brightness features and the ambient brightness features, obtains the expected ambient brightness based on the audio and video brightness features, and obtains the brightness matching score based on the similarity between the expected ambient brightness and the ambient brightness features; The second matching recognition layer performs recognition based on the audio-visual sound features and the environmental sound features, obtains the expected environmental sound based on the audio-visual sound features; and obtains the sound matching score based on the similarity between the expected environmental sound and the environmental sound features; The third matching recognition layer performs recognition based on the video and audio shaking characteristics and the environmental shaking characteristics, obtains the expected shaking data based on the video and audio shaking characteristics, and obtains the stable matching score based on the similarity between the expected shaking data and the environmental shaking characteristics; The recommendation score calculation layer calculates and obtains the audio and video recommendation score based on the brightness match score, the sound match score and the stability match score.

[0041] The formula for calculating the audio and video recommendation score based on the brightness match score, sound match score and stability match score is: ; ; in, Indicates the video recommendation rating; Indicates the brightness fit score; Indicates the sound fit score; represents the stable fit score; represents the first fit weight; represents the second fit weight; Denotes the third fit weight.

[0042] The playable audio and video data is identified to obtain data table 1.

[0043] Table 1 Audio-visual fit score calculation data table Can play audio and video data Brightness match score Sound fit score Stable fit score 1 88 76 81 2 69 72 90 3 83 92 74 4 92 87 86 5 79 83 81 The present invention is based on a deep learning network to construct an audio-visual fit recognition model; the playable audio-visual data and the playback environment data are identified to obtain audio-visual features and playback environment features; a brightness fit score is obtained based on the audio-visual brightness features and the environment brightness features, a sound fit score is obtained based on the audio-visual sound features and the environment sound features, and a stability fit score is obtained based on the audio-visual shaking features and the environment shaking features; an audio-visual recommendation score is obtained by measuring the brightness fit score, the sound fit score and the stability fit score; thereby obtaining the degree of fit between the audio-visual data and the water surface playback environment data.

[0044] The audio and video playback optimization module makes recommendations based on the audio and video recommendation scores and determines the audio and video data to be played; obtains the audio and video features and playback environment data of the audio and video data to be played, and optimizes the audio and video playback based on the playback environment data and audio and video features.

[0045] The audio and video playback optimization process includes dividing the playback time of the audio and video data to be played to obtain the optimization period; the process of obtaining the optimization period includes: obtaining the audio and video features and playback environment data of the audio and video data to be played; identifying the audio and video features through a clustering algorithm, and performing cluster analysis based on the audio and video brightness features, audio and video sound features and audio and video shaking features to obtain the first period division data; identifying the playback environment data through a clustering algorithm, and performing cluster analysis based on the predicted brightness data, predicted sound data and predicted shaking data to obtain the second period division data; dividing the playback time according to the first period division data and the second period division data to obtain the optimization period.

[0046] The present invention obtains audio-visual features and playback environment data of audio-visual data to be played; identifies the audio-visual features through a clustering algorithm to obtain first period division data; identifies the playback environment data through a clustering algorithm to obtain second period division data; divides the playback time according to the first period division data and the second period division data to obtain an optimization period; thereby ensuring that the audio-visual playback data and the playback environment data tend to be stable within each optimization period.

[0047] During the optimization cycle, playback optimization is performed based on playback environment data and audio and video characteristics, including brightness optimization, sound optimization, and stability optimization; The brightness optimization includes identifying brightness optimization data according to the predicted brightness data and the brightness characteristics of the audio and video in the playback environment data, and optimizing the brightness of the audio and video according to the brightness optimization data; The sound optimization includes identifying sound optimization data based on the predicted sound data and audio and video sound features in the playback environment data, and optimizing the sound of the audio and video based on the sound optimization data; the sound optimization also includes building an AI-driven air membrane audio system; the air membrane audio system uses an intelligent algorithm to perceive environmental noise in real time and adaptively adjust the noise reduction level, which is suitable for application scenarios in large spaces; its working principle is: based on the spatial learning of the air membrane, dynamic adjustment is made to achieve a combination of active noise reduction and passive noise reduction; at the same time, based on the AI ​​algorithm to learn the user's hearing curve and preferences, provide personalized noise reduction solutions, and can switch between noise reduction and ambient sound to provide the ultimate noise reduction effect.

[0048] The stable optimization includes identifying stable optimization data according to the predicted jitter data and the audio and video jitter characteristics in the playback environment data, and optimizing the audio and video picture according to the stable optimization data.

[0049] The present invention identifies brightness optimization data based on predicted brightness data of playback environment data and audio and video brightness characteristics; identifies sound optimization data based on predicted sound data of playback environment data and audio and video sound characteristics; identifies stability optimization data based on predicted shaking data and audio and video shaking characteristics in the playback environment data; optimizes audio and video based on brightness optimization data, sound optimization data and stability optimization data, and effectively improves the playback effect of audio and video in a water surface environment.

[0050] The acquisition of the stable optimization data includes: identifying the predicted shaking data to obtain the predicted shaking data characteristics; the predicted shaking data includes the roll parameter, pitch parameter and heave parameter of the ship; using the optical flow method to detect the motion vector within the screen of the audio and video data to be played, identifying the screen shaking and change frequency, and obtaining the audio and video shaking characteristics; obtaining the predicted shaking data characteristics and audio and video shaking characteristics corresponding to the same optimization period, superimposing the shaking motion of the ship and the shaking motion of the audio and video screen, calculating the movement direction and amplitude that need to be compensated, and obtaining stable optimization data.

[0051] The present invention identifies predicted jitter data to obtain predicted jitter data features; detects the in-screen motion vector of the audio and video data to be played, identifies the screen jitter and change frequency, and obtains the audio and video jitter features; obtains stable optimization data based on the predicted jitter data features and the audio and video jitter features within the same optimization cycle; thereby accurately compensating and optimizing the audio and video picture based on the ship jitter data and the audio and video picture jitter, thereby improving the picture playback effect of the audio and video in a complex water surface environment.

[0052] In order to verify the effect of the audio and video optimization of the present invention, a comparative optimization scheme was used for testing; the comparative optimization scheme was adjusted based on static environmental parameters, and the similarity coefficient between the actual audio and video playback data and the standard audio and video data collected at a certain distance was used as a measure of the optimization effect, and the data obtained were shown in Table 2.

[0053] Table 2 Audio and video optimization solution verification data table Comparison of optimization solutions This application optimization plan Improvement effect Audio and video brightness 0.78 0.85 8.97% Audio and Video 0.82 0.90 9.75% Video Screen 0.77 0.89 15.58% It can be seen from Table 2 that the brightness optimization, sound optimization and stability optimization of the optimization scheme of the present application are all better than those of the comparison optimization scheme.

[0054] The present invention collects environmental data to obtain historical audio-visual environment data; makes predictions based on the historical audio-visual environment data to obtain predicted audio-visual environment data; obtains stored playable audio-visual data and identifies audio-visual features of the playable audio-visual data; identifies playback environment data based on the playback duration of the playable audio-visual data and the predicted audio-visual environment data; identifies playback environment data to obtain playback environment features; obtains audio-visual recommendation scores based on the audio-visual features and playback environment features; makes recommendations based on the audio-visual recommendation scores and determines audio-visual data to be played; obtains audio-visual features and playback environment data of the audio-visual data to be played, optimizes audio-visual playback based on the playback environment data and audio-visual features, and realizes intelligent recommendation and optimization of audio-visual in a complex water surface environment.

[0055] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An integrated intelligent audio and video playback system, characterized in that: include: The audio-visual environment recognition module collects environmental data according to the navigation track data to obtain historical audio-visual environment data; performs prediction according to the historical audio-visual environment data to obtain predicted audio-visual environment data; the predicted audio-visual environment data includes predicted brightness data, predicted sound data and predicted shaking data; The audio and video playback recommendation module obtains the stored playable audio and video data and identifies the audio and video features of the playable audio and video data; the audio and video features include audio and video brightness features, audio and video sound features, and audio and video shaking features; identifies the playback environment data according to the playback time of the playable audio and video data and the predicted audio and video environment data; identifies the playback environment data to obtain the playback environment features; and identifies the audio and video recommendation score according to the audio and video features and the playback environment features; The audio and video playback optimization module makes recommendations based on the audio and video recommendation scores and determines the audio and video data to be played; obtains the audio and video features and playback environment data of the audio and video data to be played, and optimizes the audio and video playback based on the playback environment data and audio and video features.

2. The integrated intelligent audio and video playback system according to claim 1, characterized in that: The process of obtaining predicted audio-visual environment data according to the historical audio-visual environment data is as follows: Acquire navigation track data of a ship on the water surface, wherein the navigation track data includes navigation coordinate points, navigation speed and navigation time points; collect historical environment data of the area where the navigation track data passes as historical audio-visual environment data; A prediction is performed based on the historical audio-visual environment data, the navigation speed and the navigation time point to obtain the predicted environment data when the ship passes the navigation coordinate point as the predicted audio-visual environment data.

3. The integrated intelligent audio and video playback system according to claim 1, characterized in that: The playable audio and video data and the playback environment data are identified through the audio and video matching recognition model; the audio and video matching recognition model includes: a data feature recognition layer, a first matching recognition layer, a second matching recognition layer, a third matching recognition layer and a recommendation score calculation layer; The data feature recognition layer obtains audio and video features according to the playable audio and video data, including audio and video brightness features, audio and video sound features, and audio and video shaking features; and obtains playback environment features according to the playback environment data, including environment brightness features, environment sound features, and environment shaking features; The first match recognition layer performs recognition based on the audio and video brightness characteristics and the ambient brightness characteristics to obtain a brightness match score; The second match recognition layer performs recognition based on the audio and video sound features and the environmental sound features to obtain a sound match score; The third match recognition layer performs recognition based on the video and audio shaking characteristics and the environment shaking characteristics to obtain a stable match score; The recommendation score calculation layer calculates and obtains the audio and video recommendation score based on the brightness match score, the sound match score and the stability match score.

4. The integrated intelligent audio and video playback system according to claim 3, characterized in that: The formula for calculating the audio and video recommendation score based on the brightness match score, sound match score and stability match score is: ； ； in, Indicates the video recommendation rating; Indicates the brightness fit score; Indicates the sound fit score; represents the stable fit score; represents the first fit weight; represents the second fit weight; Denotes the third fit weight.

5. The integrated intelligent audio and video playback system according to claim 1, characterized in that: The audio and video playback optimization process includes dividing the playback time of the audio and video data to be played to obtain an optimization period; the process of obtaining the optimization period includes: Obtaining audio and video characteristics and playback environment data of the audio and video data to be played; The audio and video features are identified by a clustering algorithm, and cluster analysis is performed according to the audio and video brightness features, audio and video sound features, and audio and video shaking features to obtain the first period division data; The playback environment data is identified by a clustering algorithm, and cluster analysis is performed according to the predicted brightness data, the predicted sound data, and the predicted shaking data to obtain the second period division data; The playback duration is divided according to the first cycle division data and the second cycle division data to obtain an optimized cycle.

6. The integrated intelligent audio and video playback system according to claim 5, characterized in that: During the optimization cycle, playback optimization is performed based on playback environment data and audio and video characteristics, including brightness optimization, sound optimization, and stability optimization; The brightness optimization includes identifying brightness optimization data according to the predicted brightness data and the brightness characteristics of the audio and video in the playback environment data, and optimizing the brightness of the audio and video according to the brightness optimization data; The sound optimization includes identifying sound optimization data according to the predicted sound data and the sound characteristics of the audio and video in the playback environment data, and optimizing the sound of the audio and video according to the sound optimization data; The stable optimization includes identifying stable optimization data according to the predicted jitter data and the audio and video jitter characteristics in the playback environment data, and optimizing the audio and video picture according to the stable optimization data.

7. The integrated intelligent audio and video playback system according to claim 6, characterized in that: The acquisition of the stable optimization data includes: Identifying the predicted sloshing data to obtain predicted sloshing data features; the predicted sloshing data includes a roll parameter, a pitch parameter, and a heave parameter of the vessel; Detecting the in-screen motion vector of the audio and video data to be played, identifying the screen shaking and the frequency of change, and obtaining the audio and video shaking characteristics; The predicted sway data features and audio-visual sway features corresponding to the same optimization period are obtained, the sway motion of the ship and the sway motion of the audio-visual picture are superimposed, the motion direction and amplitude that need to be compensated are calculated, and stable optimization data are obtained.

Images