Screen projection frame rate dynamic adjustment method, device and system, vehicle and storage medium

Abstract

The invention relates to the technical field of ecological sharing, and discloses a screen projection frame rate dynamic adjustment method, device and system, a vehicle and a storage medium. First performance data of a current screen projection system and second performance data of a mobile terminal establishing screen projection connection with the screen projection system are acquired in real time, then a target screen projection frame rate is determined based on upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data, and finally a screen projection frame rate value of the screen projection system is adjusted based on the target screen projection frame rate. The stability and the video quality of the screen projection system are balanced, the software and hardware performance and the load condition of the system are considered when the screen projection frame rate is determined, heating of double-end equipment is reduced, and energy conservation and emission reduction are achieved while the use experience of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of ecological sharing technology, and specifically to a method, device, system, vehicle and storage medium for dynamically adjusting the screen projection frame rate. Background Art

[0002] In recent years, with the improvement of the performance of vehicle hardware and software, various ecological applications on vehicle computers have become increasingly abundant. These ecological applications have not only enriched the driving and riding experience of car users, but also promoted the intelligent development and innovation of the automobile industry. In the future, with the continuous advancement of technology and the continuous expansion of market demand, vehicle computer ecological applications will become more diversified and personalized, bringing car users a more colorful driving and riding experience.

[0003] Taking mobile phone screen projection as an example, mobile phone manufacturers have built-in dedicated mobile phone corresponding special display screen projection software and car machine development kit (Software Development Kit, SDK), and can apply their own mobile phone screen projection products to cars, but the third-party screen projection applications on the market also have some problems such as freezing and black screen, frequent disconnection, high latency and poor stability. The root cause of the above problems is not only the hardware specifications of the receiving end such as the car machine, high system load, high network load and other hardware and system reasons, but also the lack of corresponding software optimization is also an important reason. The unoptimized or insufficiently optimized solutions do not consider the scenarios with high load or consider them insufficiently. When the load is high, the large data volume and bandwidth seriously affect the user experience. At present, the more common optimization solution in the industry is to use a fixed frame rate solution to reduce the maximum transmission load of the picture, but there will be a certain waste of performance and resources in some low-frequency frame scenarios. For example, when the source data is a static picture, a fixed frame rate is still used, which will lead to performance power consumption, but the user experience is not significantly improved, resulting in a certain waste of power consumption. Summary of the invention

[0004] In view of this, the present invention provides a method, device, system, vehicle and storage medium for dynamically adjusting the projection frame rate to solve the problem of reducing the maximum transmission load of the picture by adopting a fixed frame rate solution. However, there will be certain performance and resource waste in some low-frequency frame scenarios, and there will be problems of poor projection stability such as freeze and black screen due to improper frame rate setting.

[0005] In a first aspect, the present invention provides a method for dynamically adjusting the projection frame rate, the method comprising: determining the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters; acquiring in real time the first performance data of the current projection system and the second performance data of a mobile terminal that establishes a projection connection with the projection system; determining the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data, and adjusting the projection frame rate value of the projection system based on the target projection frame rate.

[0006] The method for dynamically adjusting the projection frame rate provided by the present invention determines the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters, obtains the first performance data of the current projection system and the second performance data of the mobile terminal that establishes a projection connection with the projection system in real time, and then determines the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data. Finally, the projection frame rate value of the projection system is adjusted based on the target projection frame rate. This not only balances the stability and video quality of the projection system, but also takes into account the system's hardware and software performance and load conditions when determining the projection frame rate, thereby reducing the heat generation of dual-end devices, improving user experience while saving energy and reducing emissions.

[0007] In an optional embodiment, the first performance data and the second performance data each include at least processor load, memory usage and network bandwidth, and determining the target screen projection frame rate based on the upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data includes: selecting the maximum processor load value, the maximum memory usage value and the minimum network bandwidth in the first performance data and the second performance data as the performance data of the screen projection frame rate to be calculated; determining the target screen projection frame rate based on the performance data of the screen projection frame rate to be calculated and the upper and lower limit parameters of the screen projection frame rate.

[0008] The present invention is based on the "short board of the barrel" principle and selects performance data that meets the calculation frame rate requirement from the first performance data and the second performance data to calculate the screen projection frame rate, thereby ensuring the smoothness and quality of the screen projection frame rate.

[0009] In an optional embodiment, the target screen projection frame rate is determined based on the performance data of the screen projection frame rate to be calculated and the upper and lower limit parameters of the screen projection frame rate, including: based on different preset weights corresponding to different performance data, weighted summing the performance data of the screen projection frame rate to be calculated and its corresponding weights to obtain a sum value; calculating the maximum value of the processor load of the screen projection system and multiplying the weight sum corresponding to all performance data to obtain a product value; dividing the sum value and the product value to obtain a quotient, and calculating the product of the difference between the upper and lower limit parameters and the quotient; calculating the sum of the lower limit parameter and the product to obtain the target screen projection frame rate.

[0010] The present invention dynamically calculates the target screen projection frame rate based on performance data such as system load and network status, and adjusts the screen projection frame rate value of the screen projection system in real time based on the target screen projection frame rate to balance the stability and video quality of the screen projection system.

[0011] In an optional embodiment, before obtaining in real time the first performance data of the current screen projection system and the second performance data of the mobile terminal that establishes a screen projection connection with the screen projection system, the method also includes: obtaining the current screen projection application scenario type, or, in response to the screen projection application scenario type set by the user; based on the preset screen projection frame rate requirements corresponding to different application scenario types, setting the screen projection frame rate value of the screen projection system to the screen projection frame rate corresponding to the current screen projection application scenario type or the screen projection scene type set by the user.

[0012] The present invention can automatically switch to the optimal frequency setting according to different application scenario types, ensuring that it can also perform self-learning and adjustment according to the user's operating habits and preferences, thereby providing a more personalized experience.

[0013] In an optional embodiment, the method also includes: in response to the current screen projection application scenario switching from a first application scenario type to a second application scenario type; based on the preset screen projection frame rate requirements corresponding to different application scenario types, setting the screen projection frame rate value of the screen projection system to the screen projection frame rate corresponding to the second application scenario type.

[0014] When the present invention identifies that the screen projection application scenario has changed, it can automatically switch the screen projection frame rate of the changed application scenario type, thereby automatically optimizing the video quality and frame rate under different operations and application scenarios.

[0015] In an optional embodiment, the method also includes: in response to the screen projection system and the mobile terminal establishing a screen projection connection, obtaining device attribute data of the screen projection system and the mobile terminal respectively; based on a pre-integrated database of different protocol data that can be supported by different device attribute data, determining the supportable protocol data corresponding to the screen projection system and the mobile terminal respectively, and filtering out the protocol data set supported by both the screen projection system and the mobile terminal; based on the first performance data of the screen projection system and / or the second performance data of the mobile terminal, selecting the optimal protocol data from the protocol data set as the protocol for establishing a screen projection connection between the screen projection system and the mobile terminal.

[0016] The present invention supports various car computers and smart devices, including but not limited to mobile phones, tablets and other smart devices, breaking the brand and model restrictions. It can dynamically switch different protocol parameters according to the specification data supported by different models, so that it can be automatically compatible and used in a differentiated manner, solving the poor experience of some existing models that do not support protocol usage, reducing user troubles and user usage complexity, and enhancing user experience.

[0017] In an optional embodiment, the method also includes: during the screen projection connection between the projection system and the mobile terminal, monitoring the device attribute data and protocol compatibility of the projection system and the mobile terminal; if the device attribute data of the projection system and / or the mobile terminal changes, returning to execute the database of different protocol data supportable by different device attribute data based on the pre-integrated database, and determining the supportable protocol data corresponding to the projection system and the mobile terminal respectively.

[0018] If the present invention detects changes in the device attribute data of the screen projection system and / or the mobile terminal, it can re-screen the adapted protocol for screen projection transmission, and automatically adjust according to the supported specifications and parameters of different devices to achieve wide compatibility and simplify the user experience, ensuring stable connection and data transmission even if there are protocol differences between different devices.

[0019] In an optional embodiment, the method also includes: if there is a protocol incompatibility between the projection system and the mobile terminal, using a protocol conversion tool to convert the protocol format, or prompting the user with information that the protocol is incompatible, or returning to execute a database of different protocol data that can be supported by different device attribute data that is pre-integrated, to determine the steps of supporting the protocol data corresponding to the projection system and the mobile terminal respectively.

[0020] When the present invention detects that there is protocol incompatibility between the screen projection system and the mobile terminal, a protocol conversion tool can be used to perform format conversion to ensure the normal screen projection function. The supported protocol data can also be re-determined for application, reducing the user operation steps, solving the protocol compatibility problem between devices, and ensuring basic support for system functions.

[0021] In an optional embodiment, when the screen projection application scenario type of the mobile terminal and the screen projection system is an audio and video type, the method also includes: obtaining a first timestamp of the current audio frame and a second timestamp of the current video frame; calculating the time difference between the first timestamp and the second timestamp; determining a proportional coefficient of audio and video synchronization based on the time difference, the second performance data of the mobile terminal and the first performance data of the screen projection system, and calculating the product of the time difference and the proportional coefficient to obtain an adjustment amount; adjusting the timestamp of the audio frame or the video frame based on the adjustment amount.

[0022] The present invention calculates the difference between the timestamps of the audio frame and the video frame, multiplies it by the proportional system of the audio and video synchronization adjustment to obtain an adjustment amount, and adjusts the timestamp of the audio frame or the video frame based on the adjustment amount, thereby solving the problem of audio and video asynchrony, and solving the problem of complex solutions in the industry that are difficult to align audio and video timestamps by delaying audio or delaying video.

[0023] In an optional embodiment, before obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame, the method further includes: respectively obtaining the first time delay value and the second time delay value of the audio and video sent from the mobile terminal to the screen projection system; calculating the difference between the first time delay value and the second time delay value; if the difference is greater than a preset audio and video time delay threshold, executing the step of obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame.

[0024] The present invention calculates the delay difference of the time delay from sending to receiving of audio and video respectively. If the delay difference is greater than the required maximum allowable audio and video synchronization error, the timestamp of the audio frame or the video frame can be adjusted, thereby synchronously solving the problem of audio and video asynchrony, and ensuring audio and video synchronization while ensuring low delay and high fluency of data.

[0025] In an optional embodiment, adjusting the timestamp of an audio frame or a video frame based on the adjustment amount includes: comparing a first time delay value and a second time delay value; if the first time delay value is greater than the second time delay value, adjusting the timestamp of the video frame based on the adjustment amount; if the second time delay value is greater than the first time delay value, adjusting the timestamp of the audio frame based on the adjustment amount.

[0026] The embodiment of the present invention can adjust the time stamp of the audio frame or the video frame in each playback cycle or frame interval, so as to adjust the time stamp in real time and reduce the audio and video synchronization error.

[0027] In the second aspect, the present invention provides a device for dynamically adjusting the projection frame rate, the device comprising: an upper and lower limit calculation module, used to determine the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters; a performance data acquisition module, used to acquire in real time the first performance data of the current projection system and the second performance data of the mobile terminal that establishes a projection connection with the projection system; a projection frame rate adjustment module, used to determine the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data, and adjust the projection frame rate value of the projection system based on the target projection frame rate.

[0028] In a third aspect, the present invention provides a system for dynamically adjusting the projection frame rate, the system comprising a controller and a projection system, the controller comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the method for dynamically adjusting the projection frame rate of the first aspect or any corresponding embodiment thereof.

[0029] In a fourth aspect, the present invention provides a vehicle, comprising the projection frame rate dynamic adjustment system of the third aspect.

[0030] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for dynamically adjusting the projection frame rate of the above-mentioned first aspect or any corresponding embodiment thereof.

[0031] The method for dynamically adjusting the projection frame rate provided by the present invention determines the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters, obtains the first performance data of the current projection system and the second performance data of the mobile terminal that establishes a projection connection with the projection system in real time, and then determines the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data. Finally, the projection frame rate value of the projection system is adjusted based on the target projection frame rate. This not only balances the stability and video quality of the projection system, but also takes into account the system's hardware and software performance and load conditions when determining the projection frame rate, thereby reducing the heat generation of dual-end devices, improving user experience while saving energy and reducing emissions. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 is a schematic diagram of a flow chart of a method for dynamically adjusting a screen projection frame rate according to an embodiment of the present invention;

[0034] Figure 2 is a flowchart of another method for dynamically adjusting the screen projection frame rate according to an embodiment of the present invention;

[0035] Figure 3 is a structural block diagram of a system for dynamically adjusting screen projection frame rate according to an embodiment of the present invention;

[0036] Figure 4 is a structural example diagram of a system for dynamically adjusting the screen projection frame rate according to an embodiment of the present invention;

[0037] Figure 5 is an example diagram of the interaction between various modules in the system for dynamically adjusting the screen projection frame rate according to an embodiment of the present invention;

[0038] Figure 6 is a structural block diagram of a vehicle according to an embodiment of the present invention;

[0039] Figure 7 is a structural block diagram of a device for dynamically adjusting screen projection frame rate according to an embodiment of the present invention;

[0040] Figure 8 Schematic diagram of the hardware structure of the controller according to the embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0042] According to an embodiment of the present invention, an embodiment of a method for dynamically adjusting the projection frame rate is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] In this embodiment, a method for dynamically adjusting the projection frame rate is provided, which can be used for a controller in the above-mentioned projection frame rate dynamic adjustment system. Figure 1 is a flow chart of a method for dynamically adjusting the screen projection frame rate according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0044] Step S101, based on the hardware capabilities of the screen projection system and the screen projection setting parameters, determine the upper and lower limit parameters of the screen projection frame rate.

[0045] The embodiments of the present invention do not limit the application field of the screen projection system, which can be a home screen projection device, a car screen projection device, and other fields where screens can be projected. Taking the car screen projection as an example, when the screen projection system starts, the upper and lower limit parameters of the screen projection frame rate can be initialized according to the hardware capabilities, application requirements and screen projection configuration parameters of the screen projection system. The set upper limit of the screen projection frame rate ensures that the video is smooth when the screen projection system is under high load, and the lower limit frequency ensures the video quality under the lowest load. Among them, the screen projection configuration parameters may include but are not limited to resolution, bit rate and buffer size. By accurately setting the screen projection configuration parameters, the image quality and smoothness of the video can be ensured, and it should support multiple resolution settings, including 720p, 1080p and 4K, to adapt to different display devices, just as an example.

[0046] Step S102, obtaining in real time the first performance data of the current screen projection system and the second performance data of the mobile terminal that establishes a screen projection connection with the screen projection system.

[0047] The embodiments of the present invention can obtain in real time the performance data of the current screen projection system and the performance data of the mobile terminal that has established a screen projection connection with the screen projection system, where the mobile terminal includes but is not limited to smart hardware devices such as mobile phones and tablets, and can project the screen to the screen projection system via WIFI, Bluetooth, etc. The performance data is but not limited to processor load, memory usage, network traffic, system temperature, etc., just as examples.

[0048] Step S103, determine the target screen projection frame rate based on the upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data, and adjust the screen projection frame rate value of the screen projection system based on the target screen projection frame rate.

[0049] The embodiment of the present invention monitors the changes in the performance data of the current screen projection system and the performance data of the mobile terminal to determine the adjustment amount of the screen projection frame rate, that is, the current first performance data and the second performance data can be compared with the changes in the first performance data and the second performance data corresponding to the initial screen projection frame rate, and the initial frame rate is adjusted accordingly based on the change in the performance data. For example, when the current system load is high, the environmental interference is serious, or the screen projection video has a high frame rate, the corresponding screen projection frame rate can be reduced, and the screen projection frame rate value of the screen projection system is adjusted based on the reduced screen projection frame rate. For example, when the system load is low, the resolution of the car-end screen projection is switched, the dual-screen is switched, the recording screen is re-switched, and other situations are prone to black screen and jamming, the corresponding screen projection frame rate can be increased, and then the screen projection frame rate value of the screen projection system is adjusted based on the increased screen projection frame rate, so that the screen projection system can balance the system stability and video quality based on the adjusted screen projection frame rate value, wherein the screen projection frame rate can be reduced or increased accordingly according to the upper and lower limit parameters of the screen projection frame rate to ensure the video smoothness and quality of the screen projection system under the maximum load and the minimum load, for example only.

[0050] The method for dynamically adjusting the projection frame rate provided by the present invention determines the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters, obtains the first performance data of the current projection system and the second performance data of the mobile terminal that establishes a projection connection with the projection system in real time, and then determines the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data. Finally, the projection frame rate value of the projection system is adjusted based on the target projection frame rate. This not only balances the stability and video quality of the projection system, but also takes into account the system's hardware and software performance and load conditions when determining the projection frame rate, thereby reducing the heat generation of dual-end devices, improving user experience while saving energy and reducing emissions.

[0051] In this embodiment, a method for dynamically adjusting the projection frame rate is provided, which can be used for a controller in the above-mentioned projection frame rate dynamic adjustment system. Figure 2 is a flow chart of a method for dynamically adjusting the screen projection frame rate according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0052] Step S201: Determine the upper and lower limit parameters of the screen projection frame rate based on the hardware capabilities of the screen projection system and the screen projection setting parameters. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0053] Step S202, obtain the current screen projection application scenario type, or respond to the screen projection application scenario type set by the user.

[0054] Step S203, based on the preset projection frame rate requirements corresponding to different application scenario types, the projection frame rate value of the projection system is set to the projection frame rate corresponding to the current projection application scenario type or the projection scenario type set by the user.

[0055] The embodiments of the present invention can automatically adjust the frame rate setting according to different application scenario types. The scene adaptation mechanism supports intelligent scene recognition and preset adjustment, and can automatically switch to the optimal frame rate setting according to different application scenarios. Each scene type can set its corresponding screen projection frame rate requirement. For example, movie scenes usually require a higher screen projection frame rate (such as 30-60FPS) to ensure the smoothness and picture quality of video playback. Game scenes require a higher screen projection frame rate (such as 60-120FPS) to ensure the smoothness and response speed of the game. Video conferencing scenes require a medium screen projection frame rate (such as 15-30FPS), focusing on smooth images and stability. Users can also be allowed to define personalized frame rate settings and associate them with specific application scenarios to ensure that the screen projection system can automatically adjust the frame rate settings according to the user's personalized needs. The pre-configured screen projection frame rates corresponding to different application scenarios can be stored in the system's configuration file or database so that they can be quickly called after scene recognition. Users can also be supported to manually adjust and save these preset frame rate settings to meet personalized needs, just as an example.

[0056] An embodiment of the present invention can determine the application scenario type by detecting the currently running application or process. For example, through key attributes such as the package name, it can identify that the running application is a game application, and then switch to the game scene. It can also use image processing technology to analyze the characteristics of the displayed content, such as inferring the application scenario type based on dynamic pictures. It can also provide a user interface to enable users to select or adjust the default application scenario settings. For example, after determining the current projection application scenario type, the projection frame rate corresponding to the current projection application scenario type can be determined based on the stored projection frame rate requirements corresponding to different application scenario types, and the projection frame rate value can be set to the projection system. Subsequently, the target projection frame rate can be determined based on the performance data of the current projection system and the mobile terminal, and the projection frame rate value of the projection system can be adjusted based on the target projection frame rate.

[0057] The present invention can automatically switch to the optimal frequency setting according to different application scenario types, ensuring that it can also perform self-learning and adjustment according to the user's operating habits and preferences, thereby providing a more personalized experience.

[0058] In an optional embodiment, in response to the current screen projection application scenario switching from a first application scenario type to a second application scenario type; based on the preset screen projection frame rate requirements corresponding to different application scenario types, the screen projection frame rate value of the screen projection system is set to the screen projection frame rate corresponding to the second application scenario type.

[0059] The embodiments of the present invention can also monitor application status, content changes and user behavior in real time. When it is detected that the current projection application scenario is switched from the first application scenario type to the second application scenario type, the corresponding frame rate configuration can be automatically triggered. That is, based on the preset projection frame rate requirements corresponding to different application scenario types, the projection frame rate value of the projection system is set to the projection frame rate corresponding to the second application scenario type. For example, after recognizing that the user switches from watching a movie to playing a game, the projection frame rate setting in the game mode can be immediately applied. Subsequently, the target projection frame rate can be determined based on the performance data of the current projection system and the mobile terminal, and then the projection frame rate value of the projection system can be adjusted based on the target projection frame rate.

[0060] When the present invention identifies that the screen projection application scenario has changed, it can automatically switch the screen projection frame rate of the changed application scenario type, thereby automatically optimizing the video quality and frame rate under different operations and application scenarios.

[0061] Step S204, real-time acquisition of first performance data of the current projection system and second performance data of the mobile terminal that has established a projection connection with the projection system. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0062] Step S205, determine the target screen projection frame rate based on the upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data, and adjust the screen projection frame rate value of the screen projection system based on the target screen projection frame rate.

[0063] Specifically, the first performance data and the second performance data both include at least processor load, memory application and network bandwidth, and the maximum processor load, the maximum memory usage and the minimum network bandwidth in the first performance data and the second performance data are selected as the performance data of the screen projection frame rate to be calculated; based on the performance data of the screen projection frame rate to be calculated and the upper and lower limit parameters of the screen projection frame rate, the target screen projection frame rate is determined.

[0064] The embodiments of the present invention can monitor performance data such as processing load, network bandwidth, and memory application in real time on the mobile terminal and the screen projection system, and based on the "short board of the bucket" principle, select the maximum value of the processor load (CPU load), the maximum value of memory usage, and the minimum network bandwidth in the first performance data and the second performance data as the performance data of the screen projection frame rate to be calculated, and then calculate the projection frame rate based on the performance data of the screen projection frame rate to be calculated, and limit the calculated projection frame rate to be within the upper and lower limit parameters to ensure the smoothness and quality of the screen projection.

[0065] Specifically, based on the different preset weights corresponding to different performance data, the performance data of the projection frame rate to be calculated and its corresponding weights are weightedly summed to obtain the sum value; the maximum processor load of the projection system is calculated and multiplied by the sum of the weights corresponding to all performance data to obtain the product value; the sum value and the product value are divided to obtain the quotient, and the product of the difference between the upper and lower limit parameters and the quotient is calculated; the sum of the lower limit parameter and the product is calculated to obtain the target projection frame rate.

[0066] In the embodiment of the present invention, the target screen projection frame rate can be calculated based on the performance data of the screen projection frame rate to be calculated using the following formula:

[0067]

[0068] Among them, F current Indicates the target projection frame rate; F min Indicates the lower limit parameter of the projection frame rate; F max Indicates the upper limit parameter of the screen projection frame rate; L cpu Indicates the processor load in the performance data of the screen projection frame rate to be calculated; L memory Indicates the memory usage in the performance data of the screen projection frame rate to be calculated; L network Indicates the network bandwidth in the performance data of the screen projection frame rate to be calculated; W cpu Indicates the weight corresponding to the processor load; W memory Indicates the weight corresponding to memory usage; W network Indicates the weight corresponding to the network bandwidth; L max Indicates the maximum value of the processor load.

[0069] After the embodiment of the present invention calculates the target screen projection frame rate in real time based on the above formula, the screen projection frame rate configuration of the screen projection system can be adjusted based on the target screen projection frame rate. In order to further optimize the dynamic frame rate adjustment, an artificial intelligence algorithm can also be introduced to continuously improve the frame rate adjustment strategy through historical data and user feedback. The adaptive algorithm can learn the user's usage patterns and load changes, and gradually improve the accuracy and response speed of the frame rate adjustment.

[0070] The present invention dynamically calculates the target screen projection frame rate based on performance data such as system load and network status, and adjusts the screen projection frame rate value of the screen projection system in real time based on the target screen projection frame rate to balance the stability and video quality of the screen projection system.

[0071] In an optional implementation, the embodiment of the present invention can also track any problems that occur during the screen projection process, such as black screen, freezes and delays, and can also collect log data (such as frame rate, delay and load conditions). Through detailed log records, the performance of the screen projection system in different operating scenarios is recorded, covering all key indicators in the operation of the screen projection system, such as frame rate fluctuations, network delays and processor usage, etc. By analyzing the system performance through log records, potential problems are identified, and the system's frame rate adjustment algorithm is continuously optimized, the screen projection system is optimized to ensure a stable user experience. For example, the expected frame rate is x, and the frame rates in the log data recorded on both the mobile terminal and the screen projection system are x1, x2...xn, and then the deviation is calculated with x to determine whether the deviation value is greater than the preset deviation threshold. If the deviation value is greater than the deviation threshold, the algorithm for dynamically adjusting the frame rate can be further optimized.

[0072] In an optional embodiment, in response to the screen projection system and the mobile terminal establishing a screen projection connection, the device attribute data of the screen projection system and the mobile terminal are respectively obtained; based on a pre-integrated database of different protocol data that can be supported by different device attribute data, the supportable protocol data corresponding to the screen projection system and the mobile terminal are determined, and the protocol data set supported by both the screen projection system and the mobile terminal is screened out; based on the first performance data of the screen projection system and / or the second performance data of the mobile terminal, the optimal protocol data is selected from the protocol data set as the protocol for establishing the screen projection connection between the screen projection system and the mobile terminal.

[0073] The embodiments of the present invention support the compatibility of multiple protocol standards of different device ends, including international standard communication protocols and private dynamic negotiation protocols, such as Real Time Messaging Protocol (RTMP), Real Time Streaming Protocol (RTMP) and Streaming Media Network Transmission Protocol (HTTP LiveStreaming, HLS) and other protocols. A protocol support database covering mainstream car machines and smart devices can be established in advance, including protocol types and parameters supported by different devices. It supports dynamic query and update of protocol data supported by the device each time the device is connected to ensure the real-time and accuracy of the information. That is, when the mobile terminal is connected to the projection system, the Bluetooth, WIFI and other interfaces can be used to identify the device attribute data of the projection system and the mobile terminal respectively, wherein the device attribute data includes but is not limited to the model and operating system. Then, an algorithm can be used to match the device attribute data with a pre-integrated database of different protocol data that can be supported by different device attribute data. Determine the supportable protocol data corresponding to the projection system and mobile terminal respectively, and filter out the protocol data set supported by both the projection system and the mobile terminal. If the protocol data set includes multiple protocol data, the priority of each protocol data in the protocol data set can be set based on the first performance data of the projection system and / or the second performance data of the mobile terminal, such as considering factors such as the resolution, frame rate and bandwidth supported by the device, and automatically select the protocol with the highest compatibility or the most complete functions according to the matching results. The protocol file can be saved to the path corresponding to the projection system and the mobile terminal, and the settings of the projection system and the mobile terminal can be updated. It also supports encryption and secure transmission protocols to protect the security of user data.

[0074] The present invention supports various car computers and smart devices, including but not limited to mobile phones, tablets and other smart devices, breaking the brand and model restrictions. It can dynamically switch different protocol parameters according to the specification data supported by different models, so that it can be automatically compatible and used in a differentiated manner, solving the poor experience of some existing models that do not support protocol usage, reducing user troubles and user usage complexity, and enhancing user experience.

[0075] In an optional embodiment, during the screen projection connection between the projection system and the mobile terminal, the device attribute data and protocol compatibility of the projection system and the mobile terminal are monitored; if the device attribute data of the projection system and / or the mobile terminal changes, return to execute a database of different protocol data that can be supported by different device attribute data based on a pre-integrated database, and determine the steps of the supportable protocol data corresponding to the projection system and the mobile terminal respectively.

[0076] The embodiments of the present invention can continuously monitor the device attribute data, protocol connection status and protocol compatibility of the screen projection system and the mobile terminal, and detect changes in the device status of the screen projection system and / or the mobile terminal. For example, when the mobile terminal has upgraded the system, resulting in the current transmission protocol no longer being used or having limited functions, the protocol switching process is triggered to start. Based on the pre-integrated database of different protocol data that can be supported by different device attribute data, the protocols supported by both the screen projection system and the mobile terminal can be re-screened, and the protocol with the highest compatibility or the most complete functions can be automatically selected to update the settings of the screen projection system and the mobile terminal.

[0077] If the present invention detects changes in the device attribute data of the screen projection system and / or the mobile terminal, it can re-screen the adapted protocol for screen projection transmission, and automatically adjust according to the supported specifications and parameters of different devices to achieve wide compatibility and simplify the user experience, ensuring stable connection and data transmission even if there are protocol differences between different devices.

[0078] In an optional implementation, if there is protocol incompatibility between the projection system and the mobile terminal, a protocol conversion tool is used to convert the protocol format, or the user is prompted with information that the protocol is incompatible, or the execution returns to a database of different protocol data that can be supported by different device attribute data that is pre-integrated to determine the steps for the supportable protocol data corresponding to the projection system and the mobile terminal respectively.

[0079] If the embodiment of the present invention detects that the protocols between the screen projection system and the mobile terminal are not directly compatible, a protocol conversion tool can be used to convert the protocol data format to ensure the normal screen projection function. An automatic recovery mechanism can also be provided. When the protocol switching fails or a protocol compatibility problem occurs, the user is automatically prompted with information that the protocol is incompatible, or the database of different protocol data that can be supported by different device attribute data that is pre-integrated is used to re-screen the protocols supported by both the screen projection system and the mobile terminal, automatically select the protocol with the highest compatibility or the most comprehensive functions, and update the settings of the screen projection system and the mobile terminal. This is just an example.

[0080] When the present invention detects that there is protocol incompatibility between the screen projection system and the mobile terminal, a protocol conversion tool can be used to perform format conversion to ensure the normal screen projection function. The supported protocol data can also be re-determined for application, reducing the user operation steps, solving the protocol compatibility problem between devices, and ensuring basic support for system functions.

[0081] In an optional embodiment, when the projection application scenario type of the mobile terminal and the projection system is an audio and video type, obtain the first timestamp of the current audio frame and the second timestamp of the current video frame; calculate the time difference between the first timestamp and the second timestamp; determine the proportional coefficient of audio and video synchronization based on the time difference, the second performance data of the mobile terminal and the first performance data of the projection system, and calculate the product of the time difference and the proportional coefficient to obtain the adjustment amount; adjust the timestamp of the audio frame or the video frame based on the adjustment amount.

[0082] In the embodiment of the present invention, when the screen projection application scenario type of the mobile terminal and the screen projection system is an audio and video type, the timestamp T of the current audio frame can be obtained. audio,frame and the timestamp T of the current video frame video,frame , and the time difference Δ between the first timestamp and the second timestamp synchronization can be calculated sync =|T audio,frame -T video,frame |, and the adjustment factor α can be defined as the proportional coefficient of the audio and video synchronization adjustment, ranging from 0 to 1, wherein the proportional coefficient of the audio and video synchronization adjustment is dynamically adjusted according to the current audio and video time difference. If the error is small, the corresponding proportional system can also be small, such as 0.1 or 0.2, to avoid over-adjustment. If the error is large, the corresponding proportional coefficient can also be large to quickly restore synchronization. The proportional coefficient of the audio and video synchronization adjustment can also be adjusted according to factors such as network conditions, bandwidth, device performance and load. For example, if the network delay is large or unstable, the corresponding proportional coefficient is large; if the device load is high, the corresponding proportional coefficient is also large. This is just an example.

[0083] The embodiment of the present invention can calculate the product of the time difference between the first timestamp and the second timestamp synchronization and the proportional coefficient of the audio and video synchronization adjustment to obtain the adjustment amount Δ adjust =α·Δ sync , and then the timestamp of the audio frame or video frame can be adjusted based on the adjustment amount, such as adjusting the timestamp T of the audio frame audio,adjust =T audio,frame -Δ adjust , just as an example.

[0084] In a specific embodiment, the current audio frame timestamp is 1000 ms, and the video frame timestamp is 980 ms.

[0085] Calculate the delay error: Δ sync =|1000-980|=20ms;

[0086] Set adjustment factor α = 0.5: Δ adjust =0.5×20=10ms;

[0087] Adjust video timestamp: T video,adjust=980+10=990ms;

[0088] Adjusted synchronization error: Δ sync,new =1000-990=10ms; the ultimate goal is to make the synchronization error close to the target delay range, and ensure that the audio and video playback is as synchronized as possible through real-time adjustment.

[0089] The present invention calculates the difference between the timestamps of the audio frame and the video frame, multiplies it by the proportional system of the audio and video synchronization adjustment to obtain an adjustment amount, and adjusts the timestamp of the audio frame or the video frame based on the adjustment amount, thereby solving the problem of audio and video asynchrony, and solving the problem of complex solutions in the industry that are difficult to align audio and video timestamps by delaying audio or delaying video.

[0090] In an optional embodiment, before obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame, the first time delay value and the second time delay value of the audio and video sent from the mobile terminal to the screen projection system are respectively obtained; the difference between the first time delay value and the second time delay value is calculated; if the difference is greater than a preset audio and video time delay threshold, the step of obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame is executed.

[0091] The embodiment of the present invention can obtain the first time delay D of the audio from the mobile terminal to the screen projection system. audio , get the second time delay D of the video from the mobile terminal to the screen projection system video , and then the difference between the first time delay value and the second time delay value Δ can be calculated sync =|D audio -D video |, judge Δ sync Is it greater than the preset audio and video time delay threshold T, where the audio and video time delay threshold is the maximum allowable audio and video synchronization error required by the system. If Δ sync If it is greater than the preset audio and video time delay threshold, the time difference between the first timestamp and the second timestamp synchronization can be calculated, and then the timestamp adjustment amount can be determined based on the product of the audio and video adjustment proportional coefficients. Finally, the timestamp of the audio frame or video frame is adjusted to ensure that the synchronization error is minimized.

[0092] The present invention calculates the delay difference of the time delay from sending to receiving of audio and video respectively. If the delay difference is greater than the required maximum allowable audio and video synchronization error, the timestamp of the audio frame or the video frame can be adjusted, thereby synchronously solving the problem of audio and video asynchrony, and ensuring audio and video synchronization while ensuring low delay and high fluency of data.

[0093] In an optional embodiment, adjusting the timestamp of an audio frame or a video frame based on an adjustment amount includes: comparing a first time delay value and a second time delay value; if the first time delay value is greater than the second time delay value, adjusting the timestamp of the video frame based on the adjustment amount; if the second time delay value is greater than the first time delay value, adjusting the timestamp of the audio frame based on the adjustment amount.

[0094] After calculating the adjustment amount, the embodiment of the present invention can compare the first time delay value and the second time delay value. If the time delay corresponding to the audio is larger, the video frame delay amount is reduced by T. video,adjust =T video,frame +Δ adjust If the time delay of the video is large, the audio frame delay can be reduced by T audio,adjust =T audio,frame -Δ adjust , to achieve synchronization of audio and video.

[0095] The embodiment of the present invention can adjust the time stamp of the audio frame or the video frame in each playback cycle or frame interval, so as to adjust the time stamp in real time and reduce the audio and video synchronization error.

[0096] In this embodiment, a system for dynamically adjusting the screen projection frame rate is also provided. Figure 3 As shown, the system includes a controller for executing the method for dynamically adjusting the screen projection frame rate described in the above embodiment.

[0097] Specifically, Figure 4 As shown, the projection frame rate dynamic adjustment system may include a system configuration module, a load monitoring module, a frame rate adjustment algorithm module, a protocol adaptation module, a synchronization control module and a performance testing and optimization module. Figure 5As shown, the load monitoring module is deployed in the projection system and the mobile terminal respectively, and is used to collect the performance data of both ends in real time, including but not limited to processor load, memory usage, network traffic and system temperature. When the projection starts, the system configuration module will initialize the upper and lower limit parameters of the projection frame rate. The system configuration module also has a dynamic adjustment function. The load monitoring module monitors the performance data of both ends in real time and dynamically adjusts the frame rate parameters to adapt to different device specifications. The frame rate adjustment algorithm module is integrated with a dynamic frame rate calculation algorithm, which can dynamically adjust the frame rate according to the real-time performance data of both ends. It is necessary to analyze the system load and network conditions, and make real-time adjustments to balance system stability and video quality. For example, the frame rate is reduced when the load is high and increased when the load is low. The frame rate adjustment algorithm module also integrates a scene adaptation mechanism, which can automatically adjust the frame rate settings according to different usage scenarios. The scene adaptation mechanism supports intelligent scene recognition and preset adjustments, such as automatically switching to the optimal frame based on different usage scenarios (such as watching movies, games or video conferencing). In addition, it can also self-learn and adjust according to the user's operating habits and preferences to provide a more personalized experience. The frame rate adjustment algorithm should be able to automatically trigger the corresponding frame rate configuration based on changes in application scenarios; the protocol adaptation module is responsible for dynamically adapting the protocol to support the compatibility of different devices. The protocol adaptation module should support the compatibility of multiple protocol standards, such as RTSP, RTMP and HLS. This module supports flexible protocol conversion functions and automatically adjusts protocol parameters according to the specifications supported by the device to ensure seamless compatibility between various mainstream devices; at the same time, the protocol adaptation module should also support encryption and secure transmission protocols to protect the security of user data. It can also automatically switch to a compatible protocol when it detects that the current protocol is no longer in use or the device function is limited, and update the settings of the projection system and mobile terminal; the synchronization control module designs an audio and video synchronization control algorithm to ensure that the delay of video and audio is minimized. This module also includes a delay optimization mechanism to solve the problem of audio and video asynchrony by adjusting the timestamps of audio and video. The delay optimization function also includes data cache optimization, processing delay reduction and network delay compensation. By improving the data cache strategy and processing flow, the delay in audio and video transmission and processing can be reduced. At the same time, the projection system should have an intelligent network delay compensation mechanism to automatically adjust the data transmission time to maintain synchronization; the performance testing and optimization module is responsible for comprehensive performance testing, including system performance under different loads, network environments and device configurations. The test records detailed data, including video fluency, synchronization and system stability, to provide a basis for subsequent optimization. According to the test results, the optimization module adjusts the dynamic calculation algorithm, adaptation protocol and synchronization control mechanism to solve the problems found in the test. Through repeated testing and adjustment, the optimization module ensures that the system remains stable and efficient in various actual usage scenarios and improves the user experience. Please refer to the above embodiment for detailed description, which will not be repeated here.

[0098] This embodiment also provides a vehicle, such as Figure 6 As shown, the vehicle includes the above-mentioned screen projection frame rate dynamic adjustment system.

[0099] In this embodiment, a device for dynamically adjusting the projection frame rate is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0100] This embodiment provides a device for dynamically adjusting the projection frame rate. Figure 7 As shown, it includes: an upper and lower limit calculation module 701, which is used to determine the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters; a performance data acquisition module 702, which is used to acquire in real time the first performance data of the current projection system and the second performance data of the mobile terminal that establishes a projection connection with the projection system; a projection frame rate adjustment module 703, which is used to determine the target projection frame rate based on the upper and lower limit parameters of the projection frame rate, the first performance data and the second performance data, and adjust the projection frame rate value of the projection system based on the target projection frame rate.

[0101] In some optional embodiments, the first performance data and the second performance data both include at least processor load, memory usage and network bandwidth, and the screen projection frame rate adjustment module 703 includes: a performance data selection unit, used to select the maximum processor load, the maximum memory usage and the minimum network bandwidth in the first performance data and the second performance data as the performance data of the screen projection frame rate to be calculated; a screen projection frame rate calculation unit, used to determine the target screen projection frame rate based on the performance data of the screen projection frame rate to be calculated and the upper and lower limit parameters of the screen projection frame rate.

[0102] In some optional embodiments, the projection frame rate calculation unit includes: a weighted summation subunit, which is used to perform weighted summation of the performance data of the projection frame rate to be calculated and its corresponding weights based on different preset performance data corresponding to different weights, to obtain a sum value; a product calculation subunit, which is used to calculate the maximum value of the processor load of the projection system and multiply it by the sum of the weights corresponding to all performance data to obtain a product value; a quotient calculation subunit, which is used to divide the sum value and the product value to obtain a quotient, and calculate the product of the difference between the upper and lower limit parameters and the quotient; a target projection frame rate calculation subunit, which is used to calculate the sum of the lower limit parameter and the product to obtain a target projection frame rate.

[0103] In some optional embodiments, before obtaining in real time the first performance data of the current screen projection system and the second performance data of the mobile terminal that establishes a screen projection connection with the screen projection system, the screen projection frame rate dynamic adjustment device also includes: a scene type acquisition module, used to obtain the current screen projection application scene type, or, in response to the screen projection application scene type set by the user; a screen projection frame rate setting module, used to set the screen projection frame rate value of the screen projection system to the projection frame rate corresponding to the current screen projection application scene type or the screen projection scene type set by the user based on the preset projection frame rate requirements corresponding to different application scene types.

[0104] In some optional embodiments, the screen projection frame rate dynamic adjustment device also includes: a scene type switching module, which is used to respond to the current screen projection application scene switching from a first application scene type to a second application scene type; a frame rate setting module, which is used to set the screen projection frame rate value of the screen projection system to the projection frame rate corresponding to the second application scene type based on the preset screen projection frame rate requirements corresponding to different application scene types.

[0105] In some optional embodiments, the screen projection frame rate dynamic adjustment device also includes: a device attribute acquisition module, which is used to obtain the device attribute data of the screen projection system and the mobile terminal respectively in response to the screen projection system and the mobile terminal establishing a screen projection connection; a protocol matching module, which is used to determine the supportable protocol data corresponding to the screen projection system and the mobile terminal respectively based on a pre-integrated database of different protocol data that can be supported by different device attribute data, and filter out a set of protocol data supported by both the screen projection system and the mobile terminal; a protocol screening module, which is used to select the optimal protocol data from the protocol data set as the protocol for establishing a screen projection connection between the screen projection system and the mobile terminal based on the first performance data of the screen projection system and / or the second performance data of the mobile terminal.

[0106] In some optional embodiments, the screen projection frame rate dynamic adjustment device also includes: a protocol monitoring module, which is used to monitor the device attribute data and protocol compatibility of the screen projection system and the mobile terminal during the screen projection connection between the screen projection system and the mobile terminal; a protocol switching module, which is used to return to execute a database of different protocol data that can be supported by different device attribute data based on a pre-integrated database, and determine the supportable protocol data corresponding to the screen projection system and the mobile terminal respectively if the device attribute data of the screen projection system and / or the mobile terminal changes.

[0107] In some optional embodiments, the screen projection frame rate dynamic adjustment device also includes: a protocol switching module, which is used to use a protocol conversion tool to convert the protocol format if there is protocol incompatibility between the screen projection system and the mobile terminal, or to prompt the user with information that the protocol is incompatible, or to return to execute a database of different protocol data that can be supported by different device attribute data based on a pre-integrated database to determine the steps of supporting protocol data corresponding to the screen projection system and the mobile terminal respectively.

[0108] In some optional embodiments, when the screen projection application scenario type of the mobile terminal and the screen projection system is an audio and video type, the screen projection frame rate dynamic adjustment device also includes: a timestamp acquisition module, used to obtain the first timestamp of the current audio frame and the second timestamp of the current video frame; a timestamp difference calculation module, used to calculate the time difference between the first timestamp and the second timestamp; an adjustment amount calculation module, used to determine the proportional coefficient of audio and video synchronization based on the time difference, the second performance data of the mobile terminal and the first performance data of the screen projection system, and calculate the product of the time difference and the proportional coefficient to obtain the adjustment amount; a timestamp adjustment module, used to adjust the timestamp of the audio frame or the video frame based on the adjustment amount.

[0109] In some optional embodiments, before obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame, the screen projection frame rate dynamic adjustment device also includes: a time delay acquisition module, used to respectively obtain the first time delay value and the second time delay value of the audio and video sent from the mobile terminal to the screen projection system; a delay difference calculation module, used to calculate the difference between the first time delay value and the second time delay value; a timestamp adjustment execution module, used to execute the step of obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame if the difference is greater than a preset audio and video time delay threshold.

[0110] In some optional embodiments, the timestamp adjustment execution module includes: a time delay size comparison unit, used to compare the size of the first time delay value and the second time delay value; a video frame adjustment unit, used to adjust the timestamp of the video frame based on the adjustment amount if the first time delay value is greater than the second time delay value; an audio frame adjustment unit, used to adjust the timestamp of the audio frame based on the adjustment amount if the second time delay value is greater than the first time delay value.

[0111] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0112] The projection frame rate dynamic adjustment device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0113] The embodiment of the present invention also provides a controller having the above Figure 7 The screen projection frame rate dynamic adjustment device shown.

[0114] See also Figure 8 , Figure 8is a schematic diagram of the structure of a controller provided by an optional embodiment of the present invention, such as Figure 8 As shown, the controller includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the controller, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple controllers can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0115] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0116] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0117] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the controller, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the controller via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0119] The controller also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 8 The example of connecting through bus is taken in the following.

[0120] The input device 30 can receive input digital or character information and generate key signal input related to the user settings and function control of the controller, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0121] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0122] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the defined scope.

Claims

1. A method for dynamically adjusting screen projection frame rate, characterized in that: The method comprises: Based on the hardware capabilities of the projection system and the projection setting parameters, determine the upper and lower limit parameters of the projection frame rate; Acquire in real time first performance data of the current screen projection system and second performance data of the mobile terminal that has established a screen projection connection with the screen projection system; Based on the upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data, the target screen projection frame rate is determined, and the screen projection frame rate value of the screen projection system is adjusted based on the target screen projection frame rate.

2. The method according to claim 1, characterized in that The first performance data and the second performance data both include at least processor load, memory usage, and network bandwidth, and determining the target screen projection frame rate based on the upper limit and lower limit parameters of the screen projection frame rate, the first performance data, and the second performance data includes: Selecting the maximum value of the processor load, the maximum value of the memory usage, and the minimum network bandwidth from the first performance data and the second performance data as the performance data of the screen projection frame rate to be calculated; Based on the performance data of the screen projection frame rate to be calculated and the upper and lower limit parameters of the screen projection frame rate, the target screen projection frame rate is determined.

3. The method according to claim 2, characterized in that The determining a target screen projection frame rate based on the performance data of the screen projection frame rate to be calculated and the upper limit and lower limit parameters of the screen projection frame rate includes: Based on different preset weights corresponding to different performance data, weighted sum is performed on the performance data of the screen projection frame rate to be calculated and its corresponding weight to obtain a sum value; Calculate the maximum value of the processor load of the projection system and the weight sum corresponding to all performance data, and multiply them to obtain a product value; Dividing the sum value and the product value to obtain a quotient, and calculating the product of the difference between the upper limit parameter and the lower limit parameter and the quotient; The sum of the lower limit parameter and the product is calculated to obtain the target projection frame rate.

4. The method according to claim 1, characterized in that: Before obtaining in real time the first performance data of the current screen projection system and the second performance data of the mobile terminal that establishes a screen projection connection with the screen projection system, the method further includes: Obtain the current screen projection application scenario type, or respond to the screen projection application scenario type set by the user; Based on the preset screen projection frame rate requirements corresponding to different application scenario types, the screen projection frame rate value of the screen projection system is set to the screen projection frame rate corresponding to the current screen projection application scenario type or the screen projection scene type set by the user.

5. The method according to claim 4, characterized in that The method further comprises: In response to the current projection application scene switching from the first application scene type to the second application scene type; Based on the preset screen projection frame rate requirements corresponding to different application scenario types, the screen projection frame rate value of the screen projection system is set to the screen projection frame rate corresponding to the second application scenario type.

6. The method according to claim 1, characterized in that The method further comprises: In response to the screen projection system and the mobile terminal establishing a screen projection connection, obtaining device attribute data of the screen projection system and the mobile terminal respectively; Based on the pre-integrated database of different protocol data that can be supported by different device attribute data, determine the supportable protocol data corresponding to the projection system and the mobile terminal respectively, and filter out the protocol data set supported by both the projection system and the mobile terminal; Based on the first performance data of the screen projection system and / or the second performance data of the mobile terminal, the optimal protocol data is selected from the protocol data set as the protocol for establishing a screen projection connection between the screen projection system and the mobile terminal.

7. The method according to claim 6, characterized in that The method further comprises: During the screen projection connection between the screen projection system and the mobile terminal, monitor the device attribute data and protocol compatibility of the screen projection system and the mobile terminal; If the device attribute data of the projection system and / or the mobile terminal changes, return to execute the step of determining the supportable protocol data corresponding to the projection system and the mobile terminal respectively based on the database of different protocol data that can be supported by the pre-integrated different device attribute data.

8. The method according to claim 7, characterized in that The method further comprises: If there is protocol incompatibility between the projection system and the mobile terminal, use a protocol conversion tool to convert the protocol format, or prompt the user with information that the protocol is incompatible, or return to execute a database of different protocol data that can be supported by different device attribute data that is pre-integrated, to determine the steps for the supportable protocol data corresponding to the projection system and the mobile terminal respectively.

9. The method according to claim 1, characterized in that: When the screen projection application scenario type of the mobile terminal and the screen projection system is an audio and video type, the method further includes: Get the first timestamp of the current audio frame and the second timestamp of the current video frame; Calculate the time difference between the first timestamp and the second timestamp; Based on the time difference, the second performance data of the mobile terminal and the first performance data of the screen projection system, determine the proportional coefficient of the audio and video synchronization, and calculate the product of the time difference and the proportional coefficient to obtain an adjustment amount; The time stamp of the audio frame or the video frame is adjusted based on the adjustment amount.

10. The method according to claim 9, characterized in that Before obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame, the method further includes: Respectively obtain a first time delay value and a second time delay value of audio and video sent from the mobile terminal to the screen projection system; Calculating a difference between the first time delay value and the second time delay value; If the difference is greater than a preset audio and video time delay threshold, the step of obtaining the first timestamp of the current audio frame and the second timestamp of the current video frame is performed.

11. The method according to claim 10, characterized in that The adjusting the timestamp of the audio frame or the video frame based on the adjustment amount comprises: Compare the first time delay value and the second time delay value; If the first time delay value is greater than the second time delay value, adjusting the timestamp of the video frame based on the adjustment amount; If the second time delay value is greater than the first time delay value, the timestamp of the audio frame is adjusted based on the adjustment amount.

12. A device for dynamically adjusting screen projection frame rate, characterized in that: The device comprises: The upper and lower limit calculation module is used to determine the upper and lower limit parameters of the projection frame rate based on the hardware capabilities of the projection system and the projection setting parameters; A performance data acquisition module, used to acquire in real time first performance data of the current screen projection system and second performance data of the mobile terminal that has established a screen projection connection with the screen projection system; The screen projection frame rate adjustment module is used to determine the target screen projection frame rate based on the upper and lower limit parameters of the screen projection frame rate, the first performance data and the second performance data, and adjust the screen projection frame rate value of the screen projection system based on the target screen projection frame rate.

13. A screen projection frame rate dynamic adjustment system, characterized in that: The system includes a controller and a screen projection system, the controller includes a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the screen projection frame rate dynamic adjustment method described in any one of claims 1 to 11 by executing the computer instructions.

14. A vehicle, characterized in that: The vehicle includes the screen projection frame rate dynamic adjustment system described in claim 13.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for dynamically adjusting the screen projection frame rate according to any one of claims 1 to 11.

Images