An interactive method based on screen projection assistance

Abstract

This invention provides an interactive method based on screen projection assistance. The method includes: Step 1: After the requesting end receives assistance information sent by the assisting end, it maintains the current navigation path and launches a new route setting interface as the target user interface; Step 2: The requesting end projects the target user interface onto the assisting end; Step 3: The requesting end sends an assistance period to the assisting end and divides the assistance period into a preset number of sub-periods according to a first preset rule; Step 4: The requesting end determines whether it has received a new navigation route sent by the assisting end within each sub-period. If yes, it proceeds to the next step; otherwise, it performs corresponding operations according to a second preset rule; Step 5: The requesting end starts a navigation route switching thread to switch the current navigation route to the new navigation route. This invention realizes interaction based on screen projection assistance, allowing the co-pilot to assist in setting navigation, thus improving driving safety.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to an interaction method based on screen projection assistance. Background Technology

[0002] With the development of automotive electronics technology, car cockpits are now moving towards intelligence, and smart cockpits are becoming a trend. Smart cockpits feature multiple displays, such as instrument panel screens, central control screens, passenger-side screens, head-up displays (HUDs), and rear headrest screens, enriching the in-car user experience.

[0003] However, the current smart cockpit lacks interaction between its multiple screens. For example, the front passenger cannot help the driver set navigation or change routes through the front passenger screen, resulting in a poor user experience and insufficient intelligence. Summary of the Invention

[0004] This invention provides an interactive method based on screen projection assistance, aiming to overcome the deficiencies in the prior art, realize interactive methods based on screen projection assistance, and enable the co-pilot to assist in setting navigation, thereby improving driving safety.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] Step 1: After the requesting end receives the assistance information sent by the assisting end, it maintains the current navigation path and launches the new route setting interface as the target user interface.

[0007] Step 2: The requesting end projects the target user interface onto the assisting end;

[0008] Step 3: The requesting end sends an assistance period to the assisting end, and divides the assistance period into a preset number of sub-periods according to a first preset rule;

[0009] Step 4: The requesting end determines whether it has received a new navigation route from the assisting end within each sub-period. If yes, proceed to the next step; otherwise, perform the corresponding operation according to the second preset rule.

[0010] Step 5: The requesting end starts a navigation path switching thread to switch the current navigation route to the new navigation route.

[0011] Specifically, step 2 includes:

[0012] Step 201: The requesting end encodes the target shared interface according to the preset video encoding format and sends it to the frame buffer;

[0013] Step 202: The requesting end reads the encoded data from the frame buffer and generates a data packet to send to the assisting end through a preset transmission protocol;

[0014] Step 203: The assisting end receives the data packet, decodes and displays it, and simultaneously calculates the current packet loss rate of the data packet and feeds it back to the requesting end at a preset period.

[0015] Step 204: The requesting end adjusts the sending speed of the data packets according to the current packet loss rate.

[0016] Specifically, the preset transmission protocol is the RTP protocol.

[0017] Specifically, step 204 includes:

[0018] Step 2041: Obtain network latency parameters and calculate the current network jitter J according to the first preset formula;

[0019] Step 2042: Calculate the comprehensive parameter z according to the current packet loss rate L and the current network jitter J using the second preset formula, and set a first judgment threshold T for the comprehensive parameter z. z1 Second judgment threshold T z2 The first judgment threshold T z1 The second threshold, T, represents the threshold at which network congestion occurs. z2 A threshold indicating that the network has experienced congestion;

[0020] Step 2043: Determine whether the comprehensive parameter z does not exceed the first judgment threshold T. z1 If yes, proceed to the next step; otherwise, further determine whether the comprehensive parameter z does not exceed the second judgment threshold T. z2 If so, the data packet transmission rate is adjusted according to the first speed formula, and the current transmission rate S is taken as the transmission rate S during the most recent congestion. c Otherwise, adjust the data packet sending speed according to the second speed formula, and use the current sending speed S as the sending speed S during the most recent congestion. c .

[0021] Furthermore, after step 2043, the following is also included:

[0022] Step 2044: Set the third judgment threshold T c1 Fourth judgment threshold T c2 The third judgment threshold T c1 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The first ratio and the threshold of no network congestion, the fourth judgment threshold T c2 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The second ratio and the threshold of no network congestion, the third judgment threshold T c1 Much greater than the fourth judgment threshold Tc2 ;

[0023] Step 2045, determine whether 1 - S / S c <T c1 holds. If so, proceed to the next step; otherwise, adjust the transmission speed according to the third speed formula;

[0024] Step 2046, determine whether 1 - S / S c <T c2 holds. If so, proceed to the next step; otherwise, adjust the transmission speed according to the fourth speed formula;

[0025] Step 2047, determine whether S c <S holds. If so, adjust the transmission speed according to the fifth speed formula; otherwise, keep the current transmission speed unchanged.

[0026] Specifically, the first preset formula is: J = J p +(|D - D p |- J p ) / A, where D represents the current network delay time, D[[FINAL=1]] p represents the network delay time at the previous moment, J p represents the network jitter at the previous moment, and A represents the gain coefficient.

[0027] Specifically, the second preset formula is: z = α1 * L / L max +α2 * J / J max , where L max represents the maximum allowable packet loss rate during transmission, J max represents the maximum allowable delay jitter during transmission, α1 and α2 represent the proportions of packet loss rate and delay jitter, and α1 + α2 = 1.

[0028] Specifically, the first speed formula is: S n =(1 - 2z) z *S, and the second speed formula is: S n =k1 * S, where S n represents the transmission speed at the next moment, S represents the current transmission speed, and k1 represents the over - congestion coefficient, which is a constant.

[0029] Specifically, the third speed formula is: S n =S + S0, where S0 represents the transmission speed increase constant;

[0030] The fourth speed formula is:

[0031]

[0032] Where e represents a natural number, W represents the maximum network bandwidth, and S represents the current sending speed.

[0033] Specifically, the fifth velocity formula is: S n =S+(T z1 -z)*S c .

[0034] Specifically, performing the corresponding operation according to the second preset rule includes:

[0035] Step 401: The requesting end monitors whether a new navigation route has been received before the end of the first sub-term. If so, the countdown ends; otherwise, proceed to the next step.

[0036] Step 402: The requesting end monitors whether a new navigation route has been received before the end of the second sub-term. If so, the countdown ends; otherwise, the requesting end sends a first preset warning message to the assisting end and prompts the driver to slow down.

[0037] Step 403: The requesting terminal screen monitors whether a new navigation route has been received in the first half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a second preset warning message is sent to the assisting terminal, prompting the driver to drive at the minimum speed allowed by the current road.

[0038] Step 404: The requesting terminal monitors whether a new navigation route is received in the second half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a third preset warning message is sent to the assisting terminal, prompting the driver to stop in the emergency lane.

[0039] The beneficial effects of this invention are as follows: This invention projects the target user interface to the assisting end through the requesting end and sends an assistance period to the assisting end. If the requesting end receives a new navigation route sent by the assisting end within the assistance period, the requesting end switches the current navigation route to the new navigation route. Otherwise, it performs the corresponding operation according to the preset rules, realizing interaction based on screen projection assistance. The co-pilot can assist in setting the navigation, improving driving safety. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the interactive method based on screen projection assistance according to the present invention. Detailed Implementation

[0041] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The drawings are for reference and illustration only and do not constitute a limitation on the scope of protection of the present invention.

[0042] In the process described in the specification, claims, or drawings of this invention, each step is numbered (e.g., step 10, 20, etc.). These numbers are used only to distinguish the steps and do not represent any execution order. It should be noted that the terms "first," "second," etc., used herein are only for distinguishing the objects being described and do not represent a chronological order, nor do they indicate that "first," "second," etc., are different types.

[0043] like Figure 1 As shown, this embodiment provides an interaction method based on screen projection assistance, applied to a smart cockpit. The smart cockpit has at least two touch screens. The screen that initiates the assistance request is called the requesting end, and the screen that responds to and executes the assistance request is called the assisting end, including:

[0044] Step 1: After the requesting end receives the assistance information sent by the assisting end, it maintains the current navigation path and launches the new route setting interface as the target user interface.

[0045] In practice, if the driver wants the front passenger to assist in setting or changing the navigation route, the front passenger can click the "Assist" icon on the front passenger screen (assistance device), and then click the "Switch Navigation Route Mode" icon to activate the assistance mode. Of course, this is just one way to activate the assistance mode on the assistance device; other methods can also be used, such as voice commands or gesture commands, and this invention does not limit this method.

[0046] In practice, after receiving the assistance information containing "switch navigation path" sent by the assisting end (passenger screen), the requesting end (central control screen) keeps the navigation program on the current navigation path and generates a new route setting interface by starting a new route setting thread and places it below the current navigation path interface.

[0047] Step 2: The requesting end projects the target user interface onto the assisting end.

[0048] In practice, the requesting end and the assisting end can be connected by wired or wireless means, with the preferred method being the use of in-vehicle Ethernet connection.

[0049] In this embodiment, step 2 includes:

[0050] Step 201: The requesting end encodes the target shared interface according to the preset video encoding format and sends it to the frame buffer.

[0051] In specific implementation, the preset video encoding format can be H.264 encoding, or other encoding formats such as H.265 that can achieve video encoding transmission.

[0052] Step 202: The requesting end reads the encoded data from the frame buffer and generates a data packet to send to the assisting end through a preset transmission protocol.

[0053] In this embodiment, the preset transmission protocol is the RTP protocol.

[0054] Step 203: The assisting end receives the data packet, decodes and displays it, and simultaneously calculates the current packet loss rate of the data packet and feeds it back to the requesting end at a preset period.

[0055] Step 204: The requesting end adjusts the sending speed of the data packets according to the current packet loss rate.

[0056] In this embodiment, step 204 includes:

[0057] Step 2041: Obtain network latency parameters and calculate the current network jitter J according to the first preset formula.

[0058] In this embodiment, the first preset formula is: J=J p +(|DD p |-J p ) / A, where D represents the current network latency, D p J represents the network latency at the previous moment. p This indicates the network jitter at the previous moment, and A represents the gain coefficient, which can be set according to the actual transmission effect.

[0059] Step 2042: Calculate the comprehensive parameter z according to the current packet loss rate L and the current network jitter J using the second preset formula, and set a first judgment threshold T for the comprehensive parameter z. z1 Second judgment threshold T z2 The first judgment threshold T z1 The second threshold, T, represents the threshold at which network congestion occurs. z2 This indicates a threshold indicating that the network has experienced congestion.

[0060] In this embodiment, the second preset formula is: z = α1 * L / L max +α2*J / J max , where L max J represents the maximum allowable packet loss rate during transmission. max This represents the maximum allowable delay jitter during transmission, where α1 and α2 represent the proportions of packet loss rate and delay jitter, and α1+α2=1.

[0061] Generally speaking, packet loss rate is a relatively important parameter for measuring network transmission quality, so α1 is usually greater than α2. The specific value can be set by the transmission effect.

[0062] The higher the judgment threshold of the comprehensive parameter z, the lower the user's requirements for the network. The first judgment threshold T... z1 Second judgment threshold T z2 It can be configured according to the user's network requirements. This is easy to understand; T... z2 >T z1 For example, T can be set z1 =0.15, T z2 =0.20.

[0063] Step 2043: Determine whether the comprehensive parameter z does not exceed the first judgment threshold T. z1 If yes, proceed to the next step; otherwise, further determine whether the comprehensive parameter z does not exceed the second judgment threshold T. z2 If so, the data packet transmission rate is adjusted according to the first speed formula, and the current transmission rate S is taken as the transmission rate S during the most recent congestion. c Otherwise, adjust the data packet sending speed according to the second speed formula, and use the current sending speed S as the sending speed S during the most recent congestion. c .

[0064] In this embodiment, the first speed formula is: S n =(1-2z) z *S, the second velocity formula is: S n =k1*S, where S n Let S represent the transmission speed at the next moment, S represent the current transmission speed, and k1 represent the congestion coefficient, which is a constant.

[0065] In another embodiment of the invention, the method further includes the following step after step 2043:

[0066] Step 2044: Set the third judgment threshold T c1 Fourth judgment threshold T c2 The third judgment threshold T c1 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The fourth judgment threshold T is the first ratio (e.g., 90%) and the threshold of no network congestion. c2 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The third judgment threshold T is the second ratio (e.g., 99%) and the threshold of no network congestion. c1 Much greater than the fourth judgment threshold T c2 .

[0067] In this embodiment, the third judgment threshold T c1 =1 - First ratio; The fourth judgment threshold T c2= 1 - Second ratio.

[0068] For example, if the first ratio is 90%, then T c1 = 0.1, indicating that the current transmission speed has reached 90% of the transmission speed S during the most recent congestion and there is no network congestion; c

[0069] If the second ratio is 99%, then T c2 = 0.01, indicating that the current transmission speed has reached 99% of the transmission speed S during the most recent congestion and there is no network congestion. c

[0070] Step 2045, determine whether 1 - S / S c < T c1 holds. If so, proceed to the next step; otherwise, adjust the transmission speed according to the third speed formula.

[0071] In this embodiment, the third speed formula is: S n = S + S0, where S0 represents the transmission speed increase constant. For example, S0 = 100k.

[0072] Step 2046, determine whether 1 - S / S c < T c2 holds. If so, proceed to the next step; otherwise, adjust the transmission speed according to the fourth speed formula.

[0073] In this embodiment, the fourth speed formula is:

[0074]

[0075] where e represents the natural number, W represents the network maximum bandwidth, and S represents the current transmission speed.

[0076] Step 2047, determine whether S c < S holds. If so, adjust the transmission speed according to the fifth speed formula; otherwise, keep the current transmission speed unchanged.

[0077] In this embodiment, the fifth speed formula is: S n = S + (T z1 - z) * S <0000​​​​​​In this embodiment, the assistance period is calculated by the requesting end by obtaining the distance to the first intersection ahead on the current navigation path and based on the distance and the current vehicle speed. The assistance period is displayed on the assisting end in a countdown manner.

[0080] For example, if the assistance period is 60 seconds, it can be set into three sub-periods on an average basis: the first segment is from 60 to 40 seconds, the second segment is from 39 to 20 seconds, and the third segment is from 19 to 0 seconds. Of course, the duration of each sub-period can also be set according to other rules.

[0081] Step 4: The requesting end determines whether it has received a new navigation route from the assisting end within each sub-period. If yes, proceed to the next step; otherwise, perform the corresponding operation according to the second preset rule.

[0082] This embodiment uses the example of dividing the assistance period into three sub-periods to illustrate the execution of the corresponding operation according to the second preset rule:

[0083] Step 401: The requesting end monitors whether a new navigation route has been received before the end of the first sub-term. If so, the countdown ends; otherwise, proceed to the next step.

[0084] Step 402: The requesting end monitors whether a new navigation route has been received before the end of the second sub-term. If so, the countdown ends; otherwise, it sends a first preset warning message to the assisting end and prompts the driver to slow down.

[0085] In practice, the first preset warning message can be set as needed, for example: "You will pass through the intersection ahead in XXX seconds. Please complete the new route setting as soon as possible!", and displayed with a yellow background.

[0086] Step 403: The requesting terminal monitors whether a new navigation route has been received in the first half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a second preset warning message is sent to the assisting terminal, prompting the driver to drive at the minimum speed allowed by the current road.

[0087] In practice, the second preset warning message can be set as needed, for example: "You will pass through the intersection ahead in XXX seconds. Please complete the new route setting as soon as possible!", and displayed with an orange background.

[0088] Step 404: The requesting terminal monitors whether a new navigation route is received in the second half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a third preset warning message is sent to the assisting terminal, prompting the driver to stop in the emergency lane.

[0089] In practice, the third preset warning message can be set as needed, for example: "You will pass through the intersection ahead in XXX seconds. Please complete the new route setting as soon as possible!", and displayed with a red background.

[0090] Step 5: The requesting end starts a navigation path switching thread to switch the current navigation route to the new navigation route.

[0091] The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. An interactive method based on screen projection assistance, characterized in that, include: Step 1: After the requesting end receives the assistance information sent by the assisting end, it maintains the current navigation path and launches the new route setting interface as the target user interface. Step 2: The requesting end projects the target user interface onto the assisting end; Step 3: The requesting end sends an assistance period to the assisting end, and divides the assistance period into a preset number of sub-periods according to a first preset rule, wherein the first preset rule is based on an average or non-average method. Step 4: The requesting end determines whether it has received a new navigation route from the assisting end within each sub-period. If yes, it proceeds to the next step; otherwise, it performs corresponding operations according to the second preset rule. The corresponding operations according to the second preset rule include: Step 401: The requesting end monitors whether a new navigation route has been received before the end of the first sub-term. If so, the countdown ends; otherwise, proceed to the next step. Step 402: The requesting end monitors whether a new navigation route has been received before the end of the second sub-term. If so, the countdown ends; otherwise, the requesting end sends a first preset warning message to the assisting end and prompts the driver to slow down. Step 403: The requesting terminal screen monitors whether a new navigation route has been received in the first half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a second preset warning message is sent to the assisting terminal, and the driver is prompted to drive at the minimum speed allowed by the current road. Step 404: The requesting terminal screen monitors whether a new navigation route is received in the second half of the time before the end of the third sub-term. If so, the countdown ends; otherwise, a third preset warning message is sent to the assisting terminal, prompting the driver to stop in the emergency lane. Step 5: The requesting end starts a navigation path switching thread to switch the current navigation route to the new navigation route.

2. The interactive method based on screen projection assistance according to claim 1, characterized in that, Step 2 includes: Step 201: The requesting end encodes the target shared interface according to the preset video encoding format and sends it to the frame buffer; Step 202: The requesting end reads the encoded data from the frame buffer and generates a data packet to send to the assisting end through a preset transmission protocol; Step 203: The assisting end receives the data packet, decodes and displays it, and simultaneously calculates the current packet loss rate of the data packet and feeds it back to the requesting end at a preset period. Step 204: The requesting end adjusts the sending speed of the data packets according to the current packet loss rate.

3. The interactive method based on screen projection assistance according to claim 2, characterized in that, Step 204 includes: Step 2041: Obtain network latency parameters and calculate the current network jitter J according to the first preset formula; Step 2042: Calculate the comprehensive parameter z according to the current packet loss rate L and the current network jitter J using the second preset formula, and set a first judgment threshold T for the comprehensive parameter z. z1 Second judgment threshold T z2 The first judgment threshold T z1 The second threshold, T, represents the threshold at which network congestion occurs. z2 A threshold indicating that the network has experienced congestion; Step 2043: Determine whether the comprehensive parameter z does not exceed the first judgment threshold T. z1 If yes, proceed to the next step; otherwise, further determine whether the comprehensive parameter z does not exceed the second judgment threshold T. z2 If so, the data packet transmission rate is adjusted according to the first speed formula, and the current transmission rate S is taken as the transmission rate S during the most recent congestion. c Otherwise, adjust the data packet sending speed according to the second speed formula, and use the current sending speed S as the sending speed S during the most recent congestion. c .

4. The interactive method based on screen projection assistance according to claim 3, characterized in that, The process after step 2043 also includes: Step 2044: Set the third judgment threshold T c1 Fourth judgment threshold T c2 The third judgment threshold T c1 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The first ratio and the threshold of no network congestion, the fourth judgment threshold T c2 This indicates that the current transmission speed has reached the transmission speed S during the most recent congestion. c The second ratio and the threshold of no network congestion, the third judgment threshold T c1 Much greater than the fourth judgment threshold T c2 ; Step 2045, determine 1-S / S c <T c1 If the condition is met, proceed to the next step; otherwise, adjust the sending speed according to the third speed formula. Step 2046, Determine 1-S / S c <T c2 If the condition is met, proceed to the next step; otherwise, adjust the sending speed according to the fourth speed formula. Step 2047, determine whether S c <holds. If so, adjust the transmission speed according to the fifth speed formula; otherwise, keep the current transmission speed unchanged.

5. The interactive method based on screen projection assistance according to claim 4, characterized in that, The first preset formula is: J=J p +(|DD p |-J p ) / A, where D represents the current network latency, D p J represents the network latency at the previous moment. p This represents the network jitter at the previous moment, and A represents the gain coefficient.

6. The interactive method based on screen projection assistance according to claim 5, characterized in that, The second preset formula is: z = α1 * L / L max +α2*J / J max , where L max J represents the maximum allowable packet loss rate during transmission. max This represents the maximum allowable delay jitter during transmission, where α1 and α2 represent the proportions of packet loss rate and delay jitter, and α1+α2=1.

7. The interactive method based on screen projection assistance according to claim 6, characterized in that, The first velocity formula is: S n =(1-2z) z *S, the second velocity formula is: S n =k1*S, where S n Let S represent the transmission speed at the next moment, S represent the current transmission speed, and k1 represent the congestion coefficient, which is a constant.

8. The interactive method based on screen projection assistance according to claim 6, characterized in that, The formula for the third velocity is: S n =S+S0, where S0 represents the constant that increases the transmission speed; The formula for the fourth velocity is: Where e represents a natural number, W represents the maximum network bandwidth, and S represents the current sending speed.

9. The interactive method based on screen projection assistance according to claim 6, characterized in that, The fifth velocity formula is: S n =S+(T z1 -z)*S c .

Images