Remote driving image real-time transmission method, system, control device and storage medium

Abstract

The present invention discloses a remote driving image real-time transmission method, system, control device and storage medium, wherein the remote driving image real-time transmission method dynamically adjusts the bit rate of image data transmission according to the radio signal conditions and the packet loss rate, appropriately reduces the image quality without affecting the staff's observation of the vehicle's surrounding environment, and even converts the image into a grayscale image when the signal quality is extremely poor, so that the staff can observe the environmental image around the vehicle in a timely and stable manner under various conditions, thereby accurately performing remote control of the vehicle.

Description

Technical Field

[0001] The present invention relates to the field of remote driving, and in particular to a method, system, control device and storage medium for real-time transmission of remote driving images. Background Art

[0002] Remote driving technology transmits the vehicle's perception data to the remote cockpit through wireless communication, and the staff remotely controls the vehicle in the remote cockpit. This technology has great application value in the field of engineering vehicle driving with complex and high-risk working environments.

[0003] At present, the remote driving end's control of the vehicle mainly relies on the video images of the vehicle's surroundings captured by the camera installed on the vehicle. The transmission of video images is affected by the data bandwidth. In a complex construction site environment, the wireless signal will be greatly affected, resulting in reduced radio signal strength, signal-to-noise ratio, etc., which in turn affects the data bandwidth of image data transmission. This also causes the images received by the remote driving end to appear deformed, mosaic, and stuck, which has a great impact on the operator's accurate acquisition of the vehicle's surroundings. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned problems existing in the prior art and to provide a method, system, control device and storage medium for real-time transmission of remote driving images.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] The method for real-time transmission of remote driving images comprises the following steps

[0007] S1, obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, execute S2, and when it is confirmed that the packet loss rate is not equal to 0, execute S3;

[0008] S2, continue image transmission according to the maximum bit rate and execute S1;

[0009] S3, according to the current radio signal strength fed back by the receiving end, adjusting the bit rate to the product of the maximum bit rate and the current radio signal strength, wherein the current radio signal strength is calculated as a percentage, and executing S4;

[0010] S4, determines whether the adjusted bit rate is greater than the minimum bit rate, when the adjusted bit rate is greater than the minimum bit rate, execute S5; otherwise, execute S6;

[0011] S5, continue image transmission according to the adjusted bit rate, and execute S7;

[0012] S6, converting the image into a grayscale image and continuing the image transmission according to the grayscale image bit rate, and executing S7;

[0013] S7, continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, execute S3; otherwise, execute S8;

[0014] S8, increasing the bit rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and determining whether the increased bit rate adjustment coefficient is less than the current radio signal strength, and when it is determined that the increased bit rate adjustment coefficient is less than the current radio signal strength, executing S9; otherwise, executing S6;

[0015] S9, adjust the bit rate to: maximum bit rate × [current radio signal strength - bit rate adjustment coefficient], and execute S4.

[0016] Preferably, in the real-time transmission method of remote driving images, the maximum bit rate is (W×H×3)×2, the minimum bit rate is (W×H×3) / 2, and the grayscale image bit rate is (W×H×1), wherein W×H is the resolution of the image.

[0017] Preferably, in the remote driving image real-time transmission method, the bit rate adjustment coefficient is calculated according to the following formula:

[0018] △S = (n-1)×m

[0019] Wherein, △S is the bit rate adjustment coefficient, n is the number of times the packet loss rate is not 0 continuously, and m is the preset adjustment range.

[0020] Preferably, in the remote driving image real-time transmission method, the preset adjustment amplitude m is 1%.

[0021] Preferably, in the real-time transmission method of remote driving images, the packet loss rate information and radio signal strength information are fed back through the digital transmission channel of the radio station.

[0022] Preferably, in the remote driving image real-time transmission method, the location for receiving the packet loss rate information and the radio signal strength is 1-5 km away from the receiving end.

[0023] Remote driving image transmission system, including

[0024] A first packet loss rate judgment unit is used to obtain packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first rate control unit, and when it is confirmed that the packet loss rate is not equal to 0, send a signal to the first rate adjustment unit;

[0025] A first bit rate execution unit, used to continue image transmission according to the maximum bit rate, and send a signal to the first packet loss rate determination unit;

[0026] A first code rate adjustment unit, configured to adjust the code rate to the product of the maximum code rate and the current radio signal strength according to the current radio signal strength fed back by the receiving end, wherein the current radio signal strength is calculated as a percentage, and send a signal to the code rate size determination unit;

[0027] A code rate determination unit, used to determine whether the adjusted code rate is greater than the minimum code rate, and when the adjusted code rate is greater than the minimum code rate, a signal is sent to the second code rate execution unit; otherwise, a signal is sent to the third code rate execution unit;

[0028] A second code rate execution unit, for continuing image transmission according to the adjusted code rate, and sending a signal to the second packet loss rate determination unit;

[0029] A third bit rate execution unit, used for converting the image into a grayscale image and continuing the image transmission according to the grayscale image bit rate, and sending a signal to the second packet loss rate determination unit;

[0030] A second packet loss rate determination unit is used to continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first code rate adjustment unit; otherwise, send a signal to the code rate adjustment coefficient update unit;

[0031] a code rate adjustment coefficient updating unit, increasing the code rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and judging whether the increased code rate adjustment coefficient is less than the current radio signal strength, and sending a signal to the third code rate execution unit when it is determined that the increased code rate adjustment coefficient is less than the current radio signal strength; otherwise, sending a signal to the second code rate adjustment unit;

[0032] The second bit rate adjustment unit adjusts the bit rate to: maximum bit rate×[current radio signal strength-bit rate adjustment coefficient], and sends a signal to the bit rate size determination unit.

[0033] The control device includes a memory and a processor, wherein the memory stores a program, and when the program is executed, any of the above-mentioned remote driving image real-time transmission methods is implemented.

[0034] A storage medium having a program stored thereon, wherein when the program is executed, any of the above-mentioned methods for real-time transmission of remote driving images is implemented.

[0035] The advantages of the technical solution of the present invention are mainly reflected in:

[0036] This solution dynamically adjusts the bit rate of image data transmission according to the radio signal conditions and packet loss rate, appropriately reduces the image quality without affecting the staff's observation of the vehicle's surroundings, and even converts the image into a grayscale image when the signal quality is extremely poor, so that the staff can observe the environmental image around the vehicle in a timely and stable manner under various circumstances, thereby accurately performing remote control of the vehicle.

[0037] This solution adopts radio communication, which can make use of the original structure of the vehicle body. No additional equipment is required. Only the corresponding software needs to be designed, which is easy to promote and apply. At the same time, the digital transmission channel is used to feedback the packet loss rate and radio signal strength information, which can ensure the accuracy and timeliness of data acquisition and the accuracy of bit rate adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0039] The purpose, advantages and features of the present invention will be illustrated and explained by the non-limiting description of the following preferred embodiments. These embodiments are only typical examples of the application of the technical solution of the present invention, and any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection claimed by the present invention.

[0040] In the description of the scheme, it should be noted that the terms "center", "up", "down", "left", "right", "front", "back", "vertical", "horizontal", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of description and simplified description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance. Moreover, in the description of the scheme, with the operator as a reference, the direction close to the operator is the proximal end, and the direction away from the operator is the distal end.

[0041] Example 1

[0042] The following is an explanation of the remote driving image real-time transmission method disclosed by the present invention in conjunction with the accompanying drawings. The remote driving image real-time transmission method is based on data transmission by a radio station, that is, an on-board transmitter on the vehicle communicates with a receiving end in the remote cockpit via radio, and the distance between the on-board transmitter and the receiving end is between 1-5km, and their communication includes a data transmission channel and an image transmission channel. The specific technologies of the radio station communication, the data transmission channel and the image transmission channel are known technologies and will not be elaborated here.

[0043] In actual communication, the data volume of the digital transmission channel is extremely small compared to that of the image transmission channel and the digital transmission channel is bidirectional, so the digital transmission channel can exchange data in a relatively timely manner. Therefore, in the actual video transmission process, the receiving end feeds back the radio signal strength and packet loss rate to the vehicle-mounted transmitter in real time through the digital transmission channel to ensure data accuracy.

[0044] In order to avoid image deformation and freezing as much as possible, this solution dynamically and gradually adjusts the bit rate according to the detected packet loss rate and radio signal strength. When continuous packet loss occurs, the bit rate adjustment coefficient is accumulated to make the adjustment more reasonable. Even when the radio signal strength is extremely weak, the image to be transmitted is converted into a grayscale image, effectively ensuring that the operators in the remote cockpit can observe the environmental image around the vehicle in a timely and stable manner under various radio signal strength conditions, thereby accurately controlling the vehicle remotely.

[0045] During video transmission, the correspondence between image resolution and required bit rate can be set as follows: the maximum bit rate is (W×H×3)×2, the minimum bit rate is (W×H×3) / 2, and the grayscale bit rate is (W×H×1), where W×H is the image resolution. Taking the transmission of a 1080P video as an example, when transmitting with high quality, the bit rate is the maximum bit rate = (1920×1080×3)×2≈8Mbps; when transmitting with low quality, the bit rate is the minimum bit rate = (1920×1080×3) / 2≈2Mbps. In addition, in the actual research process, it is found that for a video of a certain resolution, when the bit rate is lower than (W×H×3) / 2, the image will have serious mosaics, which seriously affects the viewing. Therefore, (W×H×3) / 2 is set as the minimum bit rate. If the bit rate of the currently transmitted video is lower than the minimum bit rate, then at this time, the currently transmitted image needs to be converted into a grayscale image and transmitted at the grayscale bit rate.

[0046] The method of the present invention will be specifically described below. Figure 1 As shown, it includes the following steps

[0047] S0, transmits the image to the receiving end at the maximum bit rate (W×H×3)×2.

[0048] S1, obtain the packet loss rate information fed back by the receiving end, and determine whether the packet loss rate is equal to 0. When it is determined that the packet loss rate is equal to 0, it indicates that the data transmission is stable and there is no abnormality; execute S2, when it is confirmed that the packet loss rate is not equal to 0, it indicates that the current radio signal strength cannot achieve high-quality image transmission at the current bit rate, and execute S3.

[0049] S2, continue image transmission according to the maximum bit rate (W×H×3)×2, and execute S1, that is, if the packet loss rate is always 0, the transmission is always carried out at the maximum bit rate.

[0050] S3, according to the current radio signal strength fed back by the receiving end, adjusting the bit rate of the video to the product of the maximum bit rate and the current radio signal strength, wherein the current radio signal strength is calculated as a percentage and its range is between 0% and 100%. At this time, the adjusted bit rate is less than the maximum bit rate, and S4 is executed;

[0051] S4, determine whether the adjusted bit rate is greater than the minimum bit rate (W×H×3) / 2. When the adjusted bit rate is greater than the minimum bit rate, execute S5; otherwise, execute S6.

[0052] S5, continue image transmission according to the adjusted bit rate, and execute S7.

[0053] S6, converting the image into a grayscale image and continuing image transmission according to the grayscale image bit rate, and executing S7.

[0054] S7, continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, execute S3; otherwise, execute S8.

[0055] S8, increasing the bit rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and determining whether the increased bit rate adjustment coefficient is less than the current radio signal strength, and when it is determined that the increased bit rate adjustment coefficient is less than the current radio signal strength, executing S9; otherwise, executing S6;

[0056] The bit rate adjustment coefficient is calculated according to the following formula:

[0057] △S = (n-1)×m

[0058] Wherein, △S is the bit rate adjustment coefficient, n is the number of times the packet loss rate is not equal to 0 continuously, and m is the preset adjustment amplitude, which is 1%. That is, each time the packet loss rate is not equal to 0, when the next side determines that the packet loss rate continues to be not equal to 0, the bit rate adjustment coefficient is increased by 1% on the basis of the previous one. For example, when the packet loss rate is not equal to 0 for four consecutive times, when it occurs for the second time, the bit rate adjustment coefficient is 1%=(2-1)×1%; when the third time is initially selected, the bit rate adjustment coefficient is 1%+1%=2%=(3-1)×1%; when it occurs for the fourth time, the bit rate adjustment coefficient is 2%+1%=3%=(4-1)×1%.

[0059] S9, adjust the bit rate to: maximum bit rate × [current radio signal strength - bit rate adjustment coefficient], and execute S4.

[0060] For example, taking a video with a transmission resolution of 1080P as an example, at the beginning, it is transmitted at the maximum bit rate of 8Mbps. When the feedback signal shows that there is no packet loss, it continues to be transmitted at the maximum bit rate of 8Mbps. When the feedback signal shows that there is packet loss for the first time, the current radio signal strength fed back by the receiving end is obtained, for example, it is 90%, then the bit rate is adjusted to 8Mbps×90%=7.2 Mbps, and it is determined that the adjusted bit rate of 7.2 Mbps is greater than the minimum bit rate of 2Mbps, then the transmission is continued at 7.2 Mbps; if it is determined that the current radio signal is 20%, the bit rate is adjusted to 8Mbps×20%=1.6Mbps, and it is determined that the adjusted bit rate of 1.6Mbps is less than the minimum bit rate, then the subsequent image to be transmitted is switched to a grayscale image, and the transmission is continued at the grayscale image bit rate.

[0061] After transmitting at the adjusted bit rate, feedback data is continued to be received. If the packet loss rate is not equal to 0 for the second time after the first packet loss rate is not equal to 0, the current radio signal strength fed back by the receiving end is 90%, and the bit rate adjustment coefficient is determined to be (2-1)×1%=1%. The bit rate adjustment coefficient is determined to be less than the current radio signal strength, and the adjusted bit rate is 8Mbps×(90%-1%)=7.12Mbps. At this time, the adjusted bit rate of 7.12 Mbps is greater than the minimum bit rate of 2Mbps, and transmission continues at 7.12 Mbps.

[0062] Continue to receive feedback data. If the packet loss rate is equal to 0 at this time, and the current radio signal strength fed back by the receiving end is 100%, the bit rate is adjusted to 8Mbps×100%=8Mbps. If 8Mbps is greater than the minimum bit rate of 2Mbps, the data is transmitted at the adjusted bit rate of 8Mbps.

[0063] If, when receiving feedback data, it is determined that the packet loss rate is not equal to 0 for multiple times in a row, such as 50 times, the bit rate adjustment coefficient is determined to be 49%. At this time, the current radio signal strength fed back by the receiving end is 40%, and the subsequent image to be transmitted is converted into a grayscale image and transmitted at the grayscale image bit rate; if the radio signal strength fed back by the receiving end is 50% at this time, the adjusted bit rate is 8Mbps×(50-49)%=0.8Mbps. At this time, the adjusted bit rate is less than the minimum bit rate of 2Mbps. Similarly, the subsequent image to be transmitted is converted into a grayscale image and the grayscale image bit rate is installed for transmission.

[0064] Example 2

[0065] This embodiment discloses a remote driving image transmission system, comprising:

[0066] A first packet loss rate judgment unit is used to obtain packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first rate control unit, and when it is confirmed that the packet loss rate is not equal to 0, send a signal to the first rate adjustment unit;

[0067] A first bit rate execution unit, used to continue image transmission according to the maximum bit rate, and send a signal to the first packet loss rate determination unit;

[0068] A first code rate adjustment unit, configured to adjust the code rate to the product of the maximum code rate and the current radio signal strength according to the current radio signal strength fed back by the receiving end, wherein the current radio signal strength is calculated as a percentage, and send a signal to the code rate size determination unit;

[0069] A code rate determination unit, used to determine whether the adjusted code rate is greater than the minimum code rate, and when the adjusted code rate is greater than the minimum code rate, a signal is sent to the second code rate execution unit; otherwise, a signal is sent to the third code rate execution unit;

[0070] A second code rate execution unit, for continuing image transmission according to the adjusted code rate, and sending a signal to the second packet loss rate determination unit;

[0071] A third bit rate execution unit, used for converting the image into a grayscale image and continuing the image transmission according to the grayscale image bit rate, and sending a signal to the second packet loss rate determination unit;

[0072] A second packet loss rate determination unit is used to continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first code rate adjustment unit; otherwise, send a signal to the code rate adjustment coefficient update unit;

[0073] a code rate adjustment coefficient updating unit, increasing the code rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and judging whether the increased code rate adjustment coefficient is less than the current radio signal strength, and sending a signal to the third code rate execution unit when it is determined that the increased code rate adjustment coefficient is less than the current radio signal strength; otherwise, sending a signal to the second code rate adjustment unit;

[0074] The second bit rate adjustment unit adjusts the bit rate to: maximum bit rate×[current radio signal strength-bit rate adjustment coefficient], and sends a signal to the bit rate size determination unit.

[0075] The system is arranged at the vehicle-mounted transmitter of the vehicle. The process of the vehicle-mounted transmitter acquiring the video information around the vehicle body from the vehicle is a known technology and will not be described in detail here.

[0076] Example 3

[0077] This embodiment discloses a control device, including a memory and a processor, wherein the memory stores a program, and when the program is executed, the method for real-time transmission of remote driving images described in the above-mentioned embodiment 1 is implemented.

[0078] Example 4

[0079] A storage medium having a program stored thereon, wherein when the program is executed, the method for real-time transmission of remote driving images described in the above-mentioned embodiment 1 is implemented.

[0080] There are many implementation methods of the present invention, and all technical solutions formed by equivalent transformation or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A remote driving image real-time transmission method, characterized in that: The following steps are included S1, obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, execute S2, and when it is confirmed that the packet loss rate is not equal to 0, execute S3; S2, continue image transmission at the maximum bit rate and execute S1; the maximum bit rate is (W×H×3)×2bps, where W×H is the resolution of the image; S3, according to the current radio signal strength fed back by the receiving end, adjusting the bit rate to the product of the maximum bit rate and the current radio signal strength, wherein the current radio signal strength is calculated as a percentage, and executing S4; S4, determine whether the adjusted bit rate is greater than the minimum bit rate. When the adjusted bit rate is greater than the minimum bit rate, execute S5; otherwise, execute S6; the minimum bit rate is (W×H×3) / 2 bps; S5, continue image transmission according to the adjusted bit rate, and execute S7; S6, converting the image into a grayscale image and continuing the image transmission according to the grayscale image bit rate, and executing S7; the grayscale image bit rate is (W×H×1) bps; S7, continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, execute S3; otherwise, execute S8; S8, increasing the bit rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and determining whether the increased bit rate adjustment coefficient is less than the current radio signal strength, and when it is determined that the increased bit rate adjustment coefficient is less than the current radio signal strength, executing S9; otherwise, executing S6; S9, adjust the bit rate to: maximum bit rate × [current radio signal strength - bit rate adjustment coefficient], and execute S4.

2. The method for real-time transmission of remote driving images according to claim 1, characterized in that: The bit rate adjustment coefficient is calculated according to the following formula: △S = (n-1)×m Wherein, △S is the bit rate adjustment coefficient, n is the number of times the packet loss rate is not 0 continuously, and m is the preset adjustment range.

3. The method for real-time transmission of remote driving images according to claim 2, characterized in that: The preset adjustment range m is 1%.

4. The method for real-time transmission of remote driving images according to claim 1, characterized in that: The packet loss rate information and radio signal strength information are fed back through the data transmission channel of the radio station.

5. The method for real-time transmission of remote driving images according to any one of claims 1 to 4, characterized in that: The location for receiving the packet loss rate information and the radio signal strength is 1-5 km away from the receiving end.

6. Remote driving image transmission system, characterized by: include A first packet loss rate judgment unit is used to obtain packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first rate control unit, and when it is confirmed that the packet loss rate is not equal to 0, send a signal to the first rate adjustment unit; A first bit rate execution unit, used to continue image transmission according to the maximum bit rate, and send a signal to the first packet loss rate determination unit; A first code rate adjustment unit, configured to adjust the code rate to the product of the maximum code rate and the current radio signal strength according to the current radio signal strength fed back by the receiving end, wherein the current radio signal strength is calculated as a percentage, and send a signal to the code rate size determination unit; The maximum bit rate is (W×H×3)×2bps, where W×H is the resolution of the image; The code rate size judgment unit is used to judge whether the adjusted code rate is greater than the minimum code rate. When the adjusted code rate is greater than the minimum code rate, a signal is sent to the second code rate execution unit; otherwise, a signal is sent to the third code rate execution unit; the minimum code rate is (W×H×3) / 2 bps; A second code rate execution unit, for continuing image transmission according to the adjusted code rate, and sending a signal to the second packet loss rate determination unit; A third bit rate execution unit is used to convert the image into a grayscale image and continue the image transmission according to the grayscale image bit rate, and send a signal to the second packet loss rate determination unit; the bit rate of the grayscale image is (W×H×1) bps; A second packet loss rate determination unit is used to continue to obtain the packet loss rate information fed back by the receiving end, and when it is determined that the packet loss rate is equal to 0, send a signal to the first code rate adjustment unit; otherwise, send a signal to the code rate adjustment coefficient update unit; a code rate adjustment coefficient updating unit, increasing the code rate adjustment coefficient according to the number of times the packet loss rate is not 0 continuously, and judging whether the increased code rate adjustment coefficient is less than the current radio signal strength, and sending a signal to the third code rate execution unit when it is determined that the increased code rate adjustment coefficient is less than the current radio signal strength; otherwise, sending a signal to the second code rate adjustment unit; The second bit rate adjustment unit adjusts the bit rate to: maximum bit rate×[current radio signal strength-bit rate adjustment coefficient], and sends a signal to the bit rate size determination unit.

7. A control device, comprising a memory and a processor, wherein a program is stored in the memory, characterized in that: When the program is executed, the real-time transmission method of remote driving images described in any one of claims 1-5 is implemented.

8. A storage medium having a program stored thereon, characterized in that: When the program is executed, the real-time transmission method of remote driving images described in any one of claims 1-5 is implemented.

Images