Airborne optoelectronic payload control system and control method

Abstract

The application provides an airborne photoelectric load control system and a control method, which comprises a ground load control console, a link system and an airborne task manager, wherein the ground load control console comprises a control joystick, an attitude sensor and a load controller, the control joystick is used for outputting attitude control instructions of the photoelectric load; the attitude sensor is used for collecting attitude information of the control joystick; the load controller is used for uploading the attitude control instructions of the photoelectric load and the attitude information of the control joystick through the link system, receiving and displaying the state of the photoelectric load and the processed video image of the photoelectric load; and the airborne task manager is used for controlling the photoelectric load according to the attitude control instructions of the photoelectric load and the attitude information of the control joystick, receiving the state of the photoelectric load, and receiving and processing the video image collected in real time by the photoelectric load. The technical scheme of the application can solve the technical problem that the quality of the video image of the photoelectric load is reduced due to the slow response of the attitude change of the photoelectric load.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle control, and in particular to an airborne photoelectric payload control system and a control method. BACKGROUND

[0002] A ground personnel realizes control of an airborne photoelectric payload and monitoring of photoelectric payload collected video on a ground station. During execution of a task by the unmanned aerial vehicle, the ground operator controls the photoelectric payload to switch direction, and the video collected by the photoelectric payload will appear to switch scenes. In this case, because the photoelectric payload rotation response requires time, the ground personnel operates a joystick, and the airborne photoelectric payload attitude will appear to have visual rotation delay, and because the video collected by the photoelectric payload appears to switch scenes, the video sequence frames before and after lose correlation, affecting the encoding quality of subsequent frames, and further affecting the ground video display, which will appear to have a screen of flowers and delay. SUMMARY

[0003] The present application aims to at least solve one of the technical problems existing in the prior art.

[0004] According to an aspect of the present application, an airborne photoelectric payload control system is provided, which comprises a ground payload control console, a link system and an airborne task manager. The ground payload control console comprises a control joystick, an attitude sensor and a payload controller. The control joystick is used to output an attitude control instruction of the photoelectric payload. The attitude sensor is mounted on the control joystick and is used to collect attitude information of the control joystick. The payload controller is connected with the control joystick and the attitude sensor respectively. The payload controller is used to upload the attitude control instruction of the photoelectric payload and the attitude information of the control joystick to the airborne task manager through the link system, and receive and display the state of the photoelectric payload and the processed photoelectric payload video image downloaded by the airborne task manager through the link system. The airborne task manager is used to control the photoelectric payload according to the attitude control instruction of the photoelectric payload and the attitude information of the control joystick, receive the state of the photoelectric payload, and receive and process the video image collected by the photoelectric payload in real time.

[0005] Further, the link system comprises a ground station system and an airborne measurement and control system. The ground station system is used to upload the attitude control instruction of the photoelectric payload and the attitude information of the control joystick to the airborne task manager. The airborne measurement and control system is used to transmit the video image collected by the photoelectric payload in real time and the state of the photoelectric payload downloaded by the airborne task manager to the payload controller.

[0006] Further, the on-board task manager comprises a CPU processing module and a video processing module, the CPU processing module is used to control the opto-electric load according to the attitude control instruction of the opto-electric load and the attitude information of the control rocker, and the video processing module is used to receive the state of the opto-electric load and receive and process the video image collected by the opto-electric load in real time.

[0007] Further, the processing of the video image collected by the opto-electric load in real time by the video processing module comprises compression of the video image collected by the opto-electric load in real time and adjustment of the coding structure.

[0008] Further, the video processing module adopts H.264 / AVC standard to compress the video image collected by the opto-electric load in real time.

[0009] According to another aspect of the present application, there is provided an on-board opto-electric load control method for realizing control of the on-board opto-electric load by using the on-board opto-electric load control system as described above.

[0010] Further, the on-board opto-electric load control method specifically comprises: the control rocker outputs the attitude control instruction of the opto-electric load; the attitude sensor acquires the attitude information of the control rocker; the on-board task manager acquires the attitude control instruction of the opto-electric load and the attitude information of the control rocker through the link system; it is judged whether the angular velocity of the control rocker is greater than a first angular velocity threshold or the acceleration is greater than a first acceleration threshold according to the attitude information of the control rocker, and if so, the expected next-time opto-electric load attitude information is acquired according to the attitude control instruction of the opto-electric load and the attitude information of the control rocker; the next-time opto-electric load attitude change amount is acquired according to the attitude control instruction of the opto-electric load and the expected next-time opto-electric load attitude information; the on-board task manager controls the attitude change of the opto-electric load according to the current opto-electric load actual position attitude and the next-time opto-electric load attitude change amount.

[0011] Further, the on-board opto-electric load control method acquires the next-time opto-electric load attitude change amount Δθ according to

[0012] According to another aspect of the present application, there is provided an on-board opto-electric load video processing method for realizing video processing of the on-board opto-electric load by using the on-board task manager in the on-board opto-electric load control system as described above.

[0013] ​Further, the airborne photoelectric load video processing method specifically comprises: the airborne task manager compresses the video image collected by the photoelectric load in real time; according to the attitude change of the photoelectric load and the attitude information of the control rocker collected by the attitude sensor, the compressed video image is subjected to large scene switching detection to determine whether large scene switching occurs, if not, the video is directly encoded to complete the video processing; if yes, it is determined whether the current frame of the compressed video after the large scene switching is an I frame, if the current frame of the compressed video after the large scene switching is an I frame, the video is directly encoded to complete the video processing; if the current frame of the compressed video after the large scene switching is not an I frame, the current frame is set as an I frame, and the video is encoded after the coding rate is reduced to complete the video processing.

[0014] The technical scheme of the present application provides an airborne photoelectric load control system and a control method, the airborne photoelectric load control system collects the attitude information of the control rocker through the attitude sensor mounted on the control rocker, and combines the attitude control instruction of the photoelectric load to control the photoelectric load, so that the control of the photoelectric load direction is accelerated through the prediction of the control rocker, and the response speed of the controlled attitude change of the photoelectric load is improved. Compared with the prior art, the technical scheme of the present application can solve the technical problem that the attitude change response of the photoelectric load is not timely, which leads to the reduction of the video image quality of the photoelectric load. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings included to provide a further understanding of the embodiments of the present application, constitute a part of the specification and serve to explain the principles of the present application together with the text. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0016] Figure 1 A schematic diagram of the framework of the airborne photoelectric load control system provided by the specific embodiment of the present application is shown;

[0017] Figure 2 A flowchart of the airborne photoelectric load video processing method provided by the specific embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other in the case of no conflict. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The description of the at least one exemplary embodiment below is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0019] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should be understood that, when the terms "comprise" and / or "include" are used in the specification, it indicates the presence of the features, steps, operations, devices, components and / or combinations thereof.

[0020] Unless specifically stated otherwise, the relative arrangements of the components and steps illustrated in these embodiments and the numerical expressions and values set forth herein are not limiting of the scope of the application. It should be understood that the various parts of the drawings are not necessarily drawn to scale, and that, for the purpose of convenience and clarity, not all components can be shown in a given figure. Techniques, methods, and devices known to those of ordinary skill in the art can not be discussed in detail, but rather can be assumed to be known by those of ordinary skill in the art. In the examples shown and discussed herein, any specific values should be interpreted as merely illustrative and not as a limitation on the scope of the exemplary embodiments. Thus, other examples of the exemplary embodiments can have different values. It should be noted that like reference numerals and letters in the following drawings represent similar items, and thus once an item is defined in one drawing, it is not necessary to further discuss it in the subsequent drawings.

[0021] As Figure 1As shown, the specific embodiment according to the present application provides an airborne optoelectronic payload control system, which comprises a ground payload control console, a link system and an airborne task manager, the ground payload control console comprises a control joystick, a posture sensor and a payload controller, the control joystick is used to output the posture control instruction of the optoelectronic payload; the posture sensor is carried on the control joystick, and is used to collect the posture information of the control joystick; the payload controller is connected with the control joystick and the posture sensor respectively, and is used to upload the posture control instruction of the optoelectronic payload and the posture information of the control joystick to the airborne task manager through the link system, and receive and display the state of the optoelectronic payload and the processed video image of the optoelectronic payload transmitted by the airborne task manager through the link system; the airborne task manager is used to control the optoelectronic payload according to the posture control instruction of the optoelectronic payload and the posture information of the control joystick, receive the state of the optoelectronic payload, and receive and process the video image collected by the optoelectronic payload in real time.

[0022] By using the configuration mode, the airborne optoelectronic payload control system is provided, which can control the optoelectronic payload by combining the posture control instruction of the optoelectronic payload with the posture information of the control joystick collected by the posture sensor carried on the control joystick, can accelerate the control of the azimuth of the optoelectronic payload by predicting the control joystick, and can improve the response speed of the controlled attitude change of the optoelectronic payload. Compared with the prior art, the technical scheme of the present application can solve the technical problem that the attitude change response of the optoelectronic payload is not timely, which leads to the reduction of the video image quality of the optoelectronic payload.

[0023] Further, as one specific embodiment of the present application, in order to improve the sensitivity and accurate measurement of the posture sensor to the posture information of the control joystick, the posture sensor can be a nine-axis posture sensor. The nine-axis posture sensor is used to measure the angular velocity and acceleration of the control joystick, and is used as the posture information of the control joystick.

[0024] In addition, in the present application, the link system comprises a ground station system and an airborne measurement and control system, the ground station system is used to upload the posture control instruction of the optoelectronic payload and the posture information of the control joystick to the airborne task manager, and the airborne measurement and control system is used to transmit the video image collected by the optoelectronic payload in real time and the state of the optoelectronic payload transmitted by the airborne task manager to the payload controller.

[0025] Further, in the present application, in order to improve the video processing capability of the on-board task manager, and without affecting the scheduling of other tasks, the configurable on-board task manager comprises a CPU processing module and a video processing module, the CPU processing module is used to control the optoelectronic payload according to the attitude control instruction of the optoelectronic payload and the attitude information of the control rocker, and the video processing module is used to receive the state of the optoelectronic payload, and receive and process the video image collected by the optoelectronic payload in real time. The on-board task manager can realize the attitude control of the optoelectronic payload and the processing of the video through multi-threading.

[0026] In the present application, the CPU processing module is responsible for obtaining the attitude control instruction of the optoelectronic payload at the next moment according to the received attitude control instruction of the optoelectronic payload and the attitude information of the control rocker, directly controlling the change of the attitude of the optoelectronic payload according to the attitude control instruction of the optoelectronic payload at the next moment, so that the optoelectronic payload quickly reaches the expected attitude. The optoelectronic payload changes the attitude according to the instruction sent by the CPU processing module, and sends the collected video source code to the video processing module.

[0027] In addition, in the present application, the processing of the video image collected by the optoelectronic payload in real time by the video processing module comprises compression of the video image collected by the optoelectronic payload in real time and adjustment of the encoding structure. When the attitude of the optoelectronic payload changes, it will cause the video scene to switch, at this time, according to the attitude control instruction of the optoelectronic payload and the attitude information of the control rocker, the video encoding structure of the optoelectronic payload is adjusted, which can relatively improve the encoding quality of the video during video switching. The processed optoelectronic payload video image stream is transferred to the on-board measurement and control system.

[0028] As a specific embodiment of the present application, the on-board task manager can adopt an on-board task management computer; the video processing module can adopt H.264 / AVC standard to compress the video.

[0029] Further, in the present application, the payload controller, the link system and the on-board task manager can be used for the transmission of other control instructions in addition to the attitude control instruction of the optoelectronic payload, such as speed instructions and the like.

[0030] In order to improve the response speed of the optoelectronic payload rotation, the present application installs an attitude sensor on the ground operation control rocker, collects the angular velocity and acceleration of the control rocker when the control rocker moves, and combines the attitude control instruction of the optoelectronic payload, i.e. the azimuth angle and the pitch angle expected to be rotated by the optoelectronic payload, to speed up the prediction of the control rocker to control the azimuth of the optoelectronic payload, thereby improving the response time of the optoelectronic payload rotation.

[0031] According to another aspect of the present invention, an airborne optoelectronic payload control method is provided, which employs the airborne optoelectronic payload control system described above to control the airborne optoelectronic payload. By applying this configuration, the control of the optoelectronic payload's orientation can be accelerated through prediction of the control joystick, thereby improving the controlled response speed to changes in the optoelectronic payload's attitude.

[0032] Furthermore, in this invention, the airborne optoelectronic payload control method specifically includes: a control joystick outputting attitude control commands for the optoelectronic payload; an attitude sensor acquiring attitude information of the control joystick; an airborne task manager acquiring the attitude control commands for the optoelectronic payload and the attitude information of the control joystick through a link system; determining whether the angular velocity of the control joystick is greater than a first angular velocity threshold or whether the acceleration is greater than a first acceleration threshold based on the attitude information of the control joystick; if so, acquiring the expected attitude information of the optoelectronic payload at the next moment based on the attitude control commands for the optoelectronic payload and the attitude information of the control joystick; acquiring the attitude change of the optoelectronic payload at the next moment based on the attitude control commands for the optoelectronic payload and the expected attitude information of the optoelectronic payload at the next moment; and the airborne task manager controlling the attitude change of the optoelectronic payload based on the current actual position and attitude of the optoelectronic payload and the attitude change of the optoelectronic payload at the next moment.

[0033] Furthermore, in this invention, according to

[0034] Obtain the change in the attitude of the photoelectric load Δθ at the next moment.

[0035] Furthermore, in this invention, the attitude control command of the photoelectric load, the expected attitude information of the photoelectric load at the next moment, and the attitude change of the photoelectric load at the next moment all include azimuth and pitch angles.

[0036] As a specific embodiment of the present invention, the airborne task manager can control the rotation speed of the photoelectric load according to the current actual position and attitude of the photoelectric load and the change in attitude of the photoelectric load at the next moment, so that the photoelectric load can respond quickly to attitude changes.

[0037] Furthermore, in this invention, if the angular velocity of the control joystick is not greater than a first angular velocity threshold and the acceleration is not greater than a first acceleration threshold, then the process returns to determine whether the angular velocity of the control joystick is greater than the first angular velocity threshold or whether the acceleration is greater than the first acceleration threshold.

[0038] like Figure 2 As shown, according to another aspect of the present invention, an airborne optoelectronic payload video processing method is provided. This method implements video processing of the airborne optoelectronic payload through an airborne task manager in the airborne optoelectronic payload control system described above. Applying this configuration, the video image quality acquired by the optoelectronic payload can be optimized by considering the changing trends of the payload's attitude.

[0039] In order that the ground personnel can monitor the video collected by the photoelectric load in real time, the source code video of the photoelectric load is compressed by the video processing module of the airborne task manager, when the attitude of the photoelectric load changes greatly, the collected video scene will correspondingly appear large scene switching, the frames before and after the video sequence lose the correlation, if not processed, directly compressed, the coding quality will be affected, and then the display of the ground video will be affected, and the phenomena such as screen flashing and delay will appear. In order to solve the problem, the application provides an airborne photoelectric load video processing method, which can improve the coding quality when the video is switched by adjusting the video coding structure after detecting the large scene switching.

[0040] Further, in the application, the airborne photoelectric load video processing method specifically comprises: the airborne task manager compresses the video image collected by the photoelectric load in real time; according to the attitude change of the photoelectric load and the attitude information of the control rocker collected by the attitude sensor, large scene switching detection is performed on the compressed video image to determine whether large scene switching occurs, if not, video coding is directly performed to complete video processing; if yes, it is determined whether the current frame of the compressed video after the large scene switching is an I frame, if the current frame of the compressed video after the large scene switching is an I frame, video coding is directly performed to complete video processing; if the current frame of the compressed video after the large scene switching is not an I frame, the current frame is set as an I frame, and video coding is performed after reducing the coding rate to complete video processing.

[0041] H.264 / AVC standard divides the compressed video into I frame, B frame and P frame, when the video scene changes greatly, for the I frame, since the I frame is the key frame, the intra-frame coding is adopted, and it is not affected; for the B frame, since the forward and backward prediction mode is adopted, at least one frame of the adjacent two times is related, so the influence is limited; for the P frame, when the scene switches greatly, there is no correlation with the previous frame, and it will be greatly affected. Therefore, when the video scene switches greatly, since the frames before and after the switching lose the correlation, the coding rate control strategy needs to be modified. When the video scene switches greatly, if the current frame after the switching is not an I frame, the current frame is set as an I frame, so as to improve the coding performance. However, since the switching frame is set as an I frame, the number of I frames is too large, and the length of GOP is shortened, since the I frame is large, in order not to affect the delay, the coding rate needs to be reduced accordingly, so as to ensure the relatively high quality transmission of the video when the video scene switches greatly.

[0042] In addition, in the application, whether the large scene switching occurs is determined specifically by: determining whether the angular velocity of the photoelectric load is greater than a second angular velocity threshold, or whether the acceleration of the photoelectric load is greater than a second acceleration threshold, or whether the angular velocity of the control rocker is greater than a third angular velocity threshold, or whether the acceleration of the control rocker is greater than a third acceleration threshold.

[0043] The application improves the response speed of the attitude change of the optoelectronic payload by predicting the attitude change of the optoelectronic payload based on the change of the ground rocker state, combining the real-time state of the optoelectronic payload, and improving the angular velocity of the rotation of the optoelectronic payload, thereby reducing the visual delay of the attitude change of the optoelectronic payload during scene switching. When the video scene is determined to be large scene switching by combining the attitude change of the optoelectronic payload and the attitude information of the control rocker collected by the rocker sensor, the non-I frame is set as an I frame, the encoding code rate is reduced, the problem of encoding failure and delay caused by the video scene switching of the optoelectronic payload is solved by appropriately reducing the image quality, and the encoding quality of the video during video switching is improved. The application combines the change trend of the attitude of the optoelectronic payload, reduces the number of non-relevant encoding frames, and reduces the encoding code rate, to improve the response speed of the optoelectronic payload during large scene switching and the video encoding quality. The actual effect is good through test verification.

[0044] In summary, the application provides an airborne optoelectronic payload control system and control method. The airborne optoelectronic payload control system collects the attitude information of the control rocker by loading the attitude sensor on the control rocker, combines the attitude control instruction of the optoelectronic payload to control the optoelectronic payload, accelerates the control of the azimuth of the optoelectronic payload by predicting the control rocker, and improves the response speed of the controlled attitude change of the optoelectronic payload. Compared with the prior art, the technical scheme of the application can solve the technical problem of reducing the video image quality of the optoelectronic payload caused by the untimely response of the attitude change of the optoelectronic payload.

[0045] For the sake of brevity, the spatially relative terms, such as "on", "above", "below", "top", "bottom", and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device shown in the figures is turned over, elements described as "above" other elements or features would then be oriented "below" the other elements or features. Accordingly, the exemplary term "above" can encompass both an orientation of above and below. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0046] In addition, it should be noted that the use of "first", "second", and the like words to qualify components is merely for the convenience of distinguishing the corresponding components, and the above words have no special meaning unless otherwise stated, and therefore cannot be understood as a limitation on the scope of protection of the application.

[0047] The above merely provides the preferred embodiments of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the principles and technical scope of the present application shall fall into the scope of the present application.

Claims

1. An airborne optoelectronic payload control method, characterized in that, The airborne optoelectronic load control method employs an airborne optoelectronic load control system to control the airborne optoelectronic load. The airborne optoelectronic payload control system includes: a ground payload control console, a link system, and an airborne task manager. The ground payload control console includes a control joystick, an attitude sensor, and a payload controller. The control joystick is used to output attitude control commands for the optoelectronic payload. The attitude sensor is mounted on the control joystick and is used to collect attitude information from the control joystick. The payload controller is connected to both the control joystick and the attitude sensor. The payload controller is used to upload the attitude control commands for the optoelectronic payload and the attitude information from the control joystick to the airborne task manager via the link system, and to receive and display the status of the optoelectronic payload and the processed video images of the optoelectronic payload transmitted by the airborne task manager through the link system. The airborne task manager is used to control the optoelectronic payload according to the attitude control commands for the optoelectronic payload and the attitude information from the control joystick, receive the status of the optoelectronic payload, and receive and process the video images collected in real time by the optoelectronic payload. The airborne optoelectronic payload control method specifically includes: The control joystick outputs attitude control commands for the photoelectric load; The attitude sensor acquires the attitude information of the control joystick; The airborne task manager collects attitude control commands from the optoelectronic payload and attitude information from the control joystick through the link system; Based on the attitude information of the control joystick, determine whether the angular velocity of the control joystick is greater than the first angular velocity threshold, or whether the acceleration is greater than the first acceleration threshold. If so, obtain the expected attitude information of the photoelectric load at the next moment based on the attitude control command of the photoelectric load and the attitude information of the control joystick. The attitude change of the optoelectronic payload at the next moment is obtained based on the attitude control command of the optoelectronic payload and the expected attitude information of the optoelectronic payload at the next moment; specifically based on... Obtain the change in the attitude of the photoelectric load Δθ at the next moment; The airborne task manager controls the attitude change of the optoelectronic payload based on the current actual position and attitude of the optoelectronic payload and the attitude change of the optoelectronic payload at the next moment.

2. The airborne optoelectronic load control method according to claim 1, characterized in that, The link system includes a ground station system and an airborne telemetry and control system. The ground station system is used to upload the attitude control commands of the optoelectronic payload and the attitude information of the control joystick to the airborne mission manager. The airborne telemetry and control system is used to transmit the video images and the status of the optoelectronic payload collected in real time and transmitted by the airborne mission manager to the payload controller.

3. The airborne optoelectronic payload control method according to claim 1, characterized in that, The airborne task manager includes a CPU processing module and a video processing module. The CPU processing module is used to control the photoelectric payload according to the attitude control commands of the photoelectric payload and the attitude information of the control joystick. The video processing module is used to receive the status of the photoelectric payload and to receive and process the video images acquired by the photoelectric payload in real time.

4. The airborne optoelectronic payload control method according to claim 3, characterized in that, The video processing module processes video images acquired in real time by the photoelectric payload, including compressing the video images and adjusting the encoding structure.

5. The airborne optoelectronic payload control method according to claim 4, characterized in that, The video processing module uses the H.264 / AVC standard to compress the video images acquired in real time by the photoelectric payload.

6. A method for video processing of an airborne optoelectronic payload, characterized in that, The airborne optoelectronic payload video processing method implements the airborne optoelectronic payload video processing through the airborne task manager used in the airborne optoelectronic payload control method as described in any one of claims 1 to 5.

7. The airborne optoelectronic payload video processing method according to claim 6, characterized in that, The airborne optoelectronic payload video processing method specifically includes: The airborne task manager compresses the video images acquired in real time by the optoelectronic payload; Based on the attitude changes of the photoelectric load and the attitude information of the control joystick collected by the attitude sensor, large scene switching detection is performed on the compressed video image to determine whether a large scene switching has occurred. If not, video encoding is performed directly to complete video processing. If so, it is determined whether the current frame of the compressed video after the large scene switching is an I-frame. If the current frame of the compressed video after the large scene switching is an I-frame, video encoding is performed directly to complete video processing. If the current frame of the compressed video after the large scene switching is not an I-frame, the current frame is set to an I-frame, the encoding bitrate is reduced, and video encoding is performed to complete video processing.

Images