Unmanned aerial vehicle control method and system based on monocular camera

Abstract

The invention discloses an unmanned aerial vehicle control method and system based on a monocular camera. The unmanned aerial vehicle control system based on the monocular camera comprises an IMU unit and the monocular camera. The monocular camera comprises an image sensor and a microprocessor, the image sensor outputs an optical signal, the microprocessor converts the optical signal into an electric signal, and the electric signal also carries IMU data. The unmanned aerial vehicle control method and system based on the monocular camera, disclosed by the invention, have the beneficial effects that in a time delay allowable range, the second-level control information is preferentially adopted to replace the first-level control information so as to control controlled objects such as an engine and paddles; on the contrary, if the time delay allowable range is exceeded, the first-level control information immediately replaces the second-level control information. Wherein the second-level control information fully reflects the interaction characteristic between the airborne equipment of the unmanned aerial vehicle and the ground control station within the time delay allowable range.

Description

Technical Field

[0001] The present invention belongs to the field of interaction between unmanned aerial vehicles and ground control stations, and in particular relates to a unmanned aerial vehicle control method and system based on a monocular camera. Background Art

[0002] The invention patent application with publication number CN105138126A and subject name is shooting control method and device of drone, and electronic device, and its IPC classification number is G06F3 / 01. Its technical solution discloses "determining whether a predefined mode activation condition is satisfied; starting a preset shooting mode corresponding to the predefined mode activation condition to apply to the shooting operation. Optionally, the predefined mode activation condition includes at least one of the following: receiving a preset instruction issued by a user, detecting that the user is in a preset posture, and detecting that the drone or the user is in a preset environmental state."

[0003] It can be seen that the above invention patent application has disclosed one of the technical solutions of the drone shooting control method. However, the above technical solution focuses on defining the corresponding mode activation conditions for each preset shooting mode, so that the drone can accurately know the preset shooting mode to be activated when a certain mode activation condition is met, ensuring that the user's actual shooting needs are met; it does not further solve the problems of flexible selection of drone autonomous control mode (first-level control information) and ground control station remote control mode (second-level control information), and needs to be further improved. Summary of the invention

[0004] In view of the status of the prior art, the present invention overcomes the above defects and provides a drone control method and system based on a monocular camera.

[0005] The present invention adopts the following technical solution, a UAV control system based on a monocular camera, including an IMU unit, and also includes:

[0006] Monocular camera: The monocular camera includes an image sensor and a microprocessor. The image sensor outputs a light signal, which is converted into an electrical signal by the microprocessor. The electrical signal also carries IMU data.

[0007] A flight control unit, the flight control unit is electrically connected to the monocular camera and the IMU unit through an airborne line, the flight control unit is wirelessly connected to the ground control station through a data link, the flight control unit generates first-level control information including imaging data, coordinate data and IMU data in real time, the flight control unit transmits the first-level control information back to the ground control station through the data link, the ground control station optimizes the first-level control information into second-level control information according to a preset ground control station processing method, and the ground control station uploads the second-level control information to the flight control unit through the data link; the flight control unit controls the controlled object according to a preset decision method;

[0008] The preset decision method is specifically implemented as follows:

[0009] The timing starts when the flight control unit generates the first-level control information. Within the specified delay interval, the flight control unit determines whether to obtain the second-level control information. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information; otherwise, the flight control unit controls the controlled object according to the first-level control information.

[0010] The present invention adopts the following technical solution, and the UAV control method based on a monocular camera includes the following steps:

[0011] The image sensor outputs a light signal, which is converted into an electrical signal by a microprocessor. The electrical signal also carries the IMU data.

[0012] The flight control unit generates the first-level control information including imaging data, coordinate data and IMU data in real time. The flight control unit transmits the first-level control information back to the ground control station through the data link. The ground control station optimizes the first-level control information into the second-level control information according to the preset ground control station processing method. The ground control station uploads the second-level control information to the flight control unit through the data link. The flight control unit controls the controlled object according to the preset decision method.

[0013] The preset decision method is specifically implemented as follows:

[0014] The timing starts when the flight control unit generates the first-level control information. Within the specified delay interval, the flight control unit determines whether to obtain the second-level control information. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information; otherwise, the flight control unit controls the controlled object according to the first-level control information.

[0015] As a preferred technical solution of the above technical solution, the microprocessor includes a preprocessing module and a data processing module, wherein:

[0016] The preprocessing module is electrically connected to the image sensor, and the preprocessing module autonomously or controlledly adjusts the optical signal parameters of the image sensor;

[0017] The data processing module is electrically connected to the flight control unit. The data processing module processes the optical signal autonomously or under control to output an electrical signal, and carries the real-time imaging data and coordinate data on the electrical signal.

[0018] As a preferred technical solution of the above technical solution, the preprocessing module is controlled to adjust the optical signal parameters of the image sensor, which is specifically implemented as follows:

[0019] The flight control unit autonomously controls the pre-processing module to adjust the optical signal parameters of the image sensor; or:

[0020] The ground control station remotely controls the pre-processing module via the flight control unit to adjust the optical signal parameters of the image sensor.

[0021] As a preferred technical solution of the above technical solution, the data processing module is controlled to process the optical signal, which is specifically implemented as follows;

[0022] The flight control unit autonomously controls the data processing module to process the optical signal; or:

[0023] The ground control station remotely controls the data processing module via the flight control unit to process the optical signal.

[0024] As a preferred technical solution of the above technical solutions, the preset ground control station processing method is specifically implemented as the following steps: one or more steps of machine vision processing steps, three-dimensional data conversion steps, IMU data correction steps and PID data control steps are comprehensively applied according to actual conditions.

[0025] The monocular camera-based drone control method and system disclosed in the present invention have the following beneficial effects:

[0026] 1. The UAV control method based on monocular camera reduces the dependence on other expensive sensors, reduces the overall cost of UAV, and is conducive to large-scale application. At the same time, the monocular camera can collect image data in real time, provide rich visual information for UAV, help to realize the perception and understanding of the surrounding environment, thereby improving the autonomous control ability of UAV.

[0027] 2. Traditional drone control methods often rely on a variety of expensive sensors, such as GPS, LiDAR, depth cameras, etc. These sensors are not only costly, but also large and heavy, which puts a heavy burden on the flight performance of drones. In contrast, monocular cameras have the advantages of small size, light weight and low cost. They use low-pixel monocular cameras and onboard drone flight control units to identify obstacle locations in complex environments without requiring prior knowledge of obstacles, thus improving obstacle avoidance capabilities.

[0028] 3. The UAV control method based on monocular camera has the advantages of low cost, small size, light weight, strong real-time performance, etc., and can adapt to more application scenarios. Within the allowable delay range, the second-level control information is preferentially used to replace the first-level control information to control the controlled objects such as the engine and blades. On the contrary, if the delay exceeds the allowable range, the first-level control information immediately replaces the second-level control information. Among them, within the allowable delay range, the second-level control information fully reflects the interactive characteristics between the UAV onboard equipment and the ground control station. DETAILED DESCRIPTION

[0029] The present invention discloses a monocular camera-based unmanned aerial vehicle control method and system. The specific implementation of the present invention is further described below in conjunction with a preferred embodiment (Example 1).

[0030] Example 1.

[0031] Those skilled in the art should note that the “flight control unit”, “UAV flight control unit”, “airborne UAV flight control unit”, etc. that may be involved in various embodiments of the present invention are the same concept and are no longer distinguished.

[0032] Those skilled in the art should note that the “data link”, “data transmission link”, etc. that may be involved in various embodiments of the present invention are the same concept and are no longer distinguished.

[0033] Those skilled in the art should note that the “monocular camera”, “monocular camera head”, etc. that may be involved in various embodiments of the present invention are the same concept and are no longer distinguished.

[0034] Preferably, the monocular camera-based drone control system includes an IMU unit and further includes:

[0035] Monocular camera: The monocular camera includes an image sensor and a microprocessor. The image sensor outputs an optical signal, and the microprocessor converts the optical signal into an electrical signal to further output real-time imaging data and coordinate data carried on the electrical signal. The electrical signal also carries IMU data, thereby reflecting the three-dimensional spatial position of the drone. The monocular camera has the advantages of small size, light weight, and low cost, and is easy to install on the drone, providing the drone with rich visual information.

[0036] A flight control unit, the flight control unit is electrically connected to the monocular camera (microprocessor) and the IMU unit through an airborne line, the flight control unit is wirelessly connected to the ground control station through a data link, the flight control unit obtains real-time imaging data and coordinate data output by the monocular camera, the flight control unit obtains real-time IMU data output by the IMU unit, the flight control unit generates first-level control information including imaging data, coordinate data and IMU data in real time, the flight control unit transmits the first-level control information back to the ground control station through the data link (as an example, the data volume of the first-level control information is significantly smaller than the data volume of the real-time imaging data and coordinate data output by the monocular camera), the ground control station obtains the first-level control information, the ground control station optimizes the first-level control information into second-level control information according to a preset ground control station processing method, the ground control station uploads the second-level control information to the flight control unit through the data link, and the flight control unit obtains the second-level control information; the flight control unit controls the controlled object according to a preset decision method;

[0037] The preset decision method is specifically implemented as follows:

[0038] The timing starts from when the flight control unit generates the first-level control information. The flight control unit determines whether to obtain the second-level control information within the specified delay interval. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information. Otherwise (the second-level control information is not obtained within the specified delay interval), the flight control unit controls the controlled object according to the first-level control information; so that the flight control unit preferentially uses the second-level control information to replace the first-level control information within the allowed delay range to control the controlled objects such as the engine and the blades. Otherwise (exceeding the maximum value of the specified delay interval), the first-level control information immediately replaces the second-level control information to control the controlled objects such as the engine and the blades.

[0039] The microprocessor includes a preprocessing module and a data processing module, wherein:

[0040] The preprocessing module is electrically connected to the image sensor, and the preprocessing module autonomously or controlledly adjusts the optical signal parameters of the image sensor (for example, exposure time, frame rate, etc.), so that the image sensor outputs a finely adjusted optical signal to the preprocessing module, thereby exploring the imaging capability of the image sensor;

[0041] The data processing module is electrically connected to the flight control unit. The data processing module processes the optical signal autonomously or under control to output an electrical signal, and carries the real-time imaging data and coordinate data on the electrical signal.

[0042] Among them, as one of the parallel technical solutions of the preprocessing module, the preprocessing module is controlled to adjust the optical signal parameters of the image sensor, which is specifically implemented as follows:

[0043] The flight control unit autonomously controls the pre-processing module to adjust the optical signal parameters of the image sensor; or:

[0044] The ground control station remotely controls the pre-processing module via the flight control unit to adjust the optical signal parameters of the image sensor.

[0045] Among them, as one of the parallel technical solutions of the data processing module, the data processing module is controlled to process the optical signal, which is specifically implemented as follows;

[0046] The flight control unit autonomously controls the data processing module to process the optical signal; or:

[0047] The ground control station remotely controls the data processing module via the flight control unit to process the optical signal.

[0048] It is worth mentioning that, unlike existing small terminal devices such as handheld drone remote controls, ground control stations are larger in scale, have more equipment, and have stronger processing capabilities, and can realize functions that handheld drone remote controls cannot carry or realize.

[0049] Among them, the preset ground control station processing method is specifically implemented as the following steps: according to the actual comprehensive application of one or more steps of machine vision processing steps, three-dimensional data conversion steps, IMU data correction steps and PID data control steps, the first-level control information is optimized into the second-level control information.

[0050] The specific technical solution of the machine vision processing steps of this embodiment is described below.

[0051] Image acquisition and preprocessing: restore and archive the imaging data and coordinate data carried by the first-level control information, perform denoising on the imaging data, remove noise interference caused by environmental factors or the camera itself, and improve image quality. For example, algorithms such as median filtering and Gaussian filtering can be used to remove salt and pepper noise and Gaussian noise in the image.

[0052] Image enhancement: By adjusting image contrast, brightness and other parameters, the features in the image are made more obvious, facilitating subsequent feature extraction and analysis.

[0053] Feature extraction: Use edge detection algorithms such as the Canny edge detection algorithm to extract edge information of objects in the image.

[0054] Extract feature points from the image: such as SIFT (Scale-Invariant Feature Transform) feature points or SURF (Speeded Up Robust Features) feature points. These feature points are scale-invariant and rotation-invariant, and can stably identify objects under different viewing angles and lighting conditions.

[0055] Target recognition and tracking: Based on the extracted feature points, specific targets are identified (which can be special targets manually designated by the operator of the ground control station and not entered into the UAV's flight control unit in advance), the identified targets are tracked, and the position and motion status of the specific targets in the image are determined in real time.

[0056] Depth estimation: Combine the archived imaging data and coordinate data to estimate the depth information of specific targets in the image (in contrast, the onboard storage space of the drone is limited and cannot archive long-term imaging data and coordinate data, which can give full play to the scale advantage of the ground control station), thereby generating three-dimensional data of the drone's surrounding environment, which can be used as an effective input for the three-dimensional data conversion step.

[0057] The specific technical solution of the three-dimensional data conversion step of this embodiment is described below.

[0058] Establish the drone coordinate system: Take the drone's starting point or a specific midpoint as the origin, and extend to set up the X, Y, and Z coordinate axes.

[0059] Establish a ground coordinate system: Define the ground coordinate system according to the actual application scenario so as to convert it with the drone coordinate system.

[0060] Set up the monocular camera imaging model: describe the relationship between the position of the target in three-dimensional space and its projection on the camera image plane. Usually, the pinhole camera model is used, which involves the intrinsic and extrinsic matrix of the monocular camera. The intrinsic matrix describes the internal parameters of the monocular camera, such as focal length, pixel size, etc.; the extrinsic matrix describes the position and posture of the monocular camera in the world coordinate system.

[0061] 3D coordinate conversion: Based on the imaging data and coordinate data output by the monocular camera, combined with the monocular camera imaging model, the 2D coordinates on the image plane are converted into coordinates in 3D space. Using the known camera parameters and the position information of the target in the image, the coordinates of the target in 3D space are solved through geometric calculation or mathematical model.

[0062] Coordinate conversion and alignment: Convert the three-dimensional coordinates in the drone coordinate system into the coordinates in the ground coordinate system to facilitate data interaction and fusion with other systems.

[0063] Output conversion results: The results of the three-dimensional data conversion are sorted and formatted, and output as data that can be used by the flight control unit and ground control station, which can be used as an effective input for the IMU data correction step.

[0064] The specific technical solution of the IMU data correction step of this embodiment is described below.

[0065] Data acquisition and fusion: Restore and archive the IMU data carried by the first-level control information, especially the accelerometer data and gyroscope data. The accelerometer measures the acceleration of the drone, and the gyroscope measures the angular velocity of the drone.

[0066] Data fusion: Fuse IMU data with coordinate data to improve the accuracy of drone position and attitude estimation. Kalman filtering, extended Kalman filtering and other algorithms can be used for data fusion.

[0067] Error analysis and modeling: Analyze the sources of error in IMU data, including sensor noise, bias, temperature drift, etc. Establish an error model to describe the characteristics and variation of the error. For example, a statistical model can be used to model sensor noise, and a polynomial model can be used to model temperature drift.

[0068] Error correction algorithm: Based on the error model, design the corresponding error correction algorithm. Common error correction methods include zero bias correction, proportional factor correction, temperature compensation, etc. Zero bias correction is used to eliminate the initial bias of the sensor, proportional factor correction is used to adjust the measurement ratio of the sensor, and temperature compensation is used to compensate for the error caused by temperature changes.

[0069] Dynamic correction and update: During the flight of the drone, the changes in IMU data are monitored in real time, and dynamic correction is performed according to the error correction algorithm. The error model and correction parameters are continuously updated to adapt to different flight states and environmental conditions. For example, an adaptive filtering algorithm can be used to automatically adjust the correction parameters according to real-time data.

[0070] Output corrected data: The corrected IMU data is sorted and formatted, and output as data that can be used by the flight control unit and ground control station, which can be used as an effective input for the PID data control step.

[0071] Preferably, the drone control method based on a monocular camera comprises the following steps:

[0072] The image sensor outputs a light signal, which is converted into an electrical signal by a microprocessor to further output real-time imaging data and coordinate data carried on the electrical signal. The electrical signal also carries IMU data, thereby reflecting the three-dimensional spatial position of the drone. The monocular camera has the advantages of small size, light weight, and low cost, and is easy to install on the drone, providing the drone with rich visual information.

[0073] The flight control unit obtains real-time imaging data and coordinate data output by the monocular camera, the flight control unit obtains real-time IMU data output by the IMU unit, the flight control unit generates first-level control information including imaging data, coordinate data and IMU data in real time, the flight control unit transmits the first-level control information back to the ground control station through the data link (for example, the data volume of the first-level control information is significantly smaller than the data volume of the real-time imaging data and coordinate data output by the monocular camera), the ground control station obtains the first-level control information, the ground control station optimizes the first-level control information into second-level control information according to a preset ground control station processing method, the ground control station uploads the second-level control information to the flight control unit through the data link, and the flight control unit obtains the second-level control information; the flight control unit controls the controlled object according to a preset decision method;

[0074] The preset decision method is specifically implemented as follows:

[0075] The timing starts from when the flight control unit generates the first-level control information. The flight control unit determines whether to obtain the second-level control information within the specified delay interval. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information. Otherwise (the second-level control information is not obtained within the specified delay interval), the flight control unit controls the controlled object according to the first-level control information; so that the flight control unit preferentially uses the second-level control information to replace the first-level control information within the allowed delay range to control the controlled objects such as the engine and the blades. Otherwise (exceeding the maximum value of the specified delay interval), the first-level control information immediately replaces the second-level control information to control the controlled objects such as the engine and the blades.

[0076] It is worth mentioning that the technical solution of the UAV control system based on a monocular camera disclosed in this embodiment is also applicable to the UAV control method based on a monocular camera. The technical solution of the UAV control method based on a monocular camera will not be repeated.

[0077] The following describes the overall concept of the monocular camera-based drone control method and system disclosed in this embodiment.

[0078] Specifically, the control method and system of the unmanned aerial vehicle based on the monocular camera collects data based on the monocular camera and converts and outputs imaging data and coordinate data to reflect the three-dimensional spatial position of the unmanned aerial vehicle. The imaging data and coordinate data are transmitted to the unmanned aerial vehicle flight control unit in real time through the airborne line to output the first-level control information. The first-level control information is transmitted back to the ground control station through the data link. The ground control station optimizes the first-level control information into the second-level control information according to the machine vision technology, the three-dimensional data conversion technology, the IMU data correction technology and the PID data control technology. The second-level control information is uploaded to the unmanned aerial vehicle flight control unit through the data link. Within the preset time limit, the unmanned aerial vehicle flight control unit controls the controlled objects such as the engine according to the uploaded second-level control information. When the preset time limit is exceeded, the unmanned aerial vehicle flight control unit directly controls the controlled objects such as the engine according to the first-level control information. The first-level control information focuses on the necessary and urgent, with a small amount of data and strong real-time performance, and can respond quickly in an emergency; the second-level control information focuses on completeness and comprehensiveness, with a large amount of data and complex calculations, providing more accurate control instructions for the unmanned aerial vehicle and improving the stability and accuracy of the unmanned aerial vehicle flight.

[0079] It is worth mentioning that the technical features such as the PID data control steps involved in the patent application of the present invention should be regarded as the prior art. The specific structure, working principle and possible control method and spatial layout method of these technical features can be selected according to the conventional selection in the field, and should not be regarded as the inventive point of the patent of the present invention. The patent of the present invention will not be further elaborated.

[0080] For those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A UAV control system based on a monocular camera, characterized in that: Including the IMU unit, it also includes: Monocular camera: The monocular camera includes an image sensor and a microprocessor. The image sensor outputs a light signal, which is converted into an electrical signal by the microprocessor. The electrical signal also carries IMU data. A flight control unit, the flight control unit is electrically connected to the monocular camera and the IMU unit through an airborne line, the flight control unit is wirelessly connected to the ground control station through a data link, the flight control unit generates first-level control information including imaging data, coordinate data and IMU data in real time, the flight control unit transmits the first-level control information back to the ground control station through the data link, the ground control station optimizes the first-level control information into second-level control information according to a preset ground control station processing method, and the ground control station uploads the second-level control information to the flight control unit through the data link; the flight control unit controls the controlled object according to a preset decision method; The preset decision method is specifically implemented as follows: The timing starts when the flight control unit generates the first-level control information. Within the specified delay interval, the flight control unit determines whether to obtain the second-level control information. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information; otherwise, the flight control unit controls the controlled object according to the first-level control information.

2. The monocular camera-based unmanned aerial vehicle control system according to claim 1, characterized in that: The microprocessor includes a preprocessing module and a data processing module, wherein: The preprocessing module is electrically connected to the image sensor, and the preprocessing module autonomously or controlledly adjusts the optical signal parameters of the image sensor; The data processing module is electrically connected to the flight control unit. The data processing module processes the optical signal autonomously or under control to output an electrical signal, and carries the real-time imaging data and coordinate data on the electrical signal.

3. The monocular camera-based unmanned aerial vehicle control system according to claim 2, characterized in that: The pre-processing module is controlled to adjust the optical signal parameters of the image sensor, which is specifically implemented as follows: The flight control unit autonomously controls the pre-processing module to adjust the optical signal parameters of the image sensor; or: The ground control station remotely controls the pre-processing module via the flight control unit to adjust the optical signal parameters of the image sensor.

4. The monocular camera-based unmanned aerial vehicle control system according to claim 2, characterized in that: The data processing module is controlled to process the optical signal, which is specifically implemented as follows: The flight control unit autonomously controls the data processing module to process the optical signal; or: The ground control station remotely controls the data processing module via the flight control unit to process the optical signal.

5. The monocular camera-based unmanned aerial vehicle control system according to claim 1, characterized in that: The preset ground control station processing method is specifically implemented as the following steps: one or more steps of a machine vision processing step, a three-dimensional data conversion step, an IMU data correction step and a PID data control step are comprehensively applied according to actual conditions.

6. A UAV control method based on a monocular camera, characterized in that: The following steps are involved: The image sensor outputs a light signal, which is converted into an electrical signal by a microprocessor. The electrical signal also carries the IMU data. The flight control unit generates the first-level control information including imaging data, coordinate data and IMU data in real time. The flight control unit transmits the first-level control information back to the ground control station through the data link. The ground control station optimizes the first-level control information into the second-level control information according to the preset ground control station processing method. The ground control station uploads the second-level control information to the flight control unit through the data link. The flight control unit controls the controlled object according to the preset decision method. The preset decision method is specifically implemented as follows: The timing starts when the flight control unit generates the first-level control information. Within the specified delay interval, the flight control unit determines whether to obtain the second-level control information. If the second-level control information is obtained within the specified delay interval, the flight control unit controls the controlled object according to the second-level control information; otherwise, the flight control unit controls the controlled object according to the first-level control information.

7. The monocular camera-based drone control method according to claim 6, characterized in that: The microprocessor includes a preprocessing module and a data processing module, wherein: The preprocessing module is electrically connected to the image sensor, and the preprocessing module autonomously or controlledly adjusts the optical signal parameters of the image sensor; The data processing module is electrically connected to the flight control unit. The data processing module processes the optical signal autonomously or under control to output an electrical signal, and carries the real-time imaging data and coordinate data on the electrical signal.

8. The monocular camera-based drone control method according to claim 7, characterized in that: The pre-processing module is controlled to adjust the optical signal parameters of the image sensor, which is specifically implemented as follows: The flight control unit autonomously controls the pre-processing module to adjust the optical signal parameters of the image sensor; or: The ground control station remotely controls the pre-processing module via the flight control unit to adjust the optical signal parameters of the image sensor.

9. The monocular camera-based drone control method according to claim 7, characterized in that: The data processing module is controlled to process the optical signal, which is specifically implemented as follows: The flight control unit autonomously controls the data processing module to process the optical signal; or: The ground control station remotely controls the data processing module via the flight control unit to process the optical signal.

10. The monocular camera-based drone control method according to claim 6, characterized in that: The preset ground control station processing method is specifically implemented as the following steps: one or more steps of a machine vision processing step, a three-dimensional data conversion step, an IMU data correction step and a PID data control step are comprehensively applied according to actual conditions.